Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “evaluations”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 469 records · Page 26

Data-Informed Evaluation Framework for Integrated Energy Systems: Insights from Power, Process Heat, and Hydrogen Production Applications

The multi-criteria decision analysis (MCDA) framework provides a systematic evaluation of the diverse preferences and performance metrics associated with alternative solutions. This approach is advantageous over a single-criterion methodology, which are only valid under conditions that assume ceteris paribus or an "apples-to-apples" comparison. However, selecting suitable technologies for integrated energy systems (IES) can be likened to an "apples-to-oranges" comparison, given the heterogeneous factors at stake. These factors include economics and performance parameters, geological compatibility, and environmental impacts. Consequently, past research has often employed a mixture of qualitative and quantitative criteria tailored to the specific interests of each study. While the method proves effective in handling the intricate interplay of criteria, the resulting rankings and scores can vary from study to study. This inconsistency is introduced from the use of subjectively defined thresholds and weights. As a result, decision-makers frequently find it challenging to establish clear connections between specific criteria and the resulting scores, as the transformation of criteria into ordinal scores results in a substantial loss of information. To address this challenge, we introduce a data-informed IES evaluation framework that offers comprehensive, interpretable, and traceable evaluations backed by quantifiable rationale. First, we identified key IES evaluation criteria from a decade of literature, focusing on relevant IES applications in power, process heat, and hydrogen production. We leveraged state-of-the-art cost estimates from the Idaho National Laboratory (INL) and technical data from 78 reactor designs from the International Atomic Energy Agency (IAEA) and the Organization for Economic Co-operation and Development - Nuclear Energy Agency (OECD-NEA). Lastly, we established thresholds by analyzing the mean, variance, root mean square, and slope of values across alternatives, categorizing the preferences of decision-makers into distinct utility functions, such as linear, saturating, exponential, and stepwise. Our approach yielded two main outcomes: (1) it provided consistent assessments across different stakeholder groups and (2) it visualized uncertainties in the decision-making context via comprehensive sensitivity analysis. To demonstrate the impact of our framework, we conducted case studies on 6 reactor designs (AP1000, NuScale, BWRX-300, Xe-100, eVinci, iMSR) for the three applications. Our data-driven framework proved to be highly effective in addressing heterogenous uncertainties faced by varied decision-makers? preferences and IES applications, as well as cost and technical estimates of advanced reactors.

29 ENERGY PLANNING, POLICY, AND ECONOMY↗

WASTE TREATMENT PLANT (WTP) LIQUID EFFLUENT TREATABILITY EVALUATION (SEPTEMBER 2004)

A forecast of the radioactive, dangerous liquid effluents expected to be produced by the Waste Treatment Plant (WTP) was provided by Bechtel National, Inc. (BNI 2004). The forecast represents the liquid effluents generated from the processing of Tank Farm waste through the end-of-mission for the WTP. The WTP forecast is provided in the Appendices. The WTP liquid effluents will be stored, treated, and disposed of in the Liquid Effluent Retention Facility (LERF) and the Effluent Treatment Facility (ETF). Both facilities are located in the 200 East Area and are operated by Fluor Hanford, Inc. (FH) for the US. Department of Energy (DOE). The treatability of the WTP liquid effluents in the LERF/ETF was evaluated. The evaluation was conducted by comparing the forecast to the LERF/ETF treatability envelope (Aromi 1997), which provides information on the items which determine if a liquid effluent is acceptable for receipt and treatment at the LERF/ETF. The format of the evaluation corresponds directly to the outline of the treatability envelope document. Except where noted, the maximum annual average concentrations over the range of the 27 year forecast was evaluated against the treatability envelope. This is an acceptable approach because the volume capacity in the LERF Basin will equalize the minimum and maximum peaks. Background information on the LERF/ETF design basis is provided in the treatability envelope document.

12 MANAGEMENT OF RADIOACTIVE AND NON-RADIOACTIVE W↗

Experimental Performance Evaluation of a Hyper-Branched Polymer Electrolyte for Rechargeable Li-Air Batteries

A hyper-branched polymer (HBP) electrolyte is synthesized for rechargeable lithium-air (Li-air) battery cell and experimentally evaluated its performance in actual battery cell environment. Several real-world battery cells were fabricated with synthesized HBP electrolyte, pure lithium metal as anode and an oxygen permeable air cathode to evaluate reproducibility of the rechargeable Li-air battery cell. The effect of various conditions such as various HBP based electrolytes, discharge current -0.1~0.5 mA, cathode preparation processes and carbon contents on the battery cell performance were experimentally evaluated using the fabricated battery cells under dry air condition. Detailed HBP electrolyte synthesis procedures and experimental performance evaluation of Li-air battery cell for various conditions are presented. The experimental results showed that different conditions and processes significantly affect the Li-air battery performance. Upon taking into account the effect of different conditions and processes, optimized HBP electrolyte materials, cathode process and conditions were determined. Several Li-air battery cells were fabricated with optimized conditions and optimized battery cell materials to determine the reproducibility and performance consistency. Experimental results showed that over 55–65 h of discharge occurred over 2.5 V terminal cell voltage with all three optimized Li-air battery cells. It implied that the optimized Li-air battery cells were reproducible and were able to hold charge over 2.5 V for more than 2 days. Experimental results of the Li-air battery cell with further refined optimized materials revealed that the battery cell can discharge more than 10 days (i.e., more than 250 h) at or above 2.0 V. The experimental results also showed that the Li-air battery discharge time got shorter as the discharge-charge cycle increases due to increase in internal resistances of battery cell materials. The experimental results confirmed that the lithium-air battery cell can be reproduced without loss of performance and can hold charge more than 10 days at or over 2.0 V. The investigation results obtained may usher a pathway to manufacture a long-life rechargeable Li-air battery cell in the near future.

25 ENERGY STORAGE↗

A global model–measurement evaluation of particle light scattering coefficients at elevated relative humidity

The uptake of water by atmospheric aerosols has a pronounced effect on particle light scattering properties, which in turn are strongly dependent on the ambient relative humidity (RH). Earth system models need to account for the aerosol water uptake and its influence on light scattering in order to properly capture the overall radiative effects of aerosols. Here we present a comprehensive model–measurement evaluation of the particle light scattering enhancement factor f(RH), defined as the particle light scattering coefficient at elevated RH (here set to 85 %) divided by its dry value. The comparison uses simulations from 10 Earth system models and a global dataset of surface-based in situ measurements. In general, we find a large diversity in the magnitude of predicted f(RH) amongst the different models, which can not be explained by the site types. Based on our evaluation of sea salt scattering enhancement and simulated organic mass fraction, there is a strong indication that differences in the model parameterizations of hygroscopicity and model chemistry are driving at least some of the observed diversity in simulated f(RH). Additionally, a key point is that defining dry conditions is difficult from an observational point of view and, depending on the aerosol, may influence the measured f(RH). The definition of dry also impacts our model evaluation, because several models exhibit significant water uptake between RH = 0 % and 40 %. The multisite average ratio between model outputs and measurements is 1.64 when RH = 0 % is assumed as the model dry RH and 1.16 when RH = 40 % is the model dry RH value. The overestimation by the models is believed to originate from the hygroscopicity parameterizations at the lower RH range which may not implement all phenomena taking place (i.e., not fully dried particles and hysteresis effects). This will be particularly relevant when a location is dominated by a deliquescent aerosol such as sea salt. Our results emphasize the need to consider the measurement conditions in such comparisons and recognize that measurements referred to as dry may not be dry in model terms. Recommendations for future model–measurement evaluation and model improvements are provided.

54 ENVIRONMENTAL SCIENCES↗

CARETS: A prototype regional environmental information system. Volume 12: User evaluation of experimental land use maps and related products from the central Atlantic test site

The user interaction and evaluation phase of the USGS/NASA Central Atlantic Regional Ecological Test Site was designed to obtain the input of local, regional, State, and Federal agency users of land-resource information into the development, of a regional information system; to provide users with assistance and data resulting from CARETS research; and to have user. organizations evaluate to what extent the CARETS products meet their needs. The evaluation of CARETS land use and related products revealed that most user agencies interviewed, at all governmental levels, require more detailed data than that provided by the CARETS project. Few agencies found utility in the generalized ERTS Level I land--use maps. Level II data, though reported valuable by several users, was generally considered of secondary utility by most users. The products considered most useful by users at all levels were the high--altitude color-infrared photographs and the USGS orthophotoquads. Recommendations resulting from the evaluation reflect the need to establish a flexible and reliable system for providing more detailed raw and processed land-resource information as well as the need to improve the methods of making information available to users.

Land-use planning↗

An evaluation of the use of new Doppler methods for detecting longitudinal function abnormalities in a pacing-induced heart failure model

BACKGROUND: Doppler tissue echocardiography and color M-mode Doppler flow propagation velocity have proven useful in evaluating cross-sections of patients with left ventricular (LV) dysfunction, but experience with serial changes is limited. Purpose and methods: We tested their use by evaluating the temporal changes of LV function in a pacing-induced congestive heart failure model. Rapid ventricular pacing was initiated and maintained in 20 dogs for 4 weeks. Echocardiography was performed at baseline and weekly during brief pacing cessation. RESULTS: With rapid pacing, LV volume significantly increased and ejection fraction (57%-28%), stroke volume (37-18 mL), and mitral annulus systolic velocity (16.1-6.6 cm/s) by Doppler tissue echocardiography significantly decreased, with ejection fraction and mitral annulus systolic velocity closely correlated (r = 0.706, P <.0001). In contrast to the mitral inflow velocities, mitral annulus early diastolic velocity decreased steadily (12.3-7.3 cm/s) resulting in a dramatic decrease in mitral annulus early/late (1.22-0.57) diastolic velocity with no tendency toward pseudonormalization. The color M-mode Doppler flow propagation velocity also showed significant steady decrease (57-24 cm/s) throughout the pacing period. Multiple regression analysis chose mitral annulus systolic velocity (r = 0.895, P <.0001) and propagation velocity (r = 0.782, P <.0001) for the most important factor predicting LV systolic and diastolic function, respectively. CONCLUSIONS: Doppler tissue echocardiography and color M-mode Doppler flow could evaluate the serial deterioration in LV dysfunction throughout the pacing period. These were more useful in quantifying progressive LV dysfunction than conventional ehocardiographic techniques, and were probably relatively independent of preload. These techniques could be suitable for longitudinal evaluation in addition to the cross-sectional study.

Evaluation Studies↗

Identification of Scenarios for System Interface Design Evaluation: CAST SE-210 Output 2 Report 5 of 6

This report is one in a series of reports describing research to develop enhanced approaches to design and evaluation of flight deck interaction. The research was conducted to support the Commercial Aviation Safety Team (CAST) response to incidents and accidents caused by failure of the flight crew to maintain aircraft attitude or energy state awareness. This report partially responds to Safety Enhancement (SE) 210.2 (CAST, 2014) by describing development of operational scenarios for evaluation of flight crew interaction. Evaluations of human performance with an interface rely on asking humans to perform operational tasks in an appropriate operational context. The report includes the major sources of material for developing operational scenarios—which include system state, operational tasks, operational context, and safety events—and a method for developing an appropriate set of operational scenarios for interface evaluation.

commercial aviation↗

Evaluating Differences Among Crop Models in Simulating Soybean in-Season Growth

Crop models are useful tools for simulating agricultural systems that require continued model development and testing to increase their robustness and improve how they describe our current understanding of processes. Coordinated and “blind” evaluation of multiple models using same protocols and experimental datasets provides unique opportunities to further improve models and enhance their reliability. For soybean [Glycine max (L.) Merr.], there has been limited coordinated multi-model evaluations for the simulation of in-season plant growth dynamics. We evaluated ten dynamic soybean crop models for their simulation of in-season plant growth using data from five experiments conducted in Argentina, Brazil, France, and USA. We evaluated models after a Blind (using only phenology data) and a Full calibration (with in-season and end-of-season variables). Calibration reduced model uncertainty by reducing standard bias for the simulation of in-season variables (biomass, leaf, pod, and stem weights, and leaf area index, LAI). However, we found that most models had difficulty in reproducing leaf growth dynamics, with normalized root mean squared error (nRMSE) of 56% for leaf weight and 43% for LAI (across locations and models after Full calibration). Models with different levels of complexity and experience were capable of simulating final seed yield at maturity with reasonable accuracy (nRMSE of 8–31% after Full calibration). However, the nRMSE for pod weight (of 17–64% after Full calibration) was two-fold larger than that of seed yield. Moreover, the models differed in how they simulated timing from sowing to beginning seed growth (47–93 days) and effective seed filling period (18–54 days), owing to model structural differences in defining the reproductive developmental stages. Overall, we identified the following processes that can benefit from further model improvement: leaf expansion and senescence, reproductive phenology, and partitioning to reproductive growth. Simulation of pod wall tissue and individual seed cohorts is another aspect that many models currently lack. Model improvement can benefit from high-temporal resolution experimental datasets that concurrently account for phenology, plant growth, and partitioning. Further, we recommend collecting reproductive phenology in the field consistent with actual dry matter allocation to organs in the models and collecting multiple observations of seed and pod weight to aid model improvement for simulation of seed growth and yield formation.

Agricultural Model Intercomparison and Improvement↗

Building a Clean Energy Workforce: An Evaluation Framework for State Energy Office Workforce Programs

This fact sheet is a summary of the workforce evaluation framework that was first proposed in the white paper, An Evaluation Framework for State Energy Offices' Energy Efficiency and Clean Energy Workforce Program. This evaluation framework was developed to guide State Energy Offices (SEOs) in assessing and enhancing the workforce development programs the manage, fund, or partner on. By following this step-by-step framework, SEOs can evaluate program components, identify areas for improvement, and optimize program outcomes for greater impact.

clean energy↗

Data-Informed Evaluation Framework for Integrated Energy Systems: Insights from Power, Process Heat, and Hydrogen Production Applications

The multi-criteria decision analysis (MCDA) framework provides a systematic evaluation of the diverse preferences and performance metrics associated with alternative solutions. This approach is advantageous over a single-criterion methodology, which are only valid under conditions that assume ceteris paribus or an "apples-to-apples" comparison. However, selecting suitable technologies for integrated energy systems (IES) can be likened to an "apples-to-oranges" comparison, given the heterogeneous factors at stake. These factors include economics and performance parameters, geological compatibility, and environmental impacts. Consequently, past research has often employed a mixture of qualitative and quantitative criteria tailored to the specific interests of each study. While the method proves effective in handling the intricate interplay of criteria, the resulting rankings and scores can vary from study to study. This inconsistency is introduced from the use of subjectively defined thresholds and weights. As a result, decision-makers frequently find it challenging to establish clear connections between specific criteria and the resulting scores, as the transformation of criteria into ordinal scores results in a substantial loss of information. To address this challenge, we introduce a data-informed IES evaluation framework that offers comprehensive, interpretable, and traceable evaluations backed by quantifiable rationale. First, we identified key IES evaluation criteria from a decade of literature, focusing on relevant IES applications in power, process heat, and hydrogen production. We leveraged state-of-the-art cost estimates from the Idaho National Laboratory (INL) and technical data from 78 reactor designs from the International Atomic Energy Agency (IAEA) and the Organization for Economic Co-operation and Development - Nuclear Energy Agency (OECD-NEA). Lastly, we established thresholds by analyzing the mean, variance, root mean square, and slope of values across alternatives, categorizing the preferences of decision-makers into distinct utility functions, such as linear, saturating, exponential, and stepwise. Our approach yielded two main outcomes: (1) it provided consistent assessments across different stakeholder groups and (2) it visualized uncertainties in the decision-making context via comprehensive sensitivity analysis. To demonstrate the impact of our framework, we conducted case studies on 6 reactor designs (AP1000, NuScale, BWRX-300, Xe-100, eVinci, iMSR) for the three applications. Our data-driven framework proved to be highly effective in addressing heterogenous uncertainties faced by varied decision-makers? preferences and IES applications, as well as cost and technical estimates of advanced reactors.

29 ENERGY PLANNING, POLICY, AND ECONOMY↗

Chapter 8 - Testing and Evaluation Methods for Protection Systems

In this chapter, we aim to present different approaches used formally and informally for the evaluation of protection devices and schemes. This chapter starts with presenting background on generalization of protection device components, followed by the resources necessary to evaluate the protection system. Further, this chapter presents the different approaches used in protection evaluation with their advantages and disadvantages. Finally, this chapter closes by presenting background on technology readiness level metrics. Technology readiness level metric is borrowed from the space industry for the purpose of comparing the maturity of the different protection technologies and using the appropriate evaluation methods from the list presented here.

power systems↗

Development of a Benchmark Eddy Flux Evapotranspiration Dataset for Evaluation of Satellite-Driven Evapotranspiration Models Over the CONUS

A large sample of ground-based evapotranspiration (ET) measurements made in the United States, primarily from eddy covariance systems, were post-processed to produce a benchmark ET dataset. The dataset was produced primarily to support the intercomparison and evaluation of the OpenET satellite-based remote sensing ET (RSET) models and could also be used to evaluate ET data from other models and approaches. OpenET is a web-based service that makes field-delineated and pixel-level ET estimates from well-established RSET models readily available to water managers, agricultural producers, and the public. The benchmark dataset is composed of flux and meteorological data from a variety of providers covering native vegetation and agricultural settings. Flux footprint predictions were developed for each station and included static flux footprints developed based on average wind direction and speed, as well as dynamic hourly footprints that were generated with a physically based model of upwind source area. The two footprint prediction methods were rigorously compared to evaluate their relative spatial coverage. Data from all sources were post-processed in a consistent and reproducible manner including data handling, gap-filling, temporal aggregation, and energy balance closure correction. The resulting dataset included 243,048 daily and 5,284 monthly ET values from 194 stations, with all data falling between 1995 and 2021. We assessed average daily energy imbalance using 172 flux sites with a total of 193,021 days of data, finding that overall turbulent fluxes were understated by about 12% on average relative to available energy. Multiple linear regression analyses indicated that daily average latent energy flux may be typically understated slightly more than sensible heat flux. This dataset was developed to provide a consistent reference to support evaluation of RSET data being developed for a wide range of applications related to water accounting and water resources management at field to watershed scales.

54 ENVIRONMENTAL SCIENCES↗

Potential of deep learning methods to enhance satellite-based monitoring of nuclear power plants focusing on remote operation evaluations

The anticipated expansion of the nuclear industry and the deployment of new nuclear reactors (200 + GW of new nuclear capacity by 2050) require the development of monitoring systems that align with safety and security concerns, providing enhanced evaluation capabilities. A remote monitoring system using satellites and deep learning techniques was evaluated for its ability to detect anomalies and capture various features of nuclear reactors independently of the conditions on the ground. Satellite images of current operational and under-construction nuclear power plants were collected from Google Earth Pro as a surrogate database. Subsequently, five datasets were created from the collected images. Transfer learning technique was used for several classification tasks utilizing VGG16, ResNet50V2, Xception, DenseNet121, and MobileNetV2 pre-trained models. In the first task, the capability of the monitoring system to detect abnormal conditions or processes in a nuclear power plant was investigated. In the second task, the ability to capture operational features remotely was examined. As an example, for the purposes of this study, these features included classifying reactors based on type, power range, or onsite condition. Several evaluation metrics were used to compare the performance of the pre-trained models and the overall monitoring system. Here, the evaluation results demonstrated that deep learning techniques and pre-trained models applied to satellite images have the potential to facilitate further and expand capabilities in monitoring systems to assess plant operation details.

22 - GENERAL STUDIES OF NUCLEAR REACTORS↗

Energy-saving potential evaluation for primary schools with occupant-centric controls

Recent studies demonstrated that there is significant energy-saving potential for primary schools, which heating, ventilation, and air-conditioning (HVAC) systems with occupant-centric control (OCC) is an excellent candidate to save energy. However, such an energy impact has yet to be evaluated for different climate zones as well as different energy code versions, but is critical for technology transfer and deployment. Therefore, this paper conducts comprehensive evaluation on the energy-saving potentials for the primary schools with two advanced OCC strategies: presence-based and counting-based. In this work, ninety-six building energy models with stochastic behavior of occupants are developed and simulated, which consist of two building energy code versions, 16 climate zones, and baseline case (without OCC) and two advanced cases (with OCC). To evaluate the energy saving potentials for OCC, primary schools in the U.S. are used as an example. The results show that there is significant energy-saving potential for primary schools by considering OCC strategies, especially the counting-based case. The energy-saving potential is up to 10.2% for presence-based OCC and 12.41% for counting-based OCC. Furthermore, both climate and code version have a significant impact on energy savings from OCC strategies. The energy-saving potentials vary from 1.79% to 12.41% for different climates and code versions. This evaluation can also contribute to quantify the nationwide energy saving potential for other countries.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗

Performance evaluation of finned tube heat exchanger using curved wavy delta winglet vortex generators with circular perforations

Vortex generation is recognized as an effective passive approach to improve the heat transfer rate in fin and tube heat exchangers (FTHEs). The current study proposed innovative designs of curved wavy delta winglet vortex generators (CWDWVGs), both without and with circular perforations, to improve the heat transfer efficiency of FTHEs. There is potential to increase heat transfer performance further through various CWDWVG designs. Here, this study explores seven unique CWDWVG configurations, from 1-wave to 7-wave. A 3-D computational numerical model is utilized to evaluate the Thermo-hydraulic performance of FTHEs fitted with these different CWDWVG configurations across Reynolds numbers from 400 to 2000. This comparative analysis of the Thermo-hydraulic performance of FTHEs featuring four parallel circular tube layouts assesses configurations both with and without vortex generators (VGs) and various hole configurations. The evaluation of Thermo-hydraulic performance involves different parameters, including the London area goodness factor (LAGF), Colburn factor (j), friction factor (f), pressure drop (?P), and Nusselt number (Nu). Results demonstrate that the various CWDWVG configurations and the number of holes in them substantially affect the efficiency, as evaluated by the dimensionless Performance Evaluation Criteria (PEC). Notably, the 7-wave CWDWVGs surpassed other configurations, and integrating circular punched perforations further improved the thermal-hydraulic performance of FTHE. Specifically, the 7-wave CWDWVGs without holes demonstrated superior performance over other configurations, showing a significant increase in Nusselt number by 75.18% and 85.16% at Reynolds numbers of 2000 and 400, respectively, alongside an increase in pressure drop by 216.38% to 224.96%. Meanwhile, the 7-wave CWDWVGs with eight holes, in comparison to those without holes, exhibited a Nusselt number increase of 0.85%, a pressure drop decrease of 7.31%, and a reduction in the friction factor by 5.82%.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗

Failure evaluation of neutron-irradiated SiC/SiC composites by underwater acoustic emission

Here, understanding the statistical properties of mechanical properties of non-irradiated and neutron-irradiated SiC/SiC composites is essential for component design. This study aims to evaluate the detailed damage accumulation behavior of composites focusing on two fracture parameters: proportional limit stress (PLS) and acoustic emission (AE) onset stress. The developmental underwater AE technique, which is benefit in non-contact in-situ failure monitoring method during mechanical testing and in handle of the irradiated material, was first applied to evaluate damage accumulation behavior. Two types of chemical vapor infiltration SiC/SiC composites were used: one reinforced with Hi-Nicalon Type-S SiC fiber and one reinforced with Tyranno-SA3 SiC fiber in the form of satin-woven 2D fabrics with pyrolytic carbon interface. Neutron irradiation in the High Flux Isotope Reactor at Oak Ridge National Laboratory reached a fluence of 30 dpa at a temperature of 620–670°C. Four-point flexural tests were conducted to evaluate post-irradiation strength. Weibull statistics did not suggest marked degradation of composite strength. Detailed failure behavior evaluated by AE demonstrated no irradiation-induced change of the AE onset stress (i.e., crack initiation equivalent stress). Failure probability analysis suggests that increasing the reliability of composites (i.e., the Weibull modulus rather than strength itself) is essential to expanding the design margin and benefiting from a probabilistic design approach.

36 MATERIALS SCIENCE↗

Deep-Learning-Derived Evaluation Metrics Enable Effective Benchmarking of Computational Tools for Phosphopeptide Identification

Tandem mass spectrometry (MS/MS)-based phosphoproteomics is a powerful technology for global phosphorylation analysis. However, applying four computational pipelines to a typical mass spectrometry (MS)-based phosphoproteomic dataset from a human cancer study, we observed a large discrepancy among the reported phosphopeptide identification and phosphosite localization results, underscoring a critical need for benchmarking. While efforts have been made to compare performance of computational pipelines using data from synthetic phosphopeptides, evaluations involving real application data have been largely limited to comparing the numbers of phosphopeptide identifications due to the lack of appropriate evaluation metrics. We investigated three deep learning-derived features as potential evaluation metrics: phosphosite probability, Delta RT and spectral similarity. Predicted phosphosite probability is computed by MusiteDeep, which provides high accuracy as previously reported; Delta RT is defined as the absolute retention time (RT) difference between RTs observed and predicted by AutoRT; and spectral similarity is defined as the Pearson’s correlation coefficient between spectra observed and predicted by pDeep2. Using a synthetic peptide dataset, we found that both Delta RT and spectral similarity provided excellent discrimination between correct and incorrect peptide-spectrum matches (PSMs) both when incorrect PSMs involved wrong peptide sequences and even when incorrect PSMs were caused by only incorrect phosphosite localization. Based on these results, we used all the three deep learning-derived features as evaluation metrics to compare different computational pipelines on diverse set of phosphoproteomic datasets and showed their utility in benchmarking performance of the pipelines. The benchmark metrics demonstrated in this study will enable users to select computational pipelines and parameters for routine analysis of phosphoproteomics data and will offer guidance for developers to improve computational methods.

59 BASIC BIOLOGICAL SCIENCES↗

Evaluation of sequence tracking methods for Compton cameras based on CdZnTe arrays

In this study, the performance of sequence tracking methods for multiple interaction events in specific CdZnTe Compton imagers was evaluated using Monte Carlo simulations. The Compton imager consisted of a 6 × 6 array of virtual Frisch-grid CZT crystals, where the dimensions of each crystal were 5 × 5 × 12 mm 3 . The sequence tracking methods for another Compton imager that consists of two identical CZT crystals arrays were also evaluated. When 662 keV radiation was incident on the detectors, the percentages of the correct sequences determined by the simple comparison and deterministic methods for two sequential interactions were identical (~80 %), while those evaluated using the minimum squared difference method (55-59 %) and Three Compton method (45-55 %) for three sequential interactions, differed from each other. The reconstructed images of a 662 keV point source detected using single and double arrays were evaluated based on their angular resolution and signal-to-noise ratio, and the results showed that the double arrays outperformed single arrays.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗