Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “evaluations”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 811 records · Page 45

Advancing Diagnostic Model Evaluation to Better Understand Water Shortage Mechanisms in Institutionally Complex River Basins

Abstract Water resources systems models enable valuable inferences on consequential system stressors by representing both the geophysical processes determining the movement of water and the human elements distributing it to its various competing uses. This study contributes a diagnostic evaluation framework that pairs exploratory modeling with global sensitivity analysis to enhance our ability to make inferences on water scarcity vulnerabilities in institutionally complex river basins. Diagnostic evaluation of models representing institutionally complex river basins with many stakeholders poses significant challenges. First, it needs to exploit a large and diverse suite of simulations to capture important human‐natural system interactions as well as institutionally aware behavioral mechanisms. Second, it needs to have performance metrics that are consequential and draw on decision‐relevant model outputs that adequately capture the multisector concerns that emerge from diverse basin stakeholders. We demonstrate the proposed model diagnostic framework by evaluating how potential interactions between changing hydrologic conditions and human demands influence the frequencies and durations of water shortages of varying magnitudes experienced by hundreds of users in a subbasin of the Colorado River. We show that the dominant factors shaping these effects vary both across users and, for an individual user, across percentiles of shortage magnitude. These differences hold even for users sharing diversion locations, demand levels or water right seniority. Our findings underline the importance of detailed institutional representation for such basins, as institutions strongly shape how dominant factors of stakeholder vulnerabilities propagate through the complex network of users.

Hadjimichael, Antonia↗

Evaluating the Potential and Challenges of an Uncertainty Quantification Method for Long Short–Term Memory Models for Soil Moisture Predictions

Recently, recurrent deep networks have shown promise to harness newly available satellite–sensed data for long–term soil moisture projections. However, to be useful in forecasting, deep networks must also provide uncertainty estimates. Here we evaluated Monte Carlo dropout with an input–dependent data noise term (MCD+N), an efficient uncertainty estimation framework originally developed in computer vision, for hydrologic time series predictions. MCD+N simultaneously estimates a heteroscedastic input–dependent data noise term (a trained error model attributable to observational noise) and a network weight uncertainty term (attributable to insufficiently constrained model parameters). Although MCD+N has appealing features, many heuristic approximations were employed during its derivation, and rigorous evaluations and evidence of its asserted capability to detect dissimilarity were lacking. To address this, we provided an in–depth evaluation of the scheme's potential and limitations. We showed that for reproducing soil moisture dynamics recorded by the Soil Moisture Active Passive (SMAP) mission, MCD+N indeed gave a good estimate of predictive error, provided that we tuned a hyperparameter and used a representative training data set. The input–dependent term responded strongly to observational noise, while the model term clearly acted as a detector for physiographic dissimilarity from the training data, behaving as intended. However, when the training and test data were characteristically different, the input–dependent term could be misled, undermining its reliability. Additionally, due to the data–driven nature of the model, data noise also influences network weight uncertainty, and therefore the two uncertainty terms are correlated. Altogether, this approach has promise, but care is needed to interpret the results.

54 ENVIRONMENTAL SCIENCES↗

The DOE E3SM Model Version 2: Overview of the Physical Model and Initial Model Evaluation

Abstract This work documents version two of the Department of Energy's Energy Exascale Earth System Model (E3SM). E3SMv2 is a significant evolution from its predecessor E3SMv1, resulting in a model that is nearly twice as fast and with a simulated climate that is improved in many metrics. We describe the physical climate model in its lower horizontal resolution configuration consisting of 110 km atmosphere, 165 km land, 0.5° river routing model, and an ocean and sea ice with mesh spacing varying between 60 km in the mid‐latitudes and 30 km at the equator and poles. The model performance is evaluated with Coupled Model Intercomparison Project Phase 6 Diagnosis, Evaluation, and Characterization of Klima simulations augmented with historical simulations as well as simulations to evaluate impacts of different forcing agents. The simulated climate has many realistic features of the climate system, with notable improvements in clouds and precipitation compared to E3SMv1. E3SMv1 suffered from an excessively high equilibrium climate sensitivity (ECS) of 5.3 K. In E3SMv2, ECS is reduced to 4.0 K which is now within the plausible range based on a recent World Climate Research Program assessment. However, a number of important biases remain including a weak Atlantic Meridional Overturning Circulation, deficiencies in the characteristics and spectral distribution of tropical atmospheric variability, and a significant underestimation of the observed warming in the second half of the historical period. An analysis of single‐forcing simulations indicates that correcting the historical temperature bias would require a substantial reduction in the magnitude of the aerosol‐related forcing.

54 ENVIRONMENTAL SCIENCES↗

Evaluating Shallow Convection Parameterization Assumptions With a q t – w Quadrant Analysis

Abstract Uncertainties associated with the response of shallow clouds to global warming remain a great challenge for climate projection. Due to the small size of these clouds, parameterizations are required to represent them in both current‐ and next‐generation climate models. We present a quantitative evaluation of several important assumptions used in both mass‐flux and Assumed‐PDF Higher‐Order Closure parameterizations of shallow convection. We use large‐eddy simulations (LESs) of four different shallow convection regimes as benchmarks and apply a q t (total moisture)– w (vertical velocity) quadrant analysis technique to identify the “coherent structures” of moist and dry up/down‐drafts. The statistics of these coherent structures and the environment are then used to evaluate commonly used assumptions in mass‐flux parameterizations related to (a) the convective downdrafts and (b) the subplume variability of convective updrafts. For quantitative evaluation of the PDF closure used in AP‐HOCs, we perform offline calculations with the PDF closure used in the Cloud Layers Unified By Bi‐normals and the Simplified Higher‐Order Closure schemes by supplying the closure with statistical quantities directly calculated from the LESs. Then, the impact of parameterization assumptions embedded in the PDF closure in the same two categories is analyzed and understood through comparisons of the output from the closure with the statistics of the moist and dry up/down‐drafts from the LESs.

54 ENVIRONMENTAL SCIENCES↗

Evaluation of a New Approach for Entrainment and Detrainment Rate Estimation

Entrainment and detrainment rates (ε and δ) constitute the most critical free parameters in mass flux schemes commonly employed for cumulus parameterizations. Recently, Zhu et al. (2021) introduced a new approach that utilizes aircraft observations to simultaneously estimate ε and δ for cumulus clouds, overcoming the limitation of other observation-based approaches that solely yield ε without offering insights into δ. This study aims to comprehensively evaluate the reliability of this new approach. First, evaluation using an Explicit Mixing Parcel Model demonstrates the capability of the new approach to back-calculate predetermined ε and δ based on the physical properties before and after the entrainment mixing. Second, evaluation using large-eddy simulations illustrates that the new approach yields consistent ε and δ profiles compared to the traditional approach. Sensitivity tests indicate a weak sensitivity of the estimated δ with the new approach to the entrained air source. A decrease in the proportion of cloudy air in the assumed detrained air leads to a reduction in the estimated δ, while ε remains unaffected. Finally, the most appropriate assumptions for entrained and detrained air are discussed. Estimating ε for cumulus parameterizations involves acquiring ambient air more than 500 m away from the cloud edge as entrained air. Due to implicit mean field approximations in the traditional approach, determining the optimal assumption for detrained air properties proves challenging. Finally, this study confirms the reliability of the new approach in estimating ε and δ, providing confidence in its application to extensive observational data and advancement in parameterization.

54 ENVIRONMENTAL SCIENCES↗

Using an Isotope Enabled Mass Balance to Evaluate Existing Land Surface Models

Abstract Land surface models (LSMs) play a crucial role in elucidating water and carbon cycles by simulating processes such as plant transpiration and evaporation from bare soil, yet calibration often relies on comparing LSM outputs of landscape total evapotranspiration ( ET ) and discharge with measured bulk fluxes. Discrepancies in partitioning into component fluxes predicted by various LSMs have been noted, prompting the need for improved evaluation methods. Stable water isotopes serve as effective tracers of component hydrologic fluxes, but data and model integration challenges have hindered their widespread application. Leveraging National Ecological Observation Network measurements of water isotope ratios at 16 US sites over 3 years combined with LSM‐modeled fluxes, we employed an isotope‐enabled mass balance framework to simulate ET isotope values ( δET ) within three operational LSMs (Mosaic, Noah, and VIC) to evaluate their partitioning. Models simulating δET values consistent with observations were deemed more reflective of water cycling in these ecosystems. Mosaic exhibited the best overall performance (Kling‐Gupta Efficiency of 0.28). For both Mosaic and Noah there were robust correlations between bare soil evaporation fraction and error (negative) as well as transpiration fraction and error (positive). We found the point at which errors are smallest ( x ‐intercept of the multi‐site regression) is at a higher transpiration fraction than is currently specified in the models. Which means that transpiration fraction is underestimated on average. Stable isotope tracers offer an additional tool for model evaluation and identifying areas for improvement, potentially enhancing LSM simulations and our understanding of land‐surface hydrologic processes.

58 GEOSCIENCES↗

A single-component water-lean post-combustion CO 2 capture solvent with exceptionally low operational heat and total costs of capture – comprehensive experimental and theoretical evaluation

A comprehensive evaluation of a recently developed water-lean amine-based solvent, namely N-(2-ethoxyethyl)-3-morpholinopropan-1-amine (2-EEMPA), has been performed to analyze its post-combustion CO 2 capture performance. This evaluation comprises (1) fundamental characterization of the solvent–CO 2 interaction using vapor–liquid equilibria, kinetics and viscosity measurements; (2) process characterization of the CO 2 capture performance as measured in a laboratory scale continuous flow system and via Aspen Plus® simulation using a flue gas simulant; as well as (3) a full techno economic analysis of the capture process at industrial scale with corresponding projections of critical metrics. This paper summarizes the many parts of this comprehensive evaluation and shows how the various parts come together to empower validated conclusions about its process performance. Notably, it is projected that this solvent can operate at a regeneration heat rate of 2.0 GJ per tonne CO 2 for post-combustion capture, and at a total cost of capture of $50.6/tonne CO 2 . Overall, with further process optimization significant reductions in the capture cost are predicted.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

Database work for the new cross section standards evaluation

An effort is now underway to produce a new evaluation of the neutron standards. It is important to maintain experimental programs to increase the quality and extend the database for the neutron cross section standards in order to improve evaluations of them that will be used to convert cross section measurements made relative to those standards. Measurements have been made for most of the standard cross sections since the last evaluation of the standards. The improved database includes the cross sections for the H(n,n), 6Li(n,t), 10B(n,αγ), 10B(n,α), C(n,n), Au(n,γ), 235U(n,f) and 238U(n,f) standard reactions and ratios among them. The database also includes the 238U(n,γ) and 239Pu(n,f) cross sections in addition to the standard cross sections. Those data were included since there are many ratio measurements of those cross sections with the standards and absolute data are available for them.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗

Fission Product Yield Modeling and Evaluation

Although independent and cumulative fission product yields have been a part of evaluated libraries for decades, there have been few updates over the years. The fission product yield sub-library in the ENDF/B-VIII.0 library is still largely based on the evaluation of England and Rider from the mid-90’s, with only more recent updates to the energy dependence of 239 Pu below 2 MeV and fixes to isomeric states and missing fission products. Over the past several years, there have been a wealth of new measurements of independent and cumulative fission product yields, particularly those with short half-lives, and there have been significant improvements in the modeling of prompt and delayed fission observables. Here, we describe recent progress in the improvement of fission product yield calculations, using the BeoH code and the underlying Hauser Fesh-bach Fission Fragment Decay (HF 3 D) model, developed at Los Alamos National Laboratory. We will describe our recent calculations for consistent prompt and delayed fission observables for major and minor actinides, including new work investigating isomeric ratios. We will detail the ongoing evaluation process for energy-dependent fission product yields from thermal up to 20 MeV incident neutron energy and some validation work that has been performed for these new fission product yield calculations. Additionally, we will discuss future perspectives of this work, highlighting the need for additional data.

38 RADIATION CHEMISTRY, RADIOCHEMISTRY, AND NUCLEA↗

Stochastic evaluation of fourth-order many-body perturbation energies

A scalable, stochastic algorithm evaluating the fourth-order many-body perturbation (MP4) correction to energy is proposed. Three hundred Goldstone diagrams representing the MP4 correction are computer generated and then converted into algebraic formulas expressed in terms of Green’s functions in real space and imaginary time. They are evaluated by the direct (i.e., non-Markov, non-Metropolis) Monte Carlo (MC) integration accelerated by the redundant-walker and control-variate algorithms. The resulting MC-MP4 method is efficiently parallelized and is shown to display O(n 5.3 ) size-dependence of cost, which is nearly two ranks lower than the O(n 7 ) dependence of the deterministic MP4 algorithm. Furthermore, it evaluates the MP4/aug-cc-pVDZ energy for benzene, naphthalene, phenanthrene, and corannulene with the statistical uncertainty of 10 mE h (1.1% of the total basis-set correlation energy), 38 mE h (2.6%), 110 mE h (5.5%), and 280 mE h (9.0%), respectively, after about 10 9 MC steps.

74 ATOMIC AND MOLECULAR PHYSICS↗

Expert evaluation of LLM world models: A high-T c superconductivity case study

Large Language Models (LLMs) show great promise as a powerful tool for scientific literature exploration. However, their effectiveness in providing scientifically accurate and comprehensive answers to complex questions within specialized domains remains an active area of research. Using the field of high-temperature cuprates as an exemplar, we evaluate the ability of LLM systems to understand the literature at the level of an expert. We construct an expert-curated database of 1,726 scientific papers that covers the history of the field, and a set of 67 expert-formulated questions that probe deep understanding of the literature. We then evaluate six different LLM-based systems for answering these questions, including both commercially available closed models and a custom retrieval-augmented generation (RAG) system capable of retrieving images alongside text. Experts then evaluate the answers of these systems against a rubric that assesses balanced perspectives, factual comprehensiveness, succinctness, and evidentiary support. Among the six systems, two using RAG on curated literature outperformed existing closed models across key metrics, particularly in providing comprehensive and well-supported answers. We discuss promising aspects of LLM performances as well as critical short-comings of all the models. The set of expert-formulated questions and the rubric will be valuable for assessing expert level performance of LLM based reasoning systems.

36 MATERIALS SCIENCE↗

Dynamic PRA Methods to Evaluate the Impact on Accident Progression of Accident Tolerant Fuels

Accident tolerant fuels (ATFs) are new nuclear fuels developed in response to the accident at the Fukushima power station in March 2011. The goal of ATFs is to withstand accident scenarios through better performance compared to currently employed fuels (e.g., small-scale hydrogen generation). This paper targets a method for evaluating and comparing ATF performance from a probabilistic risk assessment (PRA) perspective by employing a newly developed combination of event trees and dynamic PRA methods. Compared to classical PRA methods based on event trees and fault trees, dynamic PRA can evaluate with higher resolution the safety impacts of physics dynamics and the timing/sequencing of events on the accident progression without the need to introduce overly conservative modeling assumptions and success criteria. In this paper, we analyze the impact on the accident progression of three different cladding configurations for two initiating events [a large break loss-of-coolant accident (LB-LOCA) and a station blackout (SBO)] by employing dynamic PRA methods. The goal is to compare the safety performance of ATFs (FeCrAl and Cr-coated cladding) and the currently employed Zr-based clad fuel. We employ two different strategies. The first focuses on the identification of success criteria discrepancies between the accident sequences generated by the classical PRA model and the set of simulation runs generated by dynamic PRA using ATF. The second one, on the other hand, directly uses dynamic PRA to evaluate the impact of timing of events (e.g., recovery actions) on accident progression. In conclusion, by applying these methods to the LB-LOCA and SBO initiating events, we show how dynamic PRA methods can provide analysts with detailed and quantitative information on the safety impact of ATFs.

97 - MATHEMATICS AND COMPUTING↗

Preliminary Evaluation of Joint Electricity-Hydrogen Concept of Operations

An initial thermal power dispatch (TPD) concept of operations was evaluated that couples a nuclear power plant to a nearby hydrogen production plant. GSE Systems’ generic pressurized water reactor full-scope simulator was modified with a TPD model comprised of a thermal power extraction and delivery system. A prototype human-system interface (HSI) was developed to interact with the TPD model and allow participants to execute the basic operating scenarios for normal operations. Four retired operators performed the evaluation, and due to COVID-19 travel restrictions, the original in-person experimental design was restructured to support a remote participator evaluation using a web meeting platform. Data from operator feedback, observations from the research team, and quantitative survey responses revealed that the initial TPD concept of operations is feasible. The operators were comfortable with the engineered system and HSI and could manage it without adverse impacts to reactor power, plant safety, or equipment. Findings are discussed in terms of both the TPD system design and HSI performance.

human factors↗

Design and evaluation of an open-source, conformable skin-cooling system for body magnetic resonance guided focused ultrasound treatments

Magnetic resonance guided focused ultrasound (MRgFUS) treatment of tumors uses inter-sonication delays to allow heat to dissipate from the skin and other near-field tissues. Despite inter-sonication delays, treatment of tumors close to the skin risks skin burns. This work has designed and evaluated an open-source, conformable, skin-cooling system for body MRgFUS treatments to reduce skin burns and enable ablation closer to the skin. A MR-compatible skin cooling system is described that features a conformable skin-cooling pad assembly with feedback control allowing continuous flow and pressure maintenance during the procedure. System performance was evaluated with hydrophone, phantom and in vivo porcine studies. Sonications were performed 10 and 5 mm from the skin surface under both control and forced convective skin-cooling conditions. 3D MR temperature imaging was acquired in real time and the accumulated thermal dose volume was measured. Gross analysis of the skin post-sonication was further performed. Device conformability was demonstrated at several body locations. Hydrophone studies demonstrated no beam aberration, but a 5–12% reduction of the peak pressure due to the presence of the skin-cooling pad assembly in the acoustic near field. Phantom evaluation demonstrated there is no MR temperature imaging precision reduction or any other artifacts present due to the coolant flow during MRgFUS sonication. The porcine studies demonstrated skin burns were reduced in size or eliminated when compared to the control condition. An open-source design of an MRgFUS active skin cooling system demonstrates device conformability with a reduction of skin burns while ablating superficial tissues.

60 APPLIED LIFE SCIENCES↗

An Evaluation of Advanced Tools for Distributed Wind Turbine Performance Estimation

We evaluate various classes of distributed wind turbine performance tools across two sites in the United States. The class of tools ranges from the simple mass conservation model to the coupled Reynolds-averaged Navier-Stokes model, all initiated by the WIND Toolkit data set. The resource estimation at the site is evaluated against measured data at the mast location. Taking a sample 100-kW wind turbine and constant losses, we evaluate Openwind, Continuum, and WindNinja tools and document annual energy production (AEP) and time-series statistics associated with the performance estimation of the wind turbine. Using a methodology that is consistent and unbiased across the three options currently available in the industry, we elaborate results at the two sample locations and discuss the probable sources of discrepancy in the AEP estimates. Two main sources of the discrepancy come from the input WIND Toolkit data and the spatial modeling techniques of the tools to capture atmospheric physics. The discussion includes additional values that these tools may bring into the energy assessment process to enhance the owners' confidence over the distributed wind power systems.

17 WIND ENERGY↗

Evaluating algorithmic bias on biomarker classification of breast cancer pathology reports

Objectives: This work evaluated algorithmic bias in biomarkers classification using electronic pathology reports from female breast cancer cases. Bias was assessed across 5 subgroups: cancer registry, race, Hispanic ethnicity, age at diagnosis, and socioeconomic status. Materials and Methods: We utilized 594 875 electronic pathology reports from 178 121 tumors diagnosed in Kentucky, Louisiana, New Jersey, New Mexico, Seattle, and Utah to train 2 deep-learning algorithms to classify breast cancer patients using their biomarkers test results. We used balanced error rate (BER), demographic parity (DP), equalized odds (EOD), and equal opportunity (EOP) to assess bias. Results: We found differences in predictive accuracy between registries, with the highest accuracy in the registry that contributed the most data (Seattle Registry, BER ratios for all registries >1.25). BER showed no significant algorithmic bias in extracting biomarkers (estrogen receptor, progesterone receptor, human epidermal growth factor receptor 2) for race, Hispanic ethnicity, age at diagnosis, or socioeconomic subgroups (BER ratio <1.25). DP, EOD, and EOP all showed insignificant results. Discussion: We observed significant differences in BER by registry, but no significant bias using the DP, EOD, and EOP metrics for socio-demographic or racial categories. This highlights the importance of employing a diverse set of metrics for a comprehensive evaluation of model fairness. Conclusion: A thorough evaluation of algorithmic biases that may affect equality in clinical care is a critical step before deploying algorithms in the real world. We found little evidence of algorithmic bias in our biomarker classification tool. Artificial intelligence tools to expedite information extraction from clinical records could accelerate clinical trial matching and improve care.

60 APPLIED LIFE SCIENCES↗

Empirical evaluation of all unique Cas9 protospacers in E. coli reveal widespread functionality and rules for gRNA design

The Cas9 nuclease has become central to modern methods and technologies in synthetic biology, largely due to the ease with which it can be targeted to specific DNA loci via guide RNAs (gRNAs). Reports vary widely on the actual specificity of this targeting, with some studies observing 60% of gRNAs possessing no activity against the genome, yet an assumption persists within the E. coli community that inactive gRNAs are rare. To resolve these contradictions, we evaluated the activity of 463 000 unique gRNAs in the E. coli K12 MG1655 genome. We show that the overwhelming majority (at least 93%) of unique gRNAs are functional while only 0.3% are nonfunctional. These nonfunctional gRNAs exhibit strong spacer self-interaction, which can either be excluded using a simple design rule or “repaired” during library design. Finally, this work provides the greater microbial synthetic biology community both a set of nearly half a million empirically evaluated E. coli gRNAs as well as a thoroughly evaluated experimental procedure, complete with appropriate controls for Cas9 activity, for conducting Cas9 assays in E. coli specifically and bacteria more generally. Lastly, we have produced a webapp to allow users to easily browse and extract gRNA sequences from the E. coli genome, which can be accessed at https://grna.ornl.gov.

Kammerdiener, Elise K. [Oak Ridge National Laborat↗

Fast Fourier transform evaluation of the Fresnel integral for gravitational-wave lensing

Gravitational waves (GWs) exhibit wave-optics effects when their wavelength is comparable to the scale of the gravitational lens. This may occur in lensing from galactic subhalos in GWs emitted by binary black-hole mergers and is gaining interest as a novel probe of dark matter. Predictions for observables in these cases ultimately rely on evaluating a Fresnel integral that quantifies the effect of lensing on the amplitude of a GW at a given frequency. However, numerical evaluation of this Fresnel integral is tricky, and several algorithms and publicly available codes that implement it have been developed. Here, we show that the dependence of this integral on the lens position can be written as a two-dimensional Fourier transform. Modern FFT techniques then enable rapid evaluation at all-sky positions simultaneously for general lenses without symmetry. Vectorization of FFT routines allows for derivatives with respect to model parameters to be obtained with only incremental additional computational cost. If the lens is axisymmetric, further speedups can be achieved with recently developed techniques for nonuniform fast Hankel transforms. To demonstrate, we make available Fresnel Integral Optimization with Nonuniform Transforms (fiona), an efficient and accurate code that is significantly faster than current methods for dense source grids, reaching 2 orders of magnitude speedups for ∼10 6 GW-emitting points. As part of FIONA , we developed code that provides vectorized nonuniform fast Hankel transforms that may have other uses (e.g., calculation of cosmological two-point correlation functions) beyond those considered here.

dark matter↗