Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “evaluation framework”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 541 records · Page 30
All sourcesNASA NTRSDOE OSTI
DOE OSTI2024

Evaluation of Stratocumulus Evolution Under Contrasting Temperature Advections in CESM2 Through a Lagrangian Framework

Abstract This study leveraged a Lagrangian framework to examine the evolution of stratocumulus clouds under cold and warm advections (CADV and WADV) in the Community Earth System Model 2 (CESM2) against observations. We found that CESM2 simulates a too rapid decline in low‐cloud fraction (LCF) and cloud liquid water path (CLWP) under CADV conditions, while it better aligns closely with observed LCF under WADV conditions but overestimates the increase in CLWP. Employing an explainable machine learning approach, we found that too rapid decreases in LCF and CLWP under CADV conditions are related to overestimated drying effects induced by sea surface temperature, whereas the substantial increase in CLWP under WADV conditions is associated with the overestimated moistening effects due to free‐tropospheric moisture and surface winds. Our findings suggest that overestimated drying effects of sea surface temperature on cloud properties might be one of crucial causes for the high equilibrium climate sensitivity in CESM2.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2021

Evaluation of KDP Estimation Algorithm Performance in Rain Using a Known-Truth Framework

Accurate estimation of specific differential phase ( K DP ) is necessary for rain rate estimation, attenuation correction, and hydrometeor classification algorithms. There are numerous published methods to process polarimetric radar observations of propagation differential phase shift (Φ DP ) and estimate K DP , but the corresponding K DP estimate uncertainty is unquantified. This study provides guidance on how commonly used K DP estimation algorithms perform in various environments. Here, we create numerous synthetic (“true”) K DP profiles, integrate over them to obtain “smoothed” Φ DP , and then add noise typical of S-band operational weather radar measurements. Each algorithm is applied to our noisy Φ DP profiles and compared to the true K DP profile such that the errors and uncertainty are quantified. The synthetic K DP profiles are Gaussian in shape, which allows systematic variations in their magnitude and width to determine how each algorithm performs in smooth, slowly changing K DP profiles, as well as steep profiles. Results demonstrate that algorithm performance is dependent on the Φ DP field received. These results are further supported by an error analysis of each algorithm for two more complicated synthetic K DP profiles. Some K DP algorithms allow users to change various tuning parameters; a subset of these tuning parameters is tested to provide guidance on how changing these parameters impacts algorithm performance. We then provide evidence that our known-truth framework provides insight into algorithm performance in observed data through two case studies.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2024

Validating automated resonance evaluation with synthetic data

The integrity and precision of nuclear data are crucial for a broad spectrum of applications, from national security and nuclear reactor design to medical diagnostics, where the associated uncertainties can significantly impact outcomes. A substantial portion of uncertainty in nuclear data originates from the subjective biases in the evaluation process, a crucial phase in the nuclear data production pipeline. Recent advancements indicate that automation of certain routines can mitigate these biases, thereby standardizing the evaluation process and enhancing reproducibility. This research aims to provide a methodology, framework, and metrics for the validation of automated nuclear data evaluation software leveraging high-quality synthetic data that closely mimic real experimental observables. An introduced error metric provides a scale and intuitive measure of the evaluation quality by quantifying the estimate’s accuracy and performance across the specified energy range. Synthetic data provides access to experimental observables and underlying resonance parameters, enabling comparison of different evaluations. The methodology is demonstrated using Ta-181 isotope data in the resolved resonance region. The Automated Resonance Identification Subroutine (ARIS), which operates without prior resonance information, was used to test and showcase the framework’s capabilities utilizing the proposed error metrics. The results demonstrate the effectiveness of the proposed approach and framework for optimizing software parameters and testing hypotheses through “what-if” controlled experiments, such as modifying assumptions about experimental conditions or average resonance parameters.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗
DOE OSTI2025

Valued peaks: Sustainable water allocation for small hydropower plants in an era of explicit ecological needs

Optimizing hydropower operations to balance economic profitability and support functioning ecosystem services is integral to river management policy. In this article, we propose a dynamic, constrained optimization framework for small hydropower plants (SHPs) to evaluate trade-offs between economic profitability and socio-ecological requirements. Specifically, we examine the balance between short-term losses in hydropower generation and the potential for compensatory benefits in the form of revenue from recreational ecosystem services, irrespective of the direct beneficiary. Our framework integrates a fish habitat model, a hydropower optimization model, and a recreational ecosystem service estimate to evaluate different environmental flow scenarios. The optimization process gives three outflow release scenarios, informed by previous streamflow realisations (dam inflow), and designed environmental flow constraints. The framework is applied and tested for the river Kuusinkijoki in North-eastern Finland, which is a habitat for migratory brown trout and grayling populations. We show that the revenue loss due to the environmental flow constraints arises through a reduction in revenue per generated energy unit and through a reduction in turbine efficiency. Additionally, the simulation results reveal that all the designed environmental flow constraints cannot be met simultaneously. Under the environmental flow scenario with both minimum flow and flow ramping rate constraints, the annual hydropower revenue decreases by 16.5 %. An annual increase of 8 % in recreational fishing visits offsets the revenue loss. In conclusion, the developed framework provides knowledge of the costs and benefits of hydropower environmental flow constraints and guides the prioritizing process of environmental measures.

13 HYDRO ENERGY↗
DOE OSTI2023

pystorms: A simulation sandbox for the development and evaluation of stormwater control algorithms

Advances in cyber–physical technologies have enabled real-time sensing and adaptive control of stormwater infrastructure. These smart stormwater systems allow for inexpensive, minimally-invasive stormwater control interventions in lieu of new construction. Importantly, however promising the area of smart stormwater control, there still remain barriers – for experts and novices alike – to access shared tools and methods for investigating, developing, and contributing to it. In an effort to make smart stormwater control research more methodical and accessible, we present pystorms, an open-source Python-based simulation sandbox that facilitates the quantitative evaluation and comparison of control strategies. pystorms consists of a collection of real world-inspired smart stormwater control scenarios on which any number of control strategies can be applied and tested via an accompanying Python programming interface and coupled stormwater simulator. pystorms provides a framework for the rigorous and efficient evaluation of smart stormwater control methodologies across diverse watersheds with only a few lines of code.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2021

A Hybrid System for Learning Classical Data in Quantum States

Deep neural network-powered artificial intelligence has rapidly changed our daily life with various applications. However, as one of the essential steps of deep neural networks, training a heavily-weighted network requires a tremendous amount of computing resources. Especially in the post Moore’s Law era, the limit of semiconductor fabrication technology has restricted the development of learning algorithms to cope with the increasing high intensity training data. Meanwhile, quantum computing has exhibited its significant potential in terms of speeding up the traditionally compute-intensive workloads. For example, Google illustrates quantum supremacy by completing a sampling calculation task in 200 seconds, which is otherwise impracticable on the world’s largest supercomputers. To this end, quantum-based learning becomes an area of interest, with the promising of a quantum speedup. In this paper, we propose GenQu, a hybrid and general-purpose quantum framework for learning classical data through quantum states. We evaluate GenQu with real datasets and conduct experiments on both simulations and real quantum computer IBM-Q. Our evaluation demonstrates that, comparing with classical solutions, the proposed models running on GenQu framework achieve similar accuracy with a much smaller number of qubits, while significantly reducing the parameter size by up to 95.86% and converging speedup by 66.67% faster.

Stein, Samuel A.↗
DOE OSTI2025

Optimal sizing of battery energy storage systems for peak shaving and demand response using a degradation-aware Bayesian Optimization-Mixed-Integer Linear Programming framework

The increasing integration of renewable energy and rising electricity demand highlight the importance of battery energy storage systems for peak shaving and demand response. Unlike prior approaches that overlook operational impacts on degradation, this study proposes a Bayesian Optimization–Mixed Integer Linear Programming framework for optimal battery energy storage system sizing. In this framework, Mixed Integer Linear Programming determines short-term scheduling while a calibrated electrochemical model iteratively evaluates degradation. The central hypothesis is that the framework can efficiently identify optimal sizes that yield realistic and economically robust outcomes. The method is tested across three scenarios: peak shaving, peak shaving with energy-reduction demand response, and peak shaving with power-reduction demand response. Results show that the framework converge to the optimum within 20 iterations out of 150 possible sizes. Under baseline conditions, the framework consistently selects the smallest feasible system, minimizing unnecessary degradation costs from oversized storage. Sensitivity analyses reveal that larger systems are favored as demand rates or incentives increase. Comparisons of demand response programs indicate that power-reduction demand response offers greater economic benefits than energy-reduction demand response, although demand savings from peak shaving remain the dominant contributor to overall performance. This study demonstrates that the proposed framework balances computational tractability with degradation fidelity, identifies critical economic thresholds for investment, and offers a practical, flexible tool to guide industrial stakeholders in cost-effective battery energy storage system deployment.

Batteries↗
DOE OSTI2025

Warm-phase microphysical evolution in large-eddy simulations of tropical cumulus congestus: evaluating drop size distribution evolution using polarimetry retrievals, in situ measurements, and a thermal-based framework

Owing to uncertainties in convective microphysics processes, improving parameterizations in Earth system models (ESMs) can benefit from observationally constrained cases suitable for scaling between cloud-resolving models and ESMs. We propose a benchmark large-eddy simulation (LES) cumulus congestus case study from the NASA Cloud, Aerosol, and Monsoon Processes Philippines Experiment (CAMP 2 Ex) for evaluating and improving ESMs in single-column model (SCM) mode. We seek observational constraints using novel polarimetric retrievals and in situ cloud microphysics measurements. Simulations using bulk and bin microphysics initialized with observed aerosol profiles are compared to cloud-top retrievals of cloud droplet effective radius (R eff ), effective variance (ν eff ), and number concentration (N d ) from the airborne Research Scanning Polarimeter (RSP). Both schemes reproduce characteristics of cloud-top N d and R eff that increase and decrease with altitude, respectively. Cloud-top N d is low-biased relative to RSP retrievals in both schemes, potentially due to limitations in both simulations and retrieval assumptions. Cloud-top R eff is low-biased in the bulk scheme but reasonably reproduced by the bin scheme. Profiles of N d and R eff are sensitive to the collision–coalescence process and the vertical variation in aerosol size distribution. Comparison of simulated and in situ droplet size distributions (DSDs) shows that, to first order, integrated moments are always sensitive to sizes < ~ 30 µm and can also be sensitive to larger sizes if the DSDs are sufficiently broad, with implications for the assumed maximum observed size retrieved by the RSP. The bin scheme captures the observed extended tail of the DSD, while the bulk scheme is unable to due to parametric constraints. Differences in expected relationships between in situ measurements of cloud cores and cloud-top retrievals by RSP demonstrate difficulty in constraining well the case presented herein. Finally, a thermal-tracking framework demonstrates that the dilution of N d throughout a thermal's lifetime is heavily determined by collision–coalescence and the height-varying aerosol distribution and that, in the absence of these, the impact of entrainment on diluting N d is largely offset by secondary activation. Implications for evaluating warm-phase convective microphysics schemes in ESMs and translating results for use on global, space-based polarimetry platforms are discussed.

Stanford, McKenna Wallace [Columbia Univ., New Yor↗
DOE OSTI2024

Market optimization and technoeconomic analysis of hydrogen-electricity coproduction systems

Decarbonization efforts across North America, Europe, and beyond rely on variable renewable energy sources such as wind and solar, as well as alternative fuels, such as hydrogen, to support the sustainable energy transition. These advancements have prompted a need for more flexibility in the electric grid to complement non-dispatchable energy sources and increased demand from electrification. Integrated energy systems are well suited to provide this flexibility, but conventional technoeconomic modeling paradigms neglect the time-varying dynamic nature of the grid and thus undervalue resource flexibility. In this work, we develop a computational optimization framework for dynamic market-based technoeconomic comparison of integrated energy systems that coproduce low-carbon electricity and hydrogen (e.g., solid oxide fuel cells, solid oxide electrolysis) against technologies that only produce electricity (e.g., natural gas combined cycle with carbon capture) or only produce hydrogen. Our framework starts with rigorous physics-based process models, built in the open-source Institute for the Design of Advanced Energy Systems (IDAES) modeling and optimization platform, for six energy process concepts. Using these rigorous models and a workflow to optimally design each technology, the framework is shown to be capable of evaluating new and emerging technologies in varying energy markets under a plethora of future scenarios (i.e., renewables penetration, carbon tax, etc.). Ultimately, our framework finds that solid oxide fuel cell-based coproduction systems achieve positive profits for 85% of the analyzed market scenarios. From these market optimization results, we use multivariate linear regression (R 2 values up to 0.99) to determine which electricity price statistics are most significant to predict the optimized annual profit of each system. The proposed framework provides a powerful tool for directly comparing flexible, multi-product energy process concepts to help discern optimal technology and integration options.

08 HYDROGEN↗
DOE OSTI2026

Revealing EDL-driven reduction mechanisms in binary, ternary, and quaternary fluorinated electrolytes via an integrated MD–DFT–ML framework

Accurately predicting solid electrolyte interphase (SEI) formation requires explicitly resolving the electric double layer (EDL) structure, which deviates significantly from that of the bulk electrolyte. Although an established molecular dynamics (MD) and Density Functional Theory (DFT) framework can model SEI formation by evaluating reduction reactions of local clusters in the EDL, it suffers from a combinatorial computational bottleneck. To overcome this limitation, we introduce a machine-learning-accelerated simulation workflow (MD–DFT–ML), integrating a gradient-boosted regression model trained on EDL composition data to efficiently predict reduction potentials. We apply this framework to seven fluorinated electrolytes comprising fluorinated anions, a fluorinated ester solvent, two types of diluent (ion-solvating ester vs. non-solvating ether), and an FEC additive. The analysis shows that the EDL selectively accumulates cation-binding species; consequently, the non–cation-binding ether diluent rarely enters the EDL and makes minimal contributions to SEI formation. DFT calculations on statistically representative EDL clusters provide reduction potentials and fluorine-release pathways, while the ML model, which substantially reduces the DFT workload, predicts cluster reduction energies with a mean absolute error of 0.1 eV. The combined MD–DFT–ML approach also quantifies contributions from different sources to LiF formation in the SEI. This methodology establishes a generalizable route for multiscale modeling electrolyte and interphase design for next-generation electrochemical energy-storage systems.

DFT-MD-ML workflow↗
DOE OSTI2024

Report on the Integration of Experimental and Modeling Data for Initial Equivalence Study of Microstructural Evolution in Irradiated LPBF 316SS

Advanced materials and manufacturing technologies are poised to improve the safety and design characteristics of nuclear technologies and meet US energy, environmental, and economic needs. In particular, metal additive manufacturing (AM) provides an opportunity to produce novel materials and component geometries, but their use is not without hurdles arising from the inherent microstructure variability that can result from the layer-by-layer build approach. Given the greater possible microstructure variability in AM materials—and the dearth of materials test reactors—it is impractical to rely solely on neutron irradiation studies to produce data for materials qualification for every possibility. This work within the Advanced Materials and Manufacturing Technologies (AMMT) Environmental Effects technical area contributes to the rapid qualification framework by developing a science-driven framework for the accelerated qualification of materials for nuclear environments. A key product of the Environmental Effects technical area of the AMMT program is the Licensing Approach with Ions and Neutrons (LAIN). This approach recognizes that whether using existing materials in new environments, newly developed materials tailored for these environments, or new manufacturing methods, the traditional decades-long approach for materials qualification does not facilitate rapid deployment. In FY 2023, the AMMT program presented a conceptual framework of specific steps to fulfill several technical challenges associated with qualifying materials for performance in radiation environments on an accelerated time frame informed by the state of the art in materials science and a review of the current regulatory landscape. The objective of this section of the Environmental Effects technical area is to critically evaluate and refine the proposed qualification framework presented under AMMT by integrating the research results of the neutron irradiations, the ion irradiations, and modeling efforts. These ongoing efforts span across Argonne National Laboratory (ANL), Idaho National Laboratory (INL), and Oak Ridge National Laboratory (ORNL) and are closely coordinated.

36 MATERIALS SCIENCE↗
DOE OSTI2020

Evaluation of a pair-based, joint-likelihood association approach for regional infrasound event identification

SUMMARY A Bayesian framework for the association of infrasonic detections is presented and evaluated for analysis at regional propagation scales. A pair-based, joint-likelihood association approach is developed that identifies events by computing the probability that individual detection pairs are attributable to a hypothetical common source and applying hierarchical clustering to identify events from the pair-based analysis. The framework is based on a Bayesian formulation introduced for infrasonic source localization and utilizes the propagation models developed for that application with modifications to improve the numerical efficiency of the analysis. Clustering analysis is completed using hierarchical analysis via weighted linkage for a non-Euclidean distance matrix defined by the negative log-joint-likelihood values. The method is evaluated using regional synthetic data with propagation distances of hundreds of kilometres in order to study the sensitivity of the method to uncertainties and errors in backazimuth and time of arrival. The method is found to be robust and stable for typical uncertainties, able to effectively distinguish noise detections within the data set from those in events, and can be made numerically efficient due to its ease of parallelization.

58 GEOSCIENCES↗
DOE OSTI2022

Deep learning categorization of infrasound array data

Here we develop a deep learning-based infrasonic detection and categorization methodology that uses convolutional neural networks with self-attention layers to identify stationary and non-stationary signals in infrasound array processing results. Using features extracted from the coherence and direction-of-arrival information from beamforming at different infrasound arrays, our model more reliably detects signals compared with raw waveform data. Using three infrasound stations maintained as part of the International Monitoring System, we construct an analyst-reviewed data set for model training and evaluation. We construct models using a 4-category framework, a generalized noise vs non-noise detection scheme, and a signal-of-interest (SOI) categorization framework that merges short duration stationary and non-stationary categories into a single SOI category. We evaluate these models using a combination of k-fold cross-validation, comparison with an existing “state-of-the-art” detector, and a transportability analysis. Although results are mixed in distinguishing stationary and non-stationary short duration signals, f-scores for the noise vs non-noise and SOI analyses are consistently above 0.96, implying that deep learning-based infrasonic categorization is a highly accurate means of identifying signals-of-interest in infrasonic data records.

47 OTHER INSTRUMENTATION↗
DOE OSTI2026

Development and Validation of the Near-Miss Safety Score (NMSS) Framework for Heavy-Duty Vehicle Safety Assessment

Heavy-duty commercial vehicles present unique safety challenges due to their size, articulation dynamics, and operational complexity. As Advanced Driver Assistance Systems (ADAS) and Automated Driving Systems (ADS) become more common in Class 8 tractor-trailers, traditional crash-based metrics are no longer sufficient to evaluate safety performance. This study introduces the Near-Miss Safety Score (NMSS)—a quantitative, physics-informed framework developed as a leading indicator of safety for advanced commercial vehicle technologies. NMSS quantifies how close a vehicle or operator comes to a collision or safety-critical event by integrating vehicle kinematics (relative distance, velocity, and acceleration) with driver or system response latency and time-to-collision. A modifier function adjusts the base score for vehicle-specific and environmental factors such as trailer articulation, load distribution, braking condition, and roadway environment. The framework enables systematic evaluation of ADAS/ADS performance under a range of operational and degraded conditions. By capturing near-miss dynamics rather than relying on crash data, NMSS provides a proactive approach to risk assessment, accelerates technology validation, and enhances interpretability for regulators and fleet operators. The proposed NMSS was validated using data collected from a motorcoach platform, demonstrating the framework’s applicability to heavy-duty safety evaluation and performance benchmarking. Results and key insights are presented in this paper.

Siekmann, Adam [ORNL] (ORCID:0000000284653935)↗
DOE OSTI2026

Scaling dynamics in low-salt-rejection reverse osmosis for high-salinity produced water desalination: Mechanistic modeling and membrane autopsy

Membrane scaling remains a critical barrier to the reliable operation of desalination systems, particularly for hypersaline produced water (PW) treatment. This study fills the knowledge gap of autopsy-based model validation for PW desalination by elucidating scaling mechanisms in a Low-Salt-Rejection Reverse Osmosis (LSRRO) system through the integration of pilot-scale experimentation and complementary modeling approaches. A semi-empirical modeling framework was developed and applied to a multistage pilot LSRRO system equipped with nanofiltration and RO membranes treating high-salinity PW from the Permian Basin. Water quality analysis showed that total dissolved solids decreased from ~130,000 mg/L to ~1900 mg/L in the permeate, then further reduced to ~300 mg/L by a second-pass RO. Two different thermodynamic modeling approaches were evaluated: the first extends the LSRRO framework by incorporating system complexity and scaling phenomena, whereas the second method explicitly captures concentration polarization in localized supersaturation. Both methods illustrate the tendency for carbonate and sulfate scaling throughout the stages. Membrane autopsies revealed a silica-dominated deposit matrix, localized CaSO 4 at Stage 2, and minor barite/celestite despite their prominence in model predictions. Quantum-chemical calculations indicated silica scaling can be rationalized by favorable adsorption of H 4 SiO 4 on Fe-oxide surfaces (ΔG ≈ −44 kJ/mol), providing a kinetic pathway for interfacial inorganic polymerization even when bulk equilibrium predictions are conservative. Overall, the thermodynamic scaling modeling and membrane autopsy revealed heterogeneous, localized deposits with limited impact on LSRRO performance, while quantum analysis rationalized the thermodynamically unfavorable precipitation formation under bulk equilibrium, reconciling model–autopsy discrepancies. These insights support targeted pretreatment and silica-specific antiscalants to extend membrane lifetime and increase recovery, providing a transferable framework for hypersaline water desalination systems. The combined experimental–computational approach provides new mechanistic insight into scaling in hypersaline membrane systems and establishes a transferable framework for predicting and mitigating scaling in next-generation desalination technologies.

Low-salt-rejection reverse osmosis↗
DOE OSTI2020

Risk Model for EM Decision Support Toolsets

The Government Office of Accountability (GAO) has published several reports identifying the need for the Department of Energy (DOE) Office of Environmental Management (EM) to address mounting costs for DoE's cleanup program. DOEEM could greatly benefit from independent decision tool-sets/models that would allow them to evaluate options and inform business decision at the enterprise level considering site-specific life cycle costs and system plans. Program goals: Develop a tool-set that enables EM to independently evaluate alternatives, assess outcomes from different contracting strategies, and inform critical decisions for the enterprise. Project goals: Adaption of a Risk Model for integration with complimentary tool-sets for project level decision making. Operational Events: Discrete event model that evaluates operational variables (e.g. capacity, throughput, maintenance constraints) and identify bottlenecks. Identifying and Bounding Project Risk: Identify risks that impact confidence in meeting goals (e.g. production). Life cycle Cost: Evaluate impacts of staffing levels, inventory, and capital investments on life cycle costs. Methods and approach: Monte Carlo Analysis is being executed to generate results: Input derives from Risk Register Data; Tied to Projected and Target Schedules; Incorporates float duration within the model. Assumes associated risk mitigation actions being completed within a timeline of five fiscal years. Metrics include: Confidence in Meeting Production Goal; Confidence in Safety Standards; Confidence in Continuous Operation; Other Metrics can be added as appropriate regarding specific site needs. The adapted risk model can be used as a stand alone decision tool or can be integrated complementary tool-sets (i.e. process and cost models) for project-specific decisions. These support tool-sets can then be integrated with others for site- and complex- level evaluations. Future work includes designing an adaptable and modular framework that would allow integration of multiple tool-sets for holistic and/or targeted evaluation of alternative strategies for decision making that could lead to risk and cost reduction across the enterprise.

12 MANAGEMENT OF RADIOACTIVE AND NON-RADIOACTIVE W↗
DOE OSTI2025

Evaluating a Transition-jump System for the Fermilab Main Injector Using Xsuite

We describe an Xsuite simulation framework for the Fermilab Main Injector (MI) along with an evaluation of transition-crossing behaviors in the accelerator. In particular, we studied the introduction of quadrupole magnets into the lattice as part of a transition-jump system that will be implemented through the $2^{nd}$ Proton Improvement Plan (PIP-II). Simulated beam losses spurred by transition-induced instabilities were assessed under several systematic effects, including MI quad errors, magnet-to-magnet variability in the jump magnets, power-supply errors, and timing jitter.

Schreckenberger, A. P. [Fermilab] (ORCID:000000016↗