SEARCH · Engineering Papers

Results for “evaluation framework”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 559 records · Page 31

All sources NASA NTRS DOE OSTI

DOE OSTI2025

Evaluating a Transition-Jump System for the Fermilab Main Injector Using Xsuite

We describe an Xsuite simulation framework for the Fermilab Main Injector (MI) along with an evaluation of transition-crossing behaviors in the accelerator. In particular, we studied the introduction of quadrupole magnets into the lattice as part of a transition-jump system that will be implemented through the $2^{nd}$ Proton Improvement Plan (PIP-II). Simulated beam losses spurred by transition-induced instabilities were assessed under several systematic effects, including MI quad errors, magnet-to-magnet variability in the jump magnets, power-supply errors, and timing jitter.

Schreckenberger, Adam Paul [Fermilab] (ORCID:00000↗

DOE OSTI2022

Evaluating efficacy of indoor non-pharmaceutical interventions against COVID-19 outbreaks with a coupled spatial-SIR agent-based simulation framework

Contagious respiratory diseases, such as COVID-19, depend on sufficiently prolonged exposures for the successful transmission of the underlying pathogen. It is important that organizations evaluate the efficacy of non-pharmaceutical interventions aimed at mitigating viral transmission among their personnel. We have developed a operational risk assessment simulation framework that couples a spatial agent-based model of movement with an agent-based SIR model to assess the relative risks of different intervention strategies. By applying our model on MIT’s Stata center, we assess the impacts of three possible dimensions of intervention: one-way vs unrestricted movement, population size allowed onsite, and frequency of leaving designated work location for breaks. We find that there is no significant impact made by one-way movement restrictions over unrestricted movement. Instead, we find that reducing the frequency at which individuals leave their workstations combined with lowering the number of individuals admitted below the current recommendations lowers the likelihood of highly connected individuals within the contact networks that emerge, which in turn lowers the overall risk of infection. We discover three classes of possible interventions based on their epidemiological effects. By assuming a direct relationship between data on secondary attack rates and transmissibility in the agent-based SIR model, we compare relative infection risk of four respiratory illnesses, MERS, SARS, COVID-19, and Measles, within the simulated area, and recommend appropriate intervention guidelines.

60 APPLIED LIFE SCIENCES↗

DOE OSTI2024

An Approach to Dynamic Human Reliability Analysis and Its Data Collection Framework

Human reliability analysis (HRA) is a method for evaluating human errors in a variety of complex systems such as nuclear power plants, military systems, aircraft, and chemical plants. Most HRA methods currently used by regulatory institutes or utilities are called static HRA and are carried out by simple worksheets or simple calculators. To date, there are many unsolved or intrinsic challenges in static HRA. For example, existing static HRA does not realistically model and evaluate human actions as they would be performed at actual systems. There is no method with HRA to objectively estimate the time required for human actions despite being essential to HRA processes. In addition, many HRA methods still rely on a dataset generated prior to the 1980s, from unrelated industry experience or simply from expert judgment. Accordingly, this study attempted to research how to overcome the challenges of existing HRA via dynamic risk assessment (a.k.a., simulation-based or computation-based risk assessment) techniques. First, this study developed a dynamic HRA method, named as PRocedure-based Investigation Method of EMRALD Risk Assessment – HRA (PRIMERA-HRA). The PRIMERA-HRA mainly concentrates on providing HRA analysts with specific guidelines on how to reasonably model human actions, assign human reliability data and evaluate output of simulation within a dynamic probabilistic risk assessment tool, called as Event Modeling Risk Assessment using Linked Diagrams (EMRALD). Second, this study also developed a module for performance shaping factors (i.e., the key concept in HRA quantification) applicable to dynamic HRA, then implemented it based on PRIMERA-HRA within the EMRALD tool. Third, this study developed an HRA data collection framework to support dynamic HRA, called as Simplified Human Error Experimental Program (SHEEP). Originally, the SHEEP study aimed to support static HRA and its data collection, but recently extended the scope to the new technologies such as dynamic HRA or HRA for advanced reactors. SHEEP focuses on the use of data collected from simplified simulators to complement—but not replace—data collection studies using full-scope simulators and actual operators. To date, many experiments were conducted under the SHEEP framework. Multiple analyses, such as human performance analysis, human error analysis, task complexity analysis, learning effect analysis and time distribution analysis, were also carried out using the collected data. Then, based on the major insights, an approach to inferring full-scope data based on simplified simulator data was proposed. The PRIMERA-HRA and SHEEP research are expected to evaluate human actions more realistically than existing static HRA, provide an opportunity to collect more HRA data with reasonable cost and labor, then contribute to enhance the quality of HRA.

99 - GENERAL AND MISCELLANEOUS↗

DOE OSTI2024

An Approach to Dynamic Human Reliability Analysis and Its Data Collection Framework

99 - GENERAL AND MISCELLANEOUS↗

DOE OSTI2023

Adaptive sampling quasi-Newton methods for zeroth-order stochastic optimization

Here, we consider unconstrained stochastic optimization problems with no available gradient information. Such problems arise in settings from derivative-free simulation optimization to reinforcement learning. We propose an adaptive sampling quasi-Newton method where we estimate the gradients using finite differences of stochastic function evaluations within a common random number framework. We develop modified versions of a norm test and an inner product quasi-Newton test to control the sample sizes used in the stochastic approximations and provide global convergence results to the neighborhood of a locally optimal solution. We present numerical experiments on simulation optimization problems to illustrate the performance of the proposed algorithm. When compared with classical zeroth-order stochastic gradient methods, we observe that our strategies of adapting the sample sizes significantly improve performance in terms of the number of stochastic function evaluations required.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2022

Design-Space Exploration for Inverse-Design of Wind Turbine Blades Using Data-Driven Methods

The state-of-the-practice aerodynamic design methods for wind turbine blades is typically based on Blade-Element Momentum (BEM) theory using a pre-designed frozen family of airfoils. The airfoils are themselves typically designed using panel methods. The design of the next-generation of large flexible rotors will need to capture the non-linear aerodynamics and three-dimensional flow to reduce the levelized cost of wind energy. Data-driven methods for aerodynamic design using data generated by computational fluid dynamics offer an attractive alternative to BEM-based methods that captures the non-linear aerodynamics of the component airfoils as well as the root and tip sections. In this work, we develop and demonstrate a framework to "smartly" explore the relevant design space in combination with an appropriate automated CFD pipeline to evaluate the aerodynamics of each design. The design-space exploration framework uses appropriate perturbations to the airfoil shape and induction profile from a baseline shape in combination with the inverse-design using BEM. The resulting designs are evaluated using an automated CFD pipeline using the in-house CFD solver framework "Mercury". We perform verification and validation to establish the capability of the Mercury framework to predict the aerodynamic performance of wind turbines. The CFD simulation of the perturbed blade shapes are optimized to restart from the converged baseline simulation to reduce the computational time. Finally, we demonstrate the design-space exploration technique for the design of the outboard section and the full rotor using perturbations to the shape and operating conditions of the NREL 5-MW turbine.

aerodynamic design methods↗

DOE OSTI2021

Understanding implications of cathode architecture on energy density of solid-state batteries

Next generation solid-state batteries (SSB) will need to leverage high voltage cathodes, as well as metallic anodes to achieve the realistic performance targets necessary to replace liquid electrolyte-based batteries in cutting-edge applications including electric vehicles. However, limitations arising from mass and charge transports, kinetics and chemo-mechanical degradation at the electrode | electrolyte interface limit the performance of present day SSBs. Optimizing composite cathode architecture, which is an integral part of solid-state batteries, is vital to realize the high-energy density and high-performance goals for next-generation solid-state batteries. Additionally, cathode architecture needs to be optimized for high loadings of active material, well-percolated ion and electron transport pathways and increased resilience against electrochemical stresses. This paper provides a first report of framework for geometric modeling of composite cathode architectures and evaluates the impact of cathode architecture on cell-level energy density using hierarchical models. Packing around primary and secondary active material particles are simulated for a range of active material particle size and solid electrolyte size distributions in the composite cathode. Impact of packing architecture on processing parameters of a given cathode composition and thickness, as well as on achievable energy density is evaluated for a range of commonly used solid electrolyte and cathode materials. Overall, the proposed framework offers a facile exploratory methodology for establishing initial metrics for scalable processing of practical and competent SSBs.

25 ENERGY STORAGE↗

DOE OSTI2021

Occupancy-Driven Stochastic Decision Framework for Ranking Commercial Building Loads

For effective integration of building operations into the evolving demand response programs of the power grid, real-time decisions concerning the use of building appliances for grid services must excel on multiple criteria, ranging from the added value to occupants' comfort to the quality of the grid services. In this paper, we present a data-driven stochastic decision-support framework to dynamically rank load control alternatives in a commercial building, addressing the needs of multiple decision criteria (e.g. occupant comfort, grid service quality) under uncertainties in occupancy patterns. We adopt a stochastic multi-criteria decision algorithm recently applied to prioritize residential on/off loads, and extend it to i) consider complex load control decisions (e.g. dimming of lights, changing zone temperature set-points) in a commercial building; and ii) systematically integrate zonal occupancy patterns to better identify short-term (and time-varying) opportunities for grid service participation. We evaluate the performance of the proposed framework for curtailment of air-conditioning, lighting, and plug-loads in a multi-zone commercial office building for a range of design choices. With the help of a prototype system that integrates an interactive \textit{Data Analytics and Visualization} frontend we demonstrate a way for the building operators to monitor and change in real-time the available flexibility in energy consumption and to develop trust in the decision recommendations by interpreting the rationale behind the ranking.

Jain, Milan↗

DOE OSTI2025

Climate-extreme modeling framework for sustainable flood management in the Arabian Peninsula

Evaluating extreme precipitation events (EPEs) is essential for building climate-resilient water management strategies, but it remains a major challenge in ungauged basins. Using the 26,070 km 2 Wadi al-Rummah basin in central Saudi Arabia as a case study, we developed an alternative, reliable, cost-effective satellite-based framework that combines empirically derived EPE thresholds, imagery-calibrated 2D hydrodynamic modeling, GRACE water-storage diagnostics, and bias-corrected CMIP6 projections to assess flood hazards and recharge potential under current and future climate scenarios in ungauged basins. The integrated approach and the resulting findings followed four key steps: (1) Identified a 22.5 mm EPE threshold, the 80th percentile of 3-day GPM/IMERG rainfall (2000–2024), aligned with flood-triggering events (Nov 2018: 23–28 mm; Apr, 2023: 42 mm); (2) Developed and calibrated a RiverFlow2D model using Sentinel-2 and PlanetScope imagery for the November 2018 flood, accurately reproducing flood depth and extent (RMSE ≤0.31 m; fuzzy-Dice ≥0.91), and estimating runoff (41 %), infiltration (25 %), and evaporation (34 %); (3) Independently validated the model with the April 2023 event (RMSE ≤0.35 m; fuzzy-Dice ≥0.86); (4) Conducted climate projections (2025–2100) from five bias-corrected NEX-GDDP CMIP6 models that revealed a 34 % increase in EPE intensity under SSP2-4.5 and 48 % under SSP5-8.5 scenarios, relative to 20th-century baselines. Our findings indicate that while intensifying extremes in the 21st century increase flood risk, the results highlight the potential for episodic recharge if effective retention strategies are employed, and offer a transferable model for climate-informed planning in data-scarce arid regions.

CMIP6↗

DOE OSTI2025

Reticular Materials and AI-Driven Computer Simulations for Seawater Mining of Valuable Metals (Final Technical Report)

This Final Technical Report describes our exploratory efforts that combine reticular materials synthesis (hydrolytically robust metal–organic frameworks, MOFs) with AI‑enabled molecular simulations to develop mechanistic, quantitative design rules for recovering lithium and other alkali-metal ions from highly dilute, competitive aqueous resources (e.g., seawater). The central outcome is a joint experimental–computational study of ion uptake in MOF‑808 (Chemical Science, 2025) that quantifies both thermodynamics and kinetics of Li + , Na + , and K + uptake and identifies how pore size, pore hydration state, dehydration penalties, and pore-window transport barriers govern selectivity. Guided by these insights, we synthesized and tested functionalized MOF‑808 and multivariate MOFs incorporating ion-recognition motifs (including carboxylates and crown-ether linkers) and evaluated uptake in synthetic seawater, highlighting framework topology and pore chemistry as levers for improved Li + /Na + discrimination. We also developed transferable simulation models, enhanced-sampling protocols, and automated workflows that enable systematic screening of porous sorbents.

42 ENGINEERING↗

DOE OSTI2024

Evaluating large scale aqueous organic redox flow battery performance with a hybrid numerical and machine learning framework

Aqueous organic redox flow battery (AORFB) is a promising cost-competitive technology for large-scale energy storage. Among existing work, the dihydroxyphenazine (DHP)-based AORFB has demonstrated high energy density and low-capacity degradation in 10 cm$^2$ cells during lab tests. However, its commercial-scale performance in more complex environments remains unknown, posing a barrier to commercialization. To address this gap, this work presents a comprehensive performance evaluation of a 780 cm$^2$ DHP-based AORFB by combining a physics-based numerical model, machine learning (ML)-based surrogate models, and ML-derived sensitivity quantification. Specifically, we first select 12 key battery parameters that include 10 physicochemical and 2 operation quantities, then select 6 performance metrics that include energy efficiency (EE), discharging capacity, charging energy, and power losses due to concentration, activation, and ohmic over-potentials. With such selection, 12800 combinations of the 12 parameters are subsequently generated using the Latin Hypercube Sampling method. Such combinations, together with 38 pre-defined State of Charge, are then integrated to a validated AORFB model developed in COMSOL to compute the performance metrics. With both input parameters and performance metrics, 60 deep neural network (DNN) surrogate models are then trained to approximate the relationship between the 10 physicochemical quantities and 6 performance metrics at each flow rate and current density. Sensitivity scores are then calculated based on the DNN models. Two additional sensitivity analysis tools, i.e., MARS, and SHAP, are also used to cross-validate the sensitivity scores from the DNN. The results demonstrate that 1) the standard potential ranks first in controlling EE and charging energy, 2) the membrane conductivity is most critical for power loss and EE, and 3) specific area and reaction rate control activation power loss.

25 ENERGY STORAGE↗

DOE OSTI2023

Experimental and Computational Mechanisms that Govern Long-Term Stability of CO 2 -Adsorbed ZIF-8-Based Porous Liquids

Porous liquids (PLs) based on the zeolitic imidazole framework ZIF-8 are attractive systems for carbon capture since the hydrophobic ZIF framework can be solvated in aqueous solvent systems without porous host degradation. However, solid ZIF-8 is known to degrade when exposed to CO 2 in wet environments, and therefore the long-term stability of ZIF-8-based PLs is unknown. Here, through aging experiments, the long-term stability of a ZIF-8 PL formed using the water, ethylene glycol, and 2-methylimidazole solvent system was systematically examined, and the mechanisms of degradation were elucidated. The PL was found to be stable for several weeks, with no ZIF framework degradation observed after aging in N 2 or air. However, for PLs aged in a CO 2 atmosphere, formation of a secondary phase occurred within 1 day from the degradation of the ZIF-8 framework. From the computational and structural evaluation of the effects of CO 2 on the PL solvent mixture, it was identified that the basic environment of the PL caused ethylene glycol to react with CO 2 forming carbonate species. These carbonate species further react within the PL to degrade ZIF-8. The mechanisms governing this process involves a multistep pathway for PL degradation and lays out a long-term evaluation strategy of PLs for carbon capture. Additionally, it clearly demonstrates the need to examine the reactivity and aging properties of all components in these complex PL systems in order to fully assess their stabilities and lifetimes.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

DOE OSTI2025

Learning to Simulate Aerosol Dynamics with Graph Neural Networks

Aerosol effects on climate, weather, and air quality depend on characteristics of individual particles, which are tremendously diverse and change in time. Particle-resolved models are the only models able to capture this diversity in particle physiochemical properties, and these models are computationally expensive. As a strategy for accelerating particle-resolved microphysics models, we introduce Graph-based Learning of Aerosol Dynamics (GLAD) and use this model to train a surrogate of the particle-resolved model PartMC-MOSAIC. GLAD implements a Graph Network-based Simulator (GNS), a machine learning framework that has been used to simulate particle-based fluid dynamics models. In GLAD, each particle is represented as a node in a graph, and the evolution of the particle population over time is simulated through learned message passing. Here, we demonstrate our GNS approach on a simple aerosol system that includes condensation of sulfuric acid onto particles composed of sulfate, black carbon, organic carbon, and water. A graph with particles as nodes is constructed, and a graph neural network (GNN) is then trained using the model output from PartMC-MOSAIC. The trained GNN can then be used for simulating and predicting aerosol dynamics over time. Results demonstrate the framework's ability to accurately learn chemical dynamics and generalize across different scenarios, achieving efficient training and prediction times. We evaluate the performance across four scenarios, highlighting the framework's robustness and adaptability in modeling aerosol microphysics and chemistry.

aerosol chemistry dynamics↗

DOE OSTI2021

EXTRACTING HUMAN RELIABILITY FINDINGS FROM HUMAN FACTORS STUDIES IN THE HUMAN SYSTEMS SIMULATION LABORATORY

Modernization of U.S. nuclear power plants (NPPs) is widespread, with most plants currently replacing and transitioning equipment, control systems, and human system interfaces (HSI)s from analog to digital displays. This conversion remedies the obsolescence of analog parts along with needs for increased intuitiveness of design, safety, and capabilities. The Human Factors and Reliability team at Idaho National Laboratory (INL) carried out twelve control room modernization studies in the newly designed Human Systems Simulation Laboratory (HSSL) over nine years. The HSSL was constructed as a testbed for evaluating human factors techniques and performance measures, HSI frameworks, and cutting-edge operational concepts in NPPs. Installing a full-scope training simulator enabled direct design and evaluation work on the same instrumentation and control (I&C) and HSIs located at U.S. plants. The subsequent addition of glass top bays afforded crews opportunities to implement operations via the simulator using full-scale representations of their home NPP. Additionally, functional HSI prototypes were created, providing an environment for operator-in-the-loop benchmark studies. The HSSL has assisted in upgrades of six commercial NPP control rooms and served as an invaluable proving ground for new NPP operations technology. Human reliability analysis (HRA) was not originally the focus of the studies; however, data relating to HRA such as type and frequency of human errors can be extracted from the studies. INL is currently extracting data from the HSSL study reports to apprise how information gathered from simulation, HSI, and other related studies can create a broad look across different data sources to help inform HRA methods.

99 GENERAL AND MISCELLANEOUS↗

DOE OSTI2022

Risk-Informed Approach for Regulatory Approval of Microreactor Transport

Pacific Northwest National Laboratory (PNNL) is addressing the challenges associated with safe transport of microreactors including the development and evaluation of regulatory options. PNNL developed a risk-informed regulatory framework for the licensing of the transportation of microreactors in which irradiated nuclear fuel is part microreactor transportation package. The framework lays out a viable regulatory pathway, including decision points for regulatory options and the supporting technical evaluations for those options in phases from near to long term. A microreactor and its contents will likely not be able to meet all the federal regulatory requirements as a Type B or fissile material transportation package under 10 CFR Part 71 (“Packaging and Transportation of Radioactive Material”). However, the regulatory framework developed by PNNL lays out a viable risk-informed licensing options that are safe and feasible. Risk assessment such as probabilistic risk assessment (PRA) can be used to show comparable safety to that provided by a Type B or fissile material package for surface transport. The framework includes guidance on applicable regulations and discusses historical precedence in using risk information for transportation licensing. The framework includes guidance for performing a microreactor transportation PRA, use and development of risk evaluation criteria, and factors such of defense-in-depth and safety margin concepts. Key advantages of using the approach are (1) increasing the likelihood of successfully obtaining regulatory transportation package approval, (2) informing the design on the relative risk significance of microreactor containment and shielding, and (3) informing the need for transportation compensatory measures. This paper focuses on two primary elements of the framework which are development of a transportation PRA for microreactor packages and development of the risk acceptance guidelines to assess the results of the PRA for regulatory decision-making.

microreactor, micro nuclear power plant, MNPP, ris↗

DOE OSTI2024

Justice Underpinning Science and Technology Research (JUST-R) Offline Tool: A Tool for Guiding Researchers in Addressing Energy Justice Considerations in Early-Stage Research

The Justice Underpinning Science and Technology Research (JUST-R) Metrics Framework is a set of metrics for assessing the energy justice implications of technologies that are currently under research and development (R&D). The JUST-R Metrics Framework guides researchers through an analysis of the many facets of their research processes, from material inputs to knowledge sources, that may contribute to energy injustice both during the research period and when the technology is scaled. This JUST-R Offline Tool consists of an Excel file and PDF guide to aid researchers, engineers, and project managers in their application of the JUST-R Metrics Framework. This tool enables researchers to 1) evaluate the baseline energy justice implications of their research; 2) develop justice-oriented changes to the research process; and 3) track the implementation of proposed changes. The metrics included in the framework are sorted into five aspects of research: Team Dynamics, Sources & Inputs, Processes & Protocols, Waste & Hazards, and Results & Dissemination. The JUST-R Tool can be found here: https://www.nrel.gov/analysis/just-r.html.

29 ENERGY PLANNING, POLICY, AND ECONOMY↗

DOE OSTI2022

Incremental approach for the life-cycle greenhouse gas analysis of carbon capture and utilization

Electro-fuels (e-fuels) are examples of carbon capture and utilization (CCU) hydrocarbon products that are derived from captured carbon dioxide (CO 2 ), while using renewable electricity as the energy feedstock. The environmental impacts of CCU products (e.g., e-fuel) are systematically quantified through life-cycle analysis (LCA). Previous studies evaluating LCA of e-fuels proposed frameworks with an expanded system boundary approach that included the entire supply chain of the production process generating the CO 2 for CCU, in addition to the supply chain of the CCU product. This expanded system boundary approach evaluates two system boundaries, and uses deduction methods to calculate the carbon intensity (CI) of the CCU product (e-fuel). This paper proposes a simpler system boundary using an incremental approach that can calculate identical CI of the CCU product (e-fuel), while avoiding the extensive calculations in the expanded system boundary framework. The proposed incremental approach allocates the burdens of the CO 2 capturing process to the CO 2 feedstock supplying the CCU production process (e.g., e-fuel production). The CI of the captured CO 2 supplied to CCU process is determined by the energy and material requirements for the CO 2 capturing process and transportation to the CCU plant. Thus, the CI of CO 2 supplied to CCU process can be directly linked to the CI of e-fuel without the need to conduct LCA of the preceding process that generates the CO 2 for CCU.

29 ENERGY PLANNING, POLICY, AND ECONOMY↗

DOE OSTI2021

Distributed Energy Resources Cybersecurity Framework & Risk Manager

Distributed Energy Resource Cybersecurity Framework (DER-CF) provides a holistic assessment for evaluating the cybersecurity posture of DER systems - filling an important gap that expands upon existing cybersecurity frameworks for more modern energy systems. The DER-CF is available as a written framework and an interactive Web tool. Distributed Energy Resource Risk Manager (DER-RM) process adheres closely to NIST's seven risk management steps: prepare, categorize, select, implement, assess, monitor, and authorize. This added feature is independent of the DER-CF's existing self-assessment and allows managers to focus on the RMF process.

cybersecurity↗