Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “model evaluation”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 793 records · Page 44
All sourcesNASA NTRSDOE OSTI
DOE OSTI2025

A Scalable Multi-Modal Framework for High-Fidelity Distributed Human Mobility Simulations

The development of data-driven models for human mobility in urban settings requires access to substantial and diverse real-world data. However, existing historical data often presents challenges such as limited volume, variety, and veracity, as well as missing data and privacy preservation concerns. Also, urban mobility modeling is inherently time-variant, complex, and multi-modal, encompassing everything from individual walking and running to private road travel and large-scale public transportation. These challenges call for innovative solutions to overcome data limitations and compute needs to model mobility behaviors accurately. To address these challenges, we propose a distributed, co-simulation-based architecture DURMOSim that integrates real-world data with scalable, high-fidelity simulations, demonstrating distributed co-simulation feasibility with existing mobility models. DURMOSim underpins a modular integration that would enable using any available mobility simulators for greater extensibility and scalability in performing various urban scenarios. In this paper, we present the design, implementation, and performance evaluation of DURMOSim, highlighting its capability to model population-scale mobility patterns. Our initial results show its ability to dynamically synchronize multiple simulation models at runtime with negligible computational overhead. We believe DURMOSim could be a robust tool for advancing urban mobility research and intelligent transportation systems.

Yoginath, Srikanth [ORNL] (ORCID:0000000184236050)↗
DOE OSTI2026

Developing Science-based fueling protocols for 250-bar hydrogen tanks onboard hydrogen ferries: Experiments and modeling

Combined modeling and experimental studies are reported of the fueling of a large (28 kg capacity) 250-bar Type IV hydrogen tank of the type being deployed on early hydrogen ferries, such as the MV Sea Change. The primary goal was to determine how such tanks can be successfully fueled with hydrogen (state of charge greater than 97%) within 45 minutes without exceeding the 82 °C temperature limit for such tanks. The modeling studies show that a gas injector is needed to avoid thermal stratification during hydrogen fueling which can result in potential hot spots. Empirically, precooling of the hydrogen to 0 °C was found to be needed in some of the cases examined, as ambient conditions greatly affected the need for a precooling to achieve the 45-minute fill time desired by end users. The experimental results afforded a calibration of the engineering model SOFIL for these large 250-bar tanks, which now enables using SOFIL to predict volume-averaged hydrogen fueling temperatures to an accuracy of ±2.7°C for these tanks. The model can therefore be used to evaluate potential scenarios for development of a standardized fueling methodology for ferries utilizing large Type-IV tanks.

08 HYDROGEN↗
DOE OSTI2025

Securing Federated Learning Against Active Reconstruction Attacks

Federated Learning (FL) has amassed notable attention for its ability to preserve user privacy while emphasizing the retainment of model training efficiency. Due to this potential, FL has been integrated in many domains, such as healthcare, finance, law, and industrial engineering, where data cannot be easily exchanged due to sensitive information and strict privacy laws. However, current research has indicated that FL protocols are easily compromised by active data reconstruction attacks employed by actively dishonest servers. The malicious modification of global model parameters allows an actively dishonest server to obtain a direct copy of users’ private data via gradient inversion. Here, this class of attacks is highly underexplored and continues to be a major challenge due to the intense threat model. In this paper, we propose OASIS as a scalable and modality-agnostic defense based on data augmentation that counteracts active data reconstruction attacks while preserving model performance. To generalize our defense, we uncover the intuition behind gradient inversion that enables these attacks and theoretically establish the conditions by which the defense can be considered robust regardless of attack design. From this, we formulate our defense with data augmentation that illustrates its ability to undermine the attack principle. We evaluate OASIS on five real-world datasets–two image-based (ImageNet and CIFAR100) and three text-based (Wikitext, Stack Overflow, and Shakespeare)–which span diverse uses cases such as vision tasks and language modeling. Comprehensive evaluations on these datasets exhibit the efficacy of OASIS and highlight its feasibility as a solution.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2026

Improbability of Post-Closure Criticality in Compacted Criticality Control Overpacks after Room Closure at Waste Isolation Pilot Plant

As part of its periodic re-certification of the Waste Isolation Pilot Plant (WIPP), an operating repository in bedded salt for the disposal of transuranic (TRU) waste from atomic energy defense activities, the United States Environmental Protection Agency expects a re-evaluation of features, events, and processes, such as post-closure nuclear criticality. Although salt creep beneficially encapsulates the TRU waste in the closed WIPP repository, the spacing between an array of waste packages is disrupted as the salt creep closes disposal rooms and containers lose structural integrity. For most TRU waste, the possibility of post-closure criticality is exceedingly small either because the salt neutronically isolates TRU waste canisters or because closure of a disposal room from salt creep does not sufficiently compact the low mass of fissile material. The criticality evaluation was updated, however, because of the introduction of criticality control overpack (CCO) containers, which may dispose up to 380 fissile gram equivalent plutonium-239 in each container. The criticality potential is evaluated through high-fidelity geomechanical modeling of a disposal room filled with CCO containers during two representative conditions: (1) large salt block fall, and (2) gradual disposal room closure from salt creep. Geomechanical models of roof fall demonstrate three tiers of CCO containers are not greatly disrupted. Geomechanical models of gradual room closure from salt creep (without brine seepage and subsequent gas generation to permit maximum room closure) were used to predict irregular arrays of closely packed CCOs after 1000 years, when room closure has asymptotically approached maximum compaction. Models of spheres or cylinders with 380 fissile gram equivalent of plutonium (as oxide) at the predicted irregular compacted spacing demonstrate that an array of CCO containers is not critical when surrounded by salt and magnesium oxide, provided the mass of hydrogenous material shipped in CCO containers (usually plastics) is controlled or boron carbide (a neutron poison) is mixed with the fissile contents.

12 MANAGEMENT OF RADIOACTIVE AND NON-RADIOACTIVE W↗
NASA NTRS1993

Development of global/chemistry model for jet-fuel thermal stability based on observations from static and flowing experiments

Two global-chemistry models for oxidative deposition of jet fuels are evaluated by integrating them into a Computational Fluid Dynamics with Chemistry (CFDC) code. A previously developed two-step global-chemistry model was found to be insufficient to describe the thermal-oxidation and -deposition rates associated with a Jet-A fuel. A new global-chemistry model has been developed systematically based on observations from flowing and static experiments. The global-autoxidation reaction is modified such that the reaction rate becomes zeroth-order with respect to the dissolved oxygen concentration. The generation of deposit-forming precursor is coupled with the autoxidation reaction by introducing a radical species ROO. A formulation for the sticking probability has also been developed. Deposition profiles are well represented by this new model under a variety of temperature and flow conditions. The model correctly predicts the changes in magnitude and spatial location of the deposition peak due to changes in flow. The CFDC model, which is designed for flowing systems, has been extended to static experiments. The model incorporates a non-depleting species F(sub s) representing all non-oxygen compounds responsible for deposition. Static experiments were found to provide a useful and inexpensive method for estimating the concentration of F(sub s) in the fuel.

V. R. Katta↗
DOE OSTI2025

Use of Rig Parameter Data in Bit Constraint Models for Improved Drilling Performance at The Geysers

Surface parameter measurements are routinely used during deep well construction to monitor and guide drilling conditions for improved performance and reduced costs. However, these measurements are of reduced value without a standard to aid in evaluation and decision making. A method is demonstrated whereby drill bit constraint models are used to interpret drilling response parameters. Drill rig parameter data for well GDC-36 at the Geysers Geothermal Field Power were acquired by Geysers Power Company and drilling contractor Kenai Drilling using Pason US DataHub and evaluated. Drilling parameters are evaluated using laboratory-validated rock reduction models for predicting the phenomenological response of drag bits (Detournay and Defourny, 1992) along with other model constraints in computational algorithms. The method is used to evaluate overall bit performance, monitor bit integrity, and detect the presence of drillstring vibrations and other conditions contributing to bit failure; comparisons are made to observations of bit wear and damage. The method will be applied in real-time to improve decision-making on subsequent wells and has applicability to development of advanced analytics on future geothermal wells using real-time electronic drilling recorder (EDR) data for improved performance and reduced drilling costs.

15 GEOTHERMAL ENERGY↗
DOE OSTI2025

Implementation of Perturbation Theory and Sensitivity Capabilities in Griffin

Griffin is a Multiphysics Object-Oriented Simulation Environment (MOOSE) based reactor Multiphysics analysis application, jointly developed by Argonne and Idaho National Laboratories under the DOE-NE NEAMS program. This fiscal year, capabilities for reactivity and sensitivity evaluation using perturbation methods were implemented and verified. The First Order Perturbation Method (FOPT) was employed to compute reactivity worth resulting from small perturbations in input parameters, while the Generalized Perturbation Theory (GPT) was used to evaluate sensitivities of a range of response types, including reaction rate ratio, k-eigenvalue, neutron generation time, and effective delayed neutron fraction. These perturbation methods enable users to quantify how response quantities change due to a perturbation in a input parameter without explicitly performing an additional transport simulation for each perturbed state. In particular, the GPT formulation accounts for indirect effects arising from flux changes by solving generalized inhomogeneous equations, for which a Neumann series-based iterative solution method was developed and implemented in Griffin. The implemented reactivity and sensitivity evaluation capabilities were verified using two test problems: an infinite homogeneous system and a two-dimensional hexagonal core. The results showed excellent agreement with reference solutions obtained by a direct method based on finite difference approximation as well as GPT-based results from the PERSENT code, confirming the accuracy of both reactivity and sensitivity evaluations. Additionally, preliminary uncertainty quantification (UQ) results were obtained by combining the sensitivity values computed using GPT and external covariance data, demonstrating that the implemented sensitivity results can be reliably used for uncertainty calculations. To further demonstrate the generality and practical strength of the implementation, the sensitivity evaluation capability was successfully applied to the Empire microreactor with a geometrically complex design that poses significant modeling challenges. The results confirm that Griffin enables sensitivity evaluations even for irregular and highly heterogeneous reactor configurations, thereby establishing a foundation for UQ applications in advanced reactor designs and analyses.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗
DOE OSTI2025

A Modelica Implementation of an Organic Rankine Cycle

Organic Rankine cycle (ORC) systems generate power from low-grade heat sources, such as geothermal sources and industrial waste heat. A key feature is that a working fluid is selected to match the temperature of the source. With the vast pool of candidate working fluids comes the challenge of developing a large number of robust thermodynamic media models. We implemented a subcritical ORC model in Modelica that uses working fluid data records and interpolation schemes in lieu of thermodynamic medium evaluation for energy recovery estimation. This is a component model that can be integrated into a larger energy system model. It does not require detailed thermodynamic, heat transfer, or machine analysis. Our ORC model fills a gap where working fluids are ready to choose or easy to add, and at the same time can be integrated into an energy system.

29 ENERGY PLANNING, POLICY, AND ECONOMY↗
DOE OSTI2025

LLM-Inference-Bench: Inference Benchmarking of Large Language Models on AI Accelerators

Large Language Models (LLMs) have propelled groundbreaking advancements across several domains and are commonly used for text generation applications. However, the computational demands of these complex models pose significant challenges, requiring efficient hardware acceleration. Benchmarking the performance of LLMs across diverse hardware platforms is crucial to understanding their scalability and throughput characteristics. We introduce LLM-Inference-Bench, a comprehensive benchmarking suite to evaluate the hardware inference performance of LLMs. We thoroughly analyze diverse hardware platforms, including GPUs from Nvidia and AMD and specialized AI accelerators, Intel Habana and SambaNova. Our evaluation includes several LLM inference frameworks and models from LLaMA, Mistral, and Qwen families with 7B and 70B parameters. Our benchmarking results reveal the strengths and limitations of various models, hardware platforms, and inference frameworks. We provide an interactive dashboard to help identify configurations for optimal performance for a given hardware platform.

Chitty-Venkata, Krishna Teja↗
DOE OSTI2025

Enhancing ChatPORT with CUDA-to-SYCL Kernel Translation Capability

Large Language Models (LLMs) have shown strong capabilities in general code translation. However, code translation involving parallel programming models remains largely unexplored. This work enhances the capabilities of code LLMs in CUDA-to-SYCL kernel translation with parameter-efficient fine-tuning. The resultant fine-tuned LLM, called ChatPORT, is an effort to provide high-fidelity translations from one programming model to another. We describe the preparation of datasets from heterogeneous computing benchmarks for model fine-tuning and testing, the parameter-efficient fine-tuning of 19 open-source code models ranging in size from 0.5 to 34 billion parameters and evaluate the correctness rates of the SYCL kernels by the fine-tuned models. The experimental results show that most code models fail to translate CUDA codes to SYCL correctly. However, fine-tuning these models using a small set of CUDA and SYCL kernels can enhance the capabilities of these models in kernel translation. Depending on the sizes of the models, the correctness rate ranges from 19.9% to 81.7% for a test dataset of 62 CUDA kernels.

Jin, Zheming [ORNL] (ORCID:000000027197780X)↗
DOE OSTI2025

Evaluating ecosystem water use efficiency and recovery dynamics during flash droughts: insights from observations and model simulations

Flash droughts (FD), rapidly emerging in a warming future, disrupt ecosystems, agriculture, and water security. Ecosystem water use efficiency (WUE), the ratio of gross primary production (GPP) to actual evapotranspiration (AET), balances carbon assimilation and water loss. FD rapidly disrupts this balance, making WUE critical for assessing plant stress and recovery. Here, this study investigates the dynamics of landscape-scale WUE, and the components of GPP and AET under FD utilizing both observed data from the Missouri Ozark AmeriFlux site (US-MOz) and version 2 of the U.S. Department of Energy’s Earth, Energy, Exascale System Model (E3SM) Land Model (ELMv2). Observations and simulations reveal GPP as dominant for WUE during earlier FD events (2005, 2007, 2012), shifting to AET in recent events (2014, 2018). This agreement indicates that the ELM can capture the shifting dynamics of GPP and AET in regulating WUE under FD conditions. However, the ELM systematically underestimates both GPP and AET and does so in a manner that does not preserve their ratio. As a result, WUE is also underestimated, suggesting that GPP is more strongly underestimated than AET. Furthermore, the ELM also underestimates the speed of GPP recovery, producing an artificially prolonged GPP recovery time following FD events. Observed environmental drivers such as vapor pressure deficit (VPD), soil moisture (SM), and predawn leaf water potential (PLWP) effectively predict WUE, but ELM primarily highlights SM, underestimating VPD’s role. This study demonstrates that relying solely on soil moisture fails to capture the rapid hydraulic recovery observed in PLWP, underscoring the necessity of integrating plant hydraulics into land surface models to improve flash drought predictability.

Evapotranspiration↗
DOE OSTI2025

Intrepid MCMC: Metropolis-Hastings with exploration

In engineering examples, one often encounters the need to sample from unnormalized distributions with complex shapes that may also be implicitly defined through a physical or numerical simulation model, making it computationally expensive to evaluate the associated density function. For such cases, MCMC has proven to be an invaluable tool. Random-walk Metropolis Methods (also known as Metropolis-Hastings (MH)), in particular, are highly popular for their simplicity, flexibility, and ease of implementation. However, most MH algorithms suffer from significant limitations when attempting to sample from distributions with multiple modes (particularly disconnected ones). Here, in this paper, we present Intrepid MCMC - a novel MH scheme that utilizes a simple coordinate transformation to significantly improve the mode-finding ability and convergence rate to the target distribution of random-walk Markov chains while retaining most of the simplicity of the vanilla MH paradigm. Through multiple examples, we showcase the improvement in the performance of Intrepid MCMC over vanilla MH for a wide variety of target distribution shapes. We also provide an analysis of the mixing behavior of the Intrepid Markov chain, as well as the efficiency of our algorithm for increasing dimensions. A thorough discussion is presented on the practical implementation of the Intrepid MCMC algorithm. Finally, its utility is highlighted through a Bayesian parameter inference problem for a two-degree-of-freedom oscillator under free vibration.

97 - MATHEMATICS AND COMPUTING↗
DOE OSTI2025

Coupled THM modeling of bentonite heating and hydration in tank tests with a new temperature-dependent water retention model

This study presents a coupled thermo-hydro-mechanical (THM) model for simulating the heating and hydration behavior of bentonite, a buffer material in deep geological repositories (DGRs). The model incorporates a new temperature-dependent soil water retention curve which captures the thermal-induced shift in water retention behavior. It also distinguishes between liquid and gas permeability, modeling intrinsic gas permeability as a function of accessible porosity to improve vapor transport and desaturation predictions. The model was validated against two large-scale tank tests, demonstrating good agreement with measured temperature, relative humidity, and water inflow data. It revealed a complex porosity evolution driven by thermal expansion, vapor movement, vapor condensation, and hydration-induced swelling during heating and hydration processes. The simulation results also suggest that the permeability of the hydration layer plays a critical role in controlling water intake. Clogging of this layer can significantly reduce the volume of water inflow during the hydration phase. Furthermore, while the model effectively captures key THM behavior, further development of the mechanical constitutive law is required to account for possible thermo-elasto-plastic volume changes and microstructural effects. Overall, the model provides a robust tool for evaluating the evolution of bentonite-based barrier material in DGRs.

Guo, Guanlong [Lawrence Berkeley National Laborato↗
DOE OSTI2024

a priori uncertainty quantification of reacting turbulence closure models using Bayesian neural networks

While many physics-based closure model forms have been posited for the sub-filter scale (SFS) in large eddy simulation (LES), vast amounts of data available from direct numerical simulations (DNS) create opportunities to leverage data-driven modeling techniques. Albeit flexible, data-driven models still depend on the dataset and the functional form of the model chosen. Increased adoption of such models requires reliable uncertainty estimates both in the data-informed and out-of-distribution regimes. Here, in this work, we employ Bayesian neural networks (BNNs) to capture both epistemic and aleatoric uncertainties in a reacting flow model. In particular, we model the filtered progress variable scalar dissipation rate which plays a key role in the dynamics of turbulent premixed flames. We demonstrate that BNN models can provide unique insights about the structure of uncertainty of the data-driven closure models. We also propose a method for the incorporation of out-of-distribution information in a BNN, which can be used for out-of-distribution query detection. The efficacy of the model is demonstrated by a priori evaluation on a dataset consisting of a variety of flame conditions and fuels.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2025

Decoupling thermal and irradiation effects on grain boundary segregation

Radiation-induced segregation (RIS) is most often measured by peak solute concentration at a boundary. However, this may give an incomplete picture of segregation quantity and phenomena. Radiation-induced and thermal segregation at grain boundaries was investigated in Fe-9.6 at.% Cr after 9 MeV Fe 3+ ion irradiation at 400 °C. The experimental results were compared to kinetic Monte Carlo (KMC) simulations. The study revealed that Cr enrichment (peak segregation) at the grain boundaries was comparable in both the irradiated and non-irradiated conditions, although irradiation resulted in broader segregation profiles in both experiments and simulations, indicating an overall increase in grain boundary segregation due to irradiation. This broadening is attributed to back diffusion into the grain interior. While it is an established phenomenon, this study offers a quantitative evaluation using experimental data and KMC modeling. Further, these results emphasize the importance of analyzing the entire segregation profile and decoupling the thermal and irradiation contributions to solute segregation.

Grain boundary segregation↗
DOE OSTI2025

Exposing Process‐Level Biases in a Global Cloud Permitting Model With ARM Observations

The emergence of global convective‐permitting models (GCPMs) represents a significant advancement in climate modeling, offering improved representation of deep convection and complex precipitation patterns. In this study, we evaluate the performance of the Simple Cloud‐Resolving E3SM Atmosphere Model (SCREAM) using its doubly periodic configuration (DP‐SCREAM) against large eddy simulations and modern observational data sets from the Atmospheric Radiation Measurement program. We introduce several new transitional cloud regime cases, such as the transition from shallow to deep convection and from stratocumulus to cumulus, as well as cold‐air outbreak scenarios. The results reveal both strengths and limitations of SCREAM, particularly in the accurate simulation of cloud transitions and midlevel convection, with varying degrees of sensitivity to horizontal and vertical resolution. Despite improvements at higher resolutions, key biases remain, including the abrupt transition from shallow to deep convection and the lack of congestus clouds. These findings underscore the need for further refinement in turbulence parameterizations and vertical grid resolution in GCPMs.

Bogenschutz, Peter A. [Lawrence Livermore National↗
DOE OSTI2025

Emulation With Uncertainty Quantification of Regional Sea‐Level Change Caused by the Antarctic Ice Sheet

Abstract Projecting regional sea‐level change under various climate‐change scenarios typically involves running forward simulations of the Earth's gravitational, rotational and deformational (GRD) response to ice‐mass change, which requires substantial computational cost if applied to probabilistic frameworks requiring thousands to millions of samples. Here we build emulators of regional sea‐level change at 27 coastal locations, due to the GRD effects associated with future Antarctic Ice Sheet mass change over the 21st century. The emulators are evaluated against a numerical sea‐level model applied to an ensemble of ice‐sheet model simulations of the Antarctic Ice Sheet through 2100. We build a physics‐based emulator using a recent sensitivity kernel approach and compare it to machine learning based emulators (neural network and conditional variational autoencoder methods). In order to quantify uncertainty, we derive well‐calibrated prediction intervals for regional sea‐level change via split‐conformal inference and linear regression, and show that Monte Carlo dropout does not yield well‐calibrated uncertainties in this instance. We also demonstrate substantial gains in computational efficiency using both the physics‐based emulator and neural networks in comparison to the numerical model for the complete regional sea‐level solution. Overall, we find the physics‐based emulator modestly outperforms the machine learning emulators for this problem.

58 GEOSCIENCES↗
DOE OSTI2025

Flower‐Type Organized Trade‐Wind Cumulus: A Multi‐Day Lagrangian Large Eddy Simulation Intercomparison Study

Shallow cumulus cloud fields in subtropical marine trade wind environments, particularly over the tropical Atlantic Ocean, show distinct organizational patterns. Among these, Flower‐type clouds are characterized by expansive stratiform cloud patches surrounded by regions of scattered convection. The objectives of this study were (a) to construct a case study of a time period during the EUREC 4 A/ATOMIC field campaign when Flower‐type organization was observed, (b) to evaluate the fidelity of a multi‐model ensemble of large eddy simulations of that case, and (c) to analyze the interaction between cloud and precipitation processes and mesoscale organization in the simulations. The simulations follow a quasi‐Lagrangian trajectory, allowing mesoscale features to develop over time in a domain that follows the boundary‐layer airmass. The results show a broad agreement in simulated thermodynamic properties across different LES codes, with Flower‐type cloud patches appearing within hours of each other. The consensus among models is consistent with observations made during the EUREC 4 A/ATOMIC field campaign on the specific day of interest. The cloud structure reveals three distinct peaks in the joint probability densities of cloud base and cloud top height, with the dominant peak at any given time influenced by the stage of cloud organization. The simulated cloud system evolution reveals consistent occurrence of maxima in liquid water path and rain rate before Flower reaches its maximum length scale. Targeted sensitivity tests reveal a weak relationship between Cloud Droplet Number concentration and the extent/degree/type of organization.

EUREC4A↗