Engineering PapersSearch

SEARCH · Engineering Papers

Results for “model evaluation”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 811 records · Page 45

Foundation models for atomistic simulation of chemistry and materials

Conventional computational methods for modeling chemical and materials systems are limited by system size and timescale, forcing a trade-off between quantum-mechanical accuracy and the sampling needed for realistic observables. Large language and vision foundation models — pre-trained on massive datasets using transformer architectures — have revolutionized many fields. It is thus interesting to ask whether a foundation model — subject to suitable data, parameter scaling and training — could enable learned simulations of chemistry and materials. Here, in this study, we review the field of machine-learned interatomic potentials (MLIPs) and posit that scaling up large and diverse chemical and materials datasets and highly expressive architectures using advanced training strategies should result in models that are: more efficient, transferable, robust to out-of-distribution scenarios, and easier to fine-tune to a variety of downstream physical observables than models trained from scratch on small datasets corresponding to specific, targeted atomistic simulation tasks. We provide specific criteria for creating such large-scale MLIP foundation models, coordinated strategies for their development, evaluation and deployment, and highlight potential emergent capabilities that could transform predictive simulations in chemistry and materials science and accelerate discovery across multiple technological domains.

Yuan, Eric C.-Y. [University of California, Berkel

A framework to evaluate machine learning crystal stability predictions

The rapid adoption of machine learning in various scientific domains calls for the development of best practices and community agreed-upon benchmarking tasks and metrics. We present Matbench Discovery as an example evaluation framework for machine learning energy models, here applied as pre-filters to first-principles computed data in a high-throughput search for stable inorganic crystals. We address the disconnect between (1) thermodynamic stability and formation energy and (2) retrospective and prospective benchmarking for materials discovery. Alongside this paper, we publish a Python package to aid with future model submissions and a growing online leaderboard with adaptive user-defined weighting of various performance metrics allowing researchers to prioritize the metrics they value most. To answer the question of which machine learning methodology performs best at materials discovery, our initial release includes random forests, graph neural networks, one-shot predictors, iterative Bayesian optimizers and universal interatomic potentials. We highlight a misalignment between commonly used regression metrics and more task-relevant classification metrics for materials discovery. Accurate regressors are susceptible to unexpectedly high false-positive rates if those accurate predictions lie close to the decision boundary at 0 eV per atom above the convex hull. The benchmark results demonstrate that universal interatomic potentials have advanced sufficiently to effectively and cheaply pre-screen thermodynamic stable hypothetical materials in future expansions of high-throughput materials databases.

Riebesell, Janosh

Laser absorption measurements of temperature, pressure, CO, and CO 2 at near-MHz rates in post-detonation fireballs with comparison to synthetic measurements

A laser absorption spectroscopy (LAS) diagnostic was used to obtain measurements of temperature, pressure, CO, and CO 2 at 500 kHz or 1 MHz in post-detonation fireballs produced by hemispherical samples of pentaerythritol tetranitrate (PETN). A quantum-cascade laser was scanned over multiple CO absorption transitions near 2008.5 cm −1 at 1 MHz, while an interband-cascade laser was scanned over a CO 2 absorption transition near 2394.8 cm −1 at 500 kHz. Light from each laser was combined onto a single path and passed through a detonation chamber approximately 83 mm above the 12-mm diameter hemispherical PETN charge. The CO and CO 2 absorption signals were post-processed to obtain time histories of temperature, pressure, species column pressures (P CO L, P CO2 L), and species column mole fractions (X CO L, X CO2 L). Additionally, schlieren imaging was performed simultaneously at 500 kHz to aid interpretation of the LAS measurements. Experimental and synthetic (i.e., CFD based) LAS measurements were compared to evaluate the accuracy of the CFD model and its ability to model the turbulent afterburning of the detonation products in air. In general, the experimental measurements exhibit reasonable agreement with the synthetic measurements at early times; thereby supporting the accuracy of the CFD model. Periods of disagreement between experimental and synthetic measurements at later times are most likely due to a reflected shock and detonator cavity jetting, which are not accounted for in the CFD model.

Schwartz, Charles J. [Purdue Univ., West Lafayette

Discriminative versus generative approaches to simulation-based inference

Most of the fundamental, emergent, and phenomenological parameters of particle and nuclear physics are determined through parametric template fits. Simulations are used to populate histograms which are then matched to data. This approach is inherently lossy, since histograms are binned and low-dimensional. Deep learning has enabled unbinned and high-dimensional parameter estimation through neural likelihood(-ratio) estimation. We compare two approaches for neural simulation-based inference (NSBI): one based on discriminative learning (classification) and one based on generative modeling. These two approaches are directly evaluated on the same datasets, with a similar level of hyperparameter optimization in both cases. In addition to a Gaussian dataset, we study NSBI using a Higgs boson dataset from the FAIR Universe Challenge. We find that both the direct likelihood and likelihood ratio estimation are able to effectively extract parameters with reasonable uncertainties. For the numerical examples and within the set of hyperparameters studied, we found that the likelihood ratio method is more accurate and/or precise. Both methods have a significant spread from the network training and would require ensembling or other mitigation strategies in practice.

high energy physics

Land change, fire, and climate weaken carbon sink in the conterminous United States

The land carbon sink of the conterminous United States was evaluated using a bottom-up modeling framework and 30-meter land change data from 1985 to 2020. This cross-scale, cross-landscape, and cross-system approach tracked fractional land cover changes and applied regional model calibration. Results show average terrestrial and aquatic carbon sinks of +110 ± 37 and +19 ± 0.5 teragrams of carbon per year, respectively. The terrestrial carbon sink, showing no clear trend, peaked in the 1990s, with more years as a carbon source since 2000, contradicting recent national and global studies. Land change had the largest impact (−70 ± 5.5 teragrams of carbon per year), exceeding impacts of climate (−33 ± 48 teragrams of carbon per year), wildfire (−7.7 ± 2.4 teragrams of carbon per year), and erosion transport (−1.9 ± 0.13 teragrams of carbon per year). The positive CO 2 fertilization effect (+69 ± 12 teragrams of carbon per year) was insufficient to maintain the carbon sink strength. Our framework reveals key paths of carbon loss, with implications for carbon budget and energy policies in the United States and beyond.

Liu, Jinxun [US Geological Survey, Reston, VA (Uni

Moltensaltpropnet

MoltenSaltPropnet is a physics-informed machine learning framework that aims to predict the thermophysical properties of molten fluoride and chloride salt mixtures, which are crucial for the design and safety of Generation IV molten salt reactors. The code processes data from the Molten-Salt Thermal Properties Database (MSTDB-TP) and the Janz compendium, converting critically evaluated correlations into fast, differentiable surrogate models for density, viscosity, thermal conductivity, and heat capacity across 448 distinct salt systems. The implementation consists of several key components: 1. Data Curation: The code parses and cleans the raw data, normalizing elemental mole fractions and extracting relevant regression coefficients for various thermophysical properties. 2. Feature Engineering: It generates fixed-length numerical descriptors that encapsulate the composition and temperature, incorporating polynomial interaction terms and dimensionality-reduction techniques to optimize model performance. 3. Coefficient Learning: Four different machine learning architectures are employed: a deep residual network (ResNet), a Kolmogorov–Arnold network (KAN), a sparsity-inducing neural network (SNN), and classical regression models. Each model learns to predict coefficients that define the temperature-dependent correlations for the thermophysical properties. 4. Property Reconstruction: The predicted coefficients are used to compute temperature-dependent property values, ensuring positivity and monotonic trends through a composite loss function that enforces physical constraints. 5. User Interface: An open-source web application enables users to filter the database, train task-specific models, and visualize the results, allowing for rapid exploration of candidate salt mixtures. MoltenSaltPropnet bridges the gap between limited experimental data and high-fidelity reactor simulations, providing a powerful tool for researchers in the field of molten salt reactors and advanced nuclear energy systems.

Retamales, Mauricio Eduardo Tano [Idaho National L

Applying Machine Learning and Bayesian Inference to Identify and Locate Moving Anthropogenic Sources Using Distributed Acoustic Sensing Data

Distributed acoustic sensing (DAS) systems, which use existing telecommunication fibers, offer high‐resolution capabilities ideal for recording anthropogenic sources. However, the complexity of urban environments and the large amount of data recorded by DAS require automated methods to efficiently detect and categorize anthropogenic sources. Here, we evaluate how well three machine learning models (k‐nearest neighbor [k‐NN], convolutional neural networks, and recurrent‐convolutional neural networks) can identify various anthropogenic sources recorded by DAS. Our findings reveal that both k‐NN and neural network methods perform well in high signal‐to‐noise ratio (SNR) settings. However, their accuracy decreases at SNRs <4. We also use Kalman filtering, a form of Bayesian inference, on backprojected locations of these sources to recover locations that generally fall within standard smartphone Global Positioning System errors. By combining machine learning and Kalman filter results, we calculate a multidimensional model of moving anthropogenic sources. These results demonstrate the potential of DAS data in urban seismology for accurately identifying and locating such sources. Depending on the research objectives, these sources can be further studied or filtered out to improve the quality of seismic data for earthquake studies. Such methods provide a valuable tool for urban seismology and seismic hazard analysis.

Luckie, Thomas William [Sandia National Laboratori

Analyzing Tradeoffs Associated with the Manufacture of Refined Fuels by Comparing Volatile Organic Compound (VOC) Emissions Model Results with Carbon Impacts

A first-of-its kind, publicly-available model for estimating emissions of volatile organic compounds from operations along the supply chain of liquid fuels and refinery products is presented and demonstrated. This standardized model permits comprehensive, reproducible, and comparable evaluation of the supply chain/life cycle VOC emissions of liquid fuels and refinery products.

09 BIOMASS FUELS

Development of Automated Atom Probe Tomography capability to study the influence of applied voltage and laser power on the final apparent composition of the analyzed specimen

This study presents the development and implementation of an autonomous Bayesian optimization (BO) framework for controlling and optimizing experimental parameters in Atom Probe Tomography (APT). Using commercial silicon needle samples as a benchmark system, we demonstrate that BO can efficiently navigate the complex parameter space of voltage and laser power to achieve target charge state ratios (specifically Si + /(Si + +Si 2+ )) with minimal experimental evaluations. Our implementation integrates Gaussian Process modeling with the CAMECA atom probe control framework, enabling autonomous adjustment of experimental conditions in real-time. Results show that the algorithm successfully converges to target ratios under different scenarios: maintaining a reference ratio, increasing the ratio (favoring Si 1+ ), and decreasing the ratio (favoring Si 2+ ). The system adapts to specimen evolution during analysis, compensating for changes in apex geometry while maintaining optimization targets. This work establishes a proof of concept for AI-driven optimization in APT, addressing the traditional challenges of manual parameter tuning and paving the way for applications to more complex materials where compositional accuracy is critical.

36 MATERIALS SCIENCE

Unlocking the Tight Oil Reservoirs of the Powder River Basin, Wyoming

The project focused on detailed geologic characterization, geomechanical studies, well completion optimization, stimulation monitoring, and field development strategies. A key aspect involved partnerships with industry and academic collaborators such as Occidental Petroleum, Southern Illinois University, Britt Rock Mechanics and Piri Technologies. Data acquisition included drilling, logging, coring, deployment of fiber optics, and microseismic monitoring. The project emphasized feedback loops for continuous model updating and integration of economic evaluations to guide development strategies.

02 PETROLEUM

Artificial Intelligence Benchmarking

AI benchmarking is a method for evaluating the effectiveness of an AI model using a set of standardized metrics, for example, high school-level math exams. These benchmarks and their results will enable ranking various AI models based on their effectiveness in performing a specific task.

Krishnan, Anjay [Fermilab]

Robust Explanations using Diverse Adversarially Trained Ensembles, Multi-Modal Contrastive Learning, and Attribution-based Confidence Metrics

The primary objective of this project is to strengthen the trustworthiness of AI systems by designing algorithms that make their internal decision-making processes more understandable to human users. This involves creating clear, interpretable explanations for AI decisions and developing metrics to assess these explanations' validity and reliability. Significant progress has been achieved through (i) developing symbolic explanations, (ii) generating meaningful interpretive insights, (iii) establishing accuracy and confidence metrics, and (iv) devising methods to evaluate the knowledge boundaries of AI models. To date, the research findings have been shared in peer-reviewed publications, with accompanying scientific and technical information (STI) detailed below.

97 MATHEMATICS AND COMPUTING

Superstructure Optimization for Brine Valorization from Brackish Water Desalination

This poster presents preliminary results from a superstructure optimization framework developed to identify cost-optimal brine valorization configurations for brackish water desalination plants across diverse U.S. regional feed chemistries. The study uses brackish groundwater compositions from Arizona, California, Florida, New Mexico, and Texas. Using Pyomo Generalized Disjunctive Programming (GDP) within the WaterTAP modeling environment, the optimization framework simultaneously evaluates thousands of candidate treatment configurations, spanning nanofiltration, reverse osmosis, and chemical precipitation, to minimize the levelized cost of water (LCOW) while meeting water recovery targets and product recovery constraints. Results across eight representative feed clusters demonstrate water recovery rates of 57–87% and net LCOW values ranging from -$0.032/m³ (net revenue-positive) to $0.80/m. Notably, no single process configuration was optimal across all feed types, underscoring the necessity of feed-specific optimization. Products targeted include calcium carbonate (CaCO₃) at $0.01/kg and sodium chloride (NaCl) at $0.10/kg, both at 95% purity, with product revenues offsetting treatment costs in several scenarios. The work advances NAWI's process systems engineering capabilities for multi-configuration screening.

58 GEOSCIENCES

A Demonstration System for Geological Thermal Energy Storage of Concentrating Solar Thermal

Energy storage is increasingly necessary as variable energy technologies are deployed. Seasonal energy storage can shift energy generation from the summer to the winter, but these technologies must have extremely large energy capacities and low costs. Geological Thermal Energy Storage (GeoTES) is proposed as a solution for long-term energy storage [1]. Excess thermal energy can be stored in permeable reservoirs such as aquifers and depleted hydrocarbon reservoirs for several months. The energy capacity cost of GeoTES is very low which makes it suitable for both daily- and seasonal storage of Concentrating Solar Thermal (CST) energy, thus enabling CST to provide value to electricity markets and thermal energy off-takers. A CST-GeoTES demonstration system has been funded by the U.S. Department of Energy, Solar Energy Technology Office. In this article, we will describe this demonstration system and progress that has been made in its development. The demonstration system will comprise a 2 MWth parabolic trough with an 8m aperture developed by Gossamer Space Frames, seven wells, and a 100 kWe power cycle. The demonstration system will be deployed in Kern County, California by Premier Resource Management. A techno-economic model for CST-GeoTES systems has also been developed [3] and is applied to the demonstration system and its planned future expansion. The model integrates the output of specialist models of each subsystem, which enables the performance and cost of both the subsurface and surface systems to be captured. Off-design models enable the performance to be evaluated at each hour of the year, before being aggregated to evaluate the economic potential of CST-GeoTES. Initial analysis indicates that CST-GeoTES can provide long duration energy storage capabilities with low marginal costs of energy capacity - leading to a low value of Levelized Cost of Storage (LCOS) compared to alternative technologies, see Figure 2. In this article, we investigate the cost and performance of the specific demonstration site being developed by PRM and explore a range of operational profiles that can deliver different value streams, such as daily and seasonal storage, capacity, and resiliancy.

14 SOLAR ENERGY

Marine Hydrogen Demonstration

This report summarizes Phase 1 of a project involving the design of a Floating Hydrogen Production and Dispensing Barge destined for the Port of San Francisco (SF). The H 2 Barge is designed to produce renewable H 2 at the rate of ~ 530 kg/day, storing 512 kg of hydrogen at 517-bar, allowing fast refueling of hydrogen fuel cell vessels and land-side hydrogen delivery trailers for distribution into the nascent SF hydrogen ecosystem. The broader considerations that impacted the H2 Barge design are also described. An account is given of a new review process formulated by the United States Coast Guard (USCG) to review this first-of-its-kind maritime implementation of hydrogen technology. The immediate goals of the H 2 Barge Project are to 1) demonstrate the feasibility, viability and methods of hydrogen production, storage and fueling in a maritime context, 2) help shape (where needed) and navigate the required local, state and federal regulatory gauntlet and 3) catalyze a “green hydrogen ecosystem” (both marine and landside) with locally produced renewable hydrogen at the San Francisco waterfront. A summary is also given of the modeling and experimental activity of Phase 1 directed to the development of science-based refueling protocols for large marine Type IV 250-bar hydrogen tanks. Combined modeling and experimental studies are reported of the filling of large (28 kg) 250-bar Type IV hydrogen tanks of the type being deployed on early hydrogen ferries, such as the MV Sea Change. The primary question was to determine how such tanks can be successfully filled (state of charge greater than 97%) within 45 minutes without exceeding the 82 °C temperature limit historically set for such tanks. The studies show that a gas injector is needed avoid thermal stratification during filling which can result in potential hot spots. Pre-cooling of the hydrogen was found to be essential in most cases, as ambient conditions greatly affect the need for a pre-cooling to achieve the 45-minute fill time. Pre-cooling cannot be supplied by nearby water, such as that found in nature (bays, lakes, rivers, etc.) because pre-cooling cooling below 0 °C was found to be necessary to avoid excessive compression heating. The experimental results afforded a calibration of the engineering model (SOFIL) for these large 250-bar tanks, which now enables using SOFIL to predict volume-averaged hydrogen filling temperatures to an accuracy of +/- 2.7°C for these tanks. The model can therefore be used to evaluate potential scenarios for development of a standardized fueling methodology for ferries utilizing large Type-IV tanks of the type examined here.

08 HYDROGEN

A Multiphysics Evaluation of Annular Uranium-Zirconium Metallic Fuels [Poster]

This study examines the performance of U-10Zr annular metallic fuel rodlets which were experimentally evaluated as part of the Advanced Fuels Campaign (AFC). The AFC mission is to develop novel fuel technologies and facilitate the implementation of those technologies by industry partners. A key objective is to improve steady-state and transient performance over current fuel types. The experiments of interest in this study included annular metallic U-Zr fuel rodlets within HT-9 cladding which were placed in SS-316 capsules and inserted in the Advanced Test Reactor (ATR). Certain mechanical and thermal conditions cannot be directly evaluated through experiments and fuel performance modeling is used to shed light on this evolution over time. In this study, BISON Multiphysics simulations are leveraged to investigate the state of the fuel system throughout and after the experimental conditions.

11 - NUCLEAR FUEL CYCLE AND FUEL MATERIALS

Biomass burning aerosol radiative effects in the Southeast Atlantic depend strongly on meteorological forcing method

Biomass burning aerosols (BBAs) from African fires may strongly impact Earth’s radiation budget in the southeast Atlantic (SEA), but the sign and magnitude of the overall radiative effect (RE) remain uncertain. Aerosol–climate models are needed to separately quantify direct, indirect, and semi-direct REs. Here, we evaluate improved simulations with the UK Met Office's Unified Model and explore REs resulting from various methods used to match observed meteorology (nudging or running forecasts reinitialized at different frequencies). REs are calculated as differences in radiative fluxes between simulations with and without smoke emissions and with and without aerosol absorption. All model setups agree on net warming for the SEA dominated by the direct effect. Simulated smoke, clouds, and the direct effect agree better with observations than previous studies using the same model, though biases in aerosol extinction and liquid water path remain. Changes in cloud droplet number concentration due to BBA self-lofting influence how cleanly we can separate cloud effects into semi-direct and indirect effects. Total RE, which remains unaffected, ranges from +3.0 to +7.9 W m −2 . The 4.9 W m −2 spread arises mainly from simulated semi-direct effects. Forecasts three days long or less probably do not allow time for plausible differences in boundary layer properties due to semi-direct effects to accumulate. Free running simulations with and without smoke accumulate differences in meteorology that are likely spurious “butterfly effects”. We recommend future research quantifying BBA REs over weeks to months to use meteorological forcing techniques that allow aerosol absorption to affect the boundary layer.

Giuffrida, Eric [Carnegie Mellon University, Pitts

Passive Energy-Saving Solutions for Clothes Dryers: A Modeling and Experimental Study for Improved Efficiency and Affordability

Dryers are integral appliances in modern households, yet their significant power consumption remains a critical challenge for reducing energy bills and upgrades. This work presents the outcomes of a comprehensive investigation aimed at lowering the operating energy costs of the dryer through modeling and experimental approaches. Specifically, the objectives of this research are twofold: (1) to reduce energy consumption without negatively impacting drying performance or time; and (2) to ensure affordability by developing retrofittable solutions characterized by low cost and a quick payback period. These strategies promise universal applicability to all dryer categories, encompassing both gas and electric models, by tackling core issues such as unnecessary heat loss and excess energy supply, both prevalent across dryer designs. Our methodology combines robust modeling frameworks and experimental validation to target energy efficiency improvements through three primary pathways: (1) effective heat loss management using ultralow-cost insulation materials tailored for dryer systems; (2) heat management across the drying cycle, facilitated by passive heat transfer mechanisms, and (3) optimization of heat supply to the load to ensure precise energy delivery, minimizing waste. These innovations are designed to seamlessly integrate with existing dryer configurations, providing a scalable and retrofittable solution that ensures affordability without requiring substantial redesigns or expensive components. Using these methods, the study demonstrates improvement opportunities in dryer energy consumption while maintaining the desired performance. The modeling component utilizes computational simulations to evaluate the thermal and energy performance of these innovations under varied operational conditions, providing foundational insights for experimental design. By addressing critical areas such as heat loss, energy recovery, and supply optimization, this work proposes impactful solutions for lowering the energy costs in domestic clothes dryers. The ultralow-cost, retrofittable nature of the proposed strategies ensures widespread adoption potential across diverse dryer categories, making significant strides toward affordable household energy practices.

Cheekatamarla, Praveen [ORNL] (ORCID:0000000248827