SEARCH · Engineering Papers

Results for “model benchmark”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 793 records · Page 44

All sources NASA NTRS DOE OSTI

DOE OSTI2023

MArVD2: a machine learning enhanced tool to discriminate between archaeal and bacterial viruses in viral datasets

Abstract Our knowledge of viral sequence space has exploded with advancing sequencing technologies and large-scale sampling and analytical efforts. Though archaea are important and abundant prokaryotes in many systems, our knowledge of archaeal viruses outside of extreme environments is limited. This largely stems from the lack of a robust, high-throughput, and systematic way to distinguish between bacterial and archaeal viruses in datasets of curated viruses. Here we upgrade our prior text-based tool (MArVD) via training and testing a random forest machine learning algorithm against a newly curated dataset of archaeal viruses. After optimization, MArVD2 presented a significant improvement over its predecessor in terms of scalability, usability, and flexibility, and will allow user-defined custom training datasets as archaeal virus discovery progresses. Benchmarking showed that a model trained with viral sequences from the hypersaline, marine, and hot spring environments correctly classified 85% of the archaeal viruses with a false detection rate below 2% using a random forest prediction threshold of 80% in a separate benchmarking dataset from the same habitats.

Vik, Dean (ORCID:000000027546899X)↗

DOE OSTI2025

Measurement of Close-in Ground Motion from an Underground Chemical Explosion

Understanding the geophysical response near an underground explosion is crucial for generating insights into the source and emplacement conditions that produce distinct observations in monitoring scenarios occurring at greater distances. Recently, Shot A of the Low Yield Nuclear Monitoring (LYNM) Physics Experiment 1 (PE1) series was conducted at the Nevada National Security Site to provide ground truth for subsurface explosion signal models. This experiment resulted in measuring near-source ground motion at distances ranging from 70 to 1000 m/kt with a 99% success rate, yielding high-fidelity knowledge of the near-field response that can serve as benchmarks for future numerical modeling and experiment planning. However, technical challenges exist in observing near-source phenomena while safeguarding sensitive data acquisition components from the detrimental effects of ground motion in the subsurface. This report outlines tools and techniques to address challenges associated with observing near-source accelerations and within the tunnel drift of the PE1 test bed. Additionally, we describe key systems designed with both modern advancements and legacy guidance to maximize the collection of high-quality ground motion data, which may be applied to constitutive and computational models, leading to new or improved understanding of the near- and far-field signals produced by underground explosions.

45 MILITARY TECHNOLOGY, WEAPONRY, AND NATIONAL DEF↗

DOE OSTI2022

Are Terrestrial Biosphere Models Fit for Simulating the Global Land Carbon Sink?

The Global Carbon Project estimates that the terrestrial biosphere has absorbed about one-third of anthropogenic CO 2 emissions during the 1959–2019 period. This sink-estimate is produced by an ensemble of terrestrial biosphere models and is consistent with the land uptake inferred from the residual of emissions and ocean uptake. The purpose of our study is to understand how well terrestrial biosphere models reproduce the processes that drive the terrestrial carbon sink. One challenge is to decide what level of agreement between model output and observation-based reference data is adequate considering that reference data are prone to uncertainties. To define such a level of agreement, we compute benchmark scores that quantify the similarity between independently derived reference data sets using multiple statistical metrics. Models are considered to perform well if their model scores reach benchmark scores. Our results show that reference data can differ considerably, causing benchmark scores to be low. Model scores are often of similar magnitude as benchmark scores, implying that model performance is reasonable given how different reference data are. While model performance is encouraging, ample potential for improvements remains, including a reduction in a positive leaf area index bias, improved representations of processes that govern soil organic carbon in high latitudes, and an assessment of causes that drive the inter-model spread of gross primary productivity in boreal regions and humid tropics. The success of future model development will increasingly depend on our capacity to reduce and account for observational uncertainties.

58 GEOSCIENCES↗

DOE OSTI2024

Resonant multiscalar production in the generic complex singlet model in the multi-TeV region

We develop benchmarks for resonant discalar production in the generic complex singlet scalar extension of the Standard Model (SM) with no additional symmetries, which contains two new scalars. These benchmarks maximize discalar resonant production modes at future 𝑝⁢𝑝 colliders: 𝑝⁢𝑝 → ℎ 2 → ℎ 1⁡ ℎ 1 , 𝑝⁢𝑝 → ℎ 2 → ℎ 1 ⁡ℎ 3 , and 𝑝⁢𝑝 → ℎ 2 → ℎ 3 ⁡ℎ 3 , where ℎ 1 is the observed SM-like Higgs boson and ℎ 2,3 are new scalars. The decays ℎ 2 → ℎ 1 ⁡ℎ 3 and ℎ 2 → ℎ 3 ⁡ℎ 3 may be the only way to discover ℎ 3 , leading to a discovery of two new scalars at once. Current LHC and projected future collider (HL-LHC, FCC-ee+HL-LHC, IL⁢C⁢500+HL-LHC) constraints on this model are used to produce benchmarks at the HL-LHC for ℎ 2 masses between 250 GeV and 1 TeV and a future 𝑝⁢𝑝 collider (FCC-hh) for ℎ 2 masses between 250 GeV and 12 TeV. We update the current LHC bounds on the singlet-Higgs boson mixing angle for these benchmarks. As the mass of ℎ 2 approaches the multi-TeV region, certain limiting behaviors of the maximum rates are uncovered due to theoretical constraints on the parameters. These limits, which can be derived analytically, are BR⁡(ℎ 2 → ℎ 1⁡ ℎ 1 ) → 0.25, BR⁡(ℎ 2 → ℎ 3 ⁡ℎ 3 ) → 0.5, and BR⁡(ℎ 2 → ℎ 1⁡ ℎ 3 ) → 0. It can also be shown that the maximum rates of 𝑝⁢𝑝 → ℎ 2 → ℎ 1 ⁡ℎ 1 and 𝑝⁢𝑝 → ℎ 2 → ℎ 3⁡ ℎ 3 approach the same value. Hence, all three ℎ 2 → ℎ 𝑖 ⁡ℎ 𝑗 decays are promising discovery modes for ℎ 2 masses at and below 𝒪⁡(1 TeV), while above 𝒪⁡(1 TeV) the decays ℎ 2 → ℎ 1 ⁡ℎ 1 and ℎ 2 → ℎ 3 ⁡ℎ 3 are more encouraging. We choose benchmark masses for ℎ 3 to produce a large range of decay signatures including multi-𝑏, multivector boson, and multi-SM-like Higgs production. As we will show, the behavior of the maximum rates leads to the surprising conclusion that in the multi-TeV region this model may be discovered in the Higgs quartet production mode via ℎ 2 → ℎ 3 ⁡ℎ 3 → 4⁢ℎ 1 decays before Higgs triple production is observed. The maximum di- and four Higgs production rates are similar in the multi-TeV range.

Extensions of Higgs sector↗

DOE OSTI2022

Attention-based convolutional capsules for evapotranspiration estimation at scale

Evapotranspiration (ET) measures the amount of water lost from the Earth's surface to the atmosphere and is an integral metric for both agricultural and environmental sciences. Understanding and quantifying ET is critical for achieving effective management of freshwater and irrigation systems. However, current ET estimation models suffer from a trade-off between accuracy and spatial coverage. In this study, we introduce our model Quench, a neural network architecture that achieves highly-accurate ET estimates over large continuous spatial extents. Quench uses our novel Attention-Based Convolutional Capsule for its neural network layers to identify areas of focus and efficiently extract ET information from satellite imagery. Benchmarks that profile our model's performance show substantive improvements in accuracy, with up to 128% increase in accuracy compared to traditional convolutional-based and process-based models. Finally, Quench also demonstrates consistent model performance over high geospatial variability and a diverse array of regions, seasons, climates, and vegetations.

54 ENVIRONMENTAL SCIENCES↗

DOE OSTI2022

Qualification of ANSI/HPS N13.1-2011 Mixing Criteria by Computational Fluid Dynamics Modeling for the 3430 Building Fan Addition and Increased Ventilation Capacity

Additional ventilation capacity has been designed for the 3430 Building filtered exhaust stack system. The updated system will increase the number of fans from two to three and include new ductwork with a larger diameter to integrate the new fan into the existing stack. Stack operations will involve running various fan combinations at any given time. The air monitoring system of the existing two-fan stack previously was found to comply with the American National Standards Institute/Health Physics Society (ANSI/HPS) N13.1-1999 standard. Full-scale, three-dimensional computational fluid dynamics (CFD) modeling was used to evaluate the modified three-fan system for compliance with the ANSI/HPS N13.1-2011 standard, which essentially is equivalent to the ANSI/HPS N13.1-1999 standard. The four mixing criteria evaluated are 1) flow angle, 2) velocity, 3) gas tracer, and 4) particle tracer. Benchmarking of the CFD modeling methodology showed good agreement with previous testing used to qualify the stack, and modeling of the existing two-fan system showed good agreement with test data collected from the 3430 Building stack. Modeling was performed to develop a suitable three-fan design. Initial modeling of the three-fan design and basic ductwork showed that flow angles and velocity uniformity were acceptable; however, the gas tracer and particle tracer mixing results were not acceptable. To meet ANSI/HPS N13.1-2011 criteria, an air blender was added to the stack design.

42 ENGINEERING↗

DOE OSTI2022

Qualification of ANSI/HPS N13.1-2011 Mixing Criteria by Computational Fluid Dynamics Modeling for the 3430 Building Fan Addition and Increased Ventilation Capacity

42 ENGINEERING↗

NASA NTRS1993

Large eddy simulation of a boundary layer with concave streamwise curvature

One of the most exciting recent developments in the field of large eddy simulation (LES) is the dynamic subgrid-scale model. The dynamic model concept is a general procedure for evaluating model constants by sampling a band of the smallest scales actually resolved in the simulation. To date, the procedure has been used primarily in conjunction with the Smagorinsky model. The dynamic procedure has the advantage that the value of the model constant need not be specified a priori, but rather is calculated as a function of space and time as the simulation progresses. This feature makes the dynamic model especially attractive for flows in complex geometries where it is difficult or impossible to calibrate model constants. The dynamic model was highly successful in benchmark tests involving homogeneous and channel flows. Having demonstrated the potential of the dynamic model in these simple flows, the overall direction of the LES effort at CTR shifted toward an evaluation of the model in more complex situations. The current test cases are basic engineering-type flows for which Reynolds averaged approaches were unable to model the turbulence to within engineering accuracy. Flows currently under investigation include a backward-facing step, wake behind a circular cylinder, airfoil at high angles of attack, separated flow in a diffuser, and boundary layer over a concave surface. Preliminary results from the backward-facing step and cylinder wake simulations are encouraging. Progress on the LES of a boundary layer on a concave surface is discussed. Although the geometry of a concave wall is not very complex, the boundary layer that develops on its surface is difficult to model due to the presence of streamwise Taylor-Gortler vortices. These vortices arise as a result of a centrifugal instability associated with the convex curvature.

Lund, Thomas S.↗

DOE OSTI2025

MLCommons Science Benchmarks

Benchmarks are a cornerstone of modern machine learning practice, providing standardized eval- uations that enable reproducibility, comparison, and scientific progress. Yet, as AI systems particularly deep learning models become increasingly dynamic, traditional static benchmarking approaches are losing their relevance. Models rapidly evolve in architecture, scale, and capability; datasets shift; and deployment contexts continuously change, creating a moving target for evaluation. Without adaptive benchmarking frame- works, both scientific assessment and real-world de- ployment risk becoming misaligned with actual system behavior. Drawing on our experience from MLCommons, educa- tional initiatives, and government programs such as the DOE s Million Parameter Consortium, we identify key barriers that hinder the broader adoption and utility of benchmarking in AI. These include substantial resource demands, limited access to specialized hardware, lack of expertise in benchmark design, and uncertainty among practitioners about how to relate benchmark results to their own application domains. Moreover, current benchmarks often emphasize peak performance on leadership-class hardware, offering limited guidance for more diverse, real-world deployment scenarios. We argue that benchmarking itself must become dy- namic in order to incorporate evolving models, updated data, and heterogeneous computational platforms while maintaining transparency, reproducibility, and inter- pretability. Democratizing this process requires not only technical innovation, but also systematic educational efforts spanning undergraduate to professional levels to develop sustained expertise in benchmark design and use. Finally, benchmarks should be framed and com- municated to support application-relevant comparisons, enabling both developers and users to make informed, context-sensitive decisions. Advancing dynamic and inclusive benchmarking practices will be essential to ensure that evaluation keeps pace with the evolving AI landscape and supports responsible, reproducible, and accessible AI deployment.

Hawks, Benjamin G. [Fermilab]↗

DOE OSTI2024

Towards interpretable Cryo-EM: disentangling latent spaces of molecular conformations

Molecules are essential building blocks of life and their different conformations (i.e., shapes) crucially determine the functional role that they play in living organisms. Cryogenic Electron Microscopy (cryo-EM) allows for acquisition of large image datasets of individual molecules. Recent advances in computational cryo-EM have made it possible to learn latent variable models of conformation landscapes. However, interpreting these latent spaces remains a challenge as their individual dimensions are often arbitrary. The key message of our work is that this interpretation challenge can be viewed as an Independent Component Analysis (ICA) problem where we seek models that have the property of identifiability. That means, they have an essentially unique solution, representing a conformational latent space that separates the different degrees of freedom a molecule is equipped with in nature. Thus, we aim to advance the computational field of cryo-EM beyond visualizations as we connect it with the theoretical framework of (nonlinear) ICA and discuss the need for identifiable models, improved metrics, and benchmarks. Moving forward, we propose future directions for enhancing the disentanglement of latent spaces in cryo-EM, refining evaluation metrics and exploring techniques that leverage physics-based decoders of biomolecular systems. Moreover, we discuss how future technological developments in time-resolved single particle imaging may enable the application of nonlinear ICA models that can discover the true conformation changes of molecules in nature. The pursuit of interpretable conformational latent spaces will empower researchers to unravel complex biological processes and facilitate targeted interventions. This has significant implications for drug discovery and structural biology more broadly. More generally, latent variable models are deployed widely across many scientific disciplines. Thus, the argument we present in this work has much broader applications in AI for science if we want to move from impressive nonlinear neural network models to mathematically grounded methods that can help us learn something new about nature.

59 BASIC BIOLOGICAL SCIENCES↗

DOE OSTI2023

Overview of Liquid-Metal PFC R&D at the University of Illinois Urbana-Champaign

The design and implementation of future flowing liquid-lithium plasma-facing components (LLPFCs) will be dependent on several factors. Of course, one of the most important is the need to be able to deal with high heat fluxes incident on the surface of the LLPFCs, but there are also several other important liquid-metal behaviors that have been identified for their critical impact on the feasibility of a LLPFC. One of these is the ability to constantly wet 100% of the plasma-facing component area and the best way to achieve that. Another key point is knowing and understanding the erosion and corrosion of the surfaces subject to a flowing liquid-lithium system and the ability for hydrogen and helium uptake by the system. The Center for Plasma Material Interactions (CPMI) has been tasked with looking at these various issues. The Mock-up Entry module for EAST device was used to investigate wetting and erosion effects and to design a suitable distribution and collection system with a liquid-lithium loop. The vapor shielding effects of lithium on the surface were also modeled and studied. A model coupling CRANE, an open-source global reaction network solver, and Zapdos, a plasma transport solver, is being developed to better understand the dynamics of the vapor cloud. Experiments on the Magnum-PSI at the Dutch Institute for Fundamental Energy Research have been carried out to study the vapor shielding effect and obtain experimental benchmarks to verify the model. Also, initial experiments using the Hybrid Illinois Device for Research and Applications have been performed to understand the pumping effects of lithium on helium. Experiments with a drop of liquid lithium (~100 mg) into a helium plasma have shown the ability of lithium to take out the cold recycling helium gas as well as hydrogen and oxygen impurity gases. The improvement in plasma performance was significant, and further understanding of this effect will have impacts on how future LLPFCs will be designed. Further investigation into the exact mechanism for helium pumping by lithium needs to be performed in the future. Finally, this paper presents a summary of the results obtained at the CPMI.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗

DOE OSTI2023

FY23 Progress on Computational Modeling of the Water-Based NSTF

This report summarizes the system-level modeling effort by Argonne National Laboratory (Argonne) of the Natural convection Shutdown heat removal Test Facility (NSTF) in FY23. As a continuation of the modeling effort from FY22, this year’s work focuses on improving the RELAP5-3D model developed previously for two-phase flow simulations. The RELAP5-3D model is updated to more accurately capture the heat loss experienced by the facility. The updated model is compared against experimental data for benchmarking purposes of the RELAP5-3D input model. By correctly accounting for heat loss, the updated RELAP5-3D model can now predict the two-phase baseline case more accurately. The onset and the duration of instability are captured well by the model. Furthermore, analyses are performed to better understand the instability mechanism experienced by the flow where the expansion of the boiling boundary in the chimney is studied in details and the fundamental frequencies of the oscillations are obtained. The updated RELAP5-3D model is further compared against four fault conditions, namely the reduction of riser header inlet flow area, depletion of system inventory, blocked riser channels, and static boiling scenario. For each fault condition, minor modifications and tuning are performed to improve the predictions of the model. The purpose of the analyses is to investigate the capability of RELAP5-3D in predicting complex two-phase flows in possible accident scenarios in actual Reactor Cavity Cooling System (RCCS). Overall, the model is able to capture the behaviors and trends of these fault conditions relatively well. Some discrepancies remain between the experimental data and the predictions, many of which are likely due to the differences in the predicted and experimental vapor generation rate. Future work will focus on the continued development of the current RELAP5-3D input model of the NSTF to both improve the accuracy of the model’s predictive capability and continue supporting the experimental program needs. The mutually beneficial relationship between analysis and experimental efforts has become integral to the parent NSTF program, and the greater objective to fully understand and accurately predict the heat removal performance of a full scale RCCS concept.

21 SPECIFIC NUCLEAR REACTORS AND ASSOCIATED PLANTS↗

NASA NTRS1995

Performance modeling & simulation of complex systems (A systems engineering design & analysis approach)

Modeling of the Multi-mission Image Processing System (MIPS) will be described as an example of the use of a modeling tool to design a distributed system that supports multiple application scenarios. This paper examines: (a) modeling tool selection, capabilities, and operation (namely NETWORK 2.5 by CACl), (b) pointers for building or constructing a model and how the MIPS model was developed, (c) the importance of benchmarking or testing the performance of equipment/subsystems being considered for incorporation the design/architecture, (d) the essential step of model validation and/or calibration using the benchmark results, (e) sample simulation results from the MIPS model, and (f) how modeling and simulation analysis affected the MIPS design process by having a supportive and informative impact.

Hall, Laverne↗

DOE OSTI2025

Achieving GPT-4o level performance in astronomy with a specialized 8B-parameter large language model

AstroSage-Llama-3.1-8B is a domain-specialized natural-language AI assistant tailored for research in astronomy, astrophysics, cosmology, and astronomical instrumentation. Trained on the complete collection of astronomy-related arXiv papers from 2007 to 2024 along with millions of synthetically-generated question-answer pairs and other astronomical literature, AstroSage-Llama-3.1-8B demonstrates remarkable proficiency on a wide range of questions. AstroSage-Llama-3.1-8B scores 80.9% on the AstroMLab-1 benchmark, greatly outperforming all models—proprietary and open-weight—in the 8-billion parameter class, and performing on par with GPT-4o. This achievement demonstrates the potential of domain specialization in AI, suggesting that focused training can yield capabilities exceeding those of much larger, general-purpose models. AstroSage-Llama-3.1-8B is freely available, enabling widespread access to advanced AI capabilities for astronomical education and research.

AI assistant↗

DOE OSTI2024

Capturing Infrastructure Interdependencies for Power Outages Prediction During Extreme Events

As extreme weather events such as hurricanes, severe thunderstorms, and floods grow in frequency and intensity, the disruption of power grid systems poses significant challenges, including widespread electrical outages, economic losses, and threats to public safety. This paper presents a forward-looking approach that leverages geographical graph-based machine learning models to predict county-level maximum power outages during such events. By capturing the intricate interdependencies within power system networks, our approach aims to provide precise and actionable predictions that can optimize emergency response efforts and enhance grid resilience. Through the integration of real-world data, including hurricane advisories and power outage records, we have trained and benchmarked multiple machine learning models, demonstrating the feasibility and potential of this method. While our initial results are promising, this paper also charts a course for advancing these models, addressing the remaining challenges, and ultimately transforming how we anticipate and respond to the impacts of extreme weather on power systems.

Lee, Sangkeun (Matt) [ORNL] (ORCID:000000021317511↗

DOE OSTI2022

Total thermal neutron cross section measurements of hydrogen dense polymers from 0.0005–20 eV

Hydrogen dense polymers, specifically polyethylene, polystyrene, and Plexiglas, have served as neutron moderator and reflector materials in hundreds of separate critical benchmark experiments because of their low cost and abundance of hydrogen. In order to accurately model and simulate these critical benchmarks, the thermal scattering law (TSL) evaluation that governs how neutrons will thermalize must be well understood and rigorously validated. To support this validation, researchers at Rensselaer Polytechnic Institute performed total neutron cross section measurements for high-density polyethylene & polystyrene over the energy range 0.0005–20 eV and for Plexiglas G & Plexiglas G-UVT over the energy range 0.0005–3 eV. Comparisons were made between the measured cross section and that predicted by the ENDF/B-VIII.0 and Oak Ridge National Laboratory/European Spallation Source/Rensselaer Polytechnic Institute TSL evaluations for polyethylene, Plexiglas, and polystyrene, as available. These experiments represent the first total neutron cross section measurements for polystyrene.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗

NASA NTRS1987

Albedos of homogeneous semi-infinite canopies - Comparison of two-stream analytic and numerical solutions

The albedo of plant canopies is treated as a problem in radiative transfer. Albedos calcualted from an iterative multistream numerical model are compared with those calculated with an analytic two-stream solution. With the assumption of a randomly homogeneous distribution of leaf positions and orientations and isotropic scattering by individual leaves, the single-scattering albedo of the canopy can be found analytically. This single-scattering solution is incorporated into the two-stream solution and used to benchmark the multistream numerical model in the single-scattering limit. Relative errors so established in the multistream model are O(0.3 percent) or less. The two-stream model is also found to be remarkably accurate, with the error in multiply scattered radiation O(5 percent) or less, corresponding to absolute errors in visible albedo of less than 0.001 and near-infrared albedo of less than or equal to 0.01. Thus the two-stream model should be adequate for many purposes, such as climate modeling, provided the assumptions of homogeneous canopy and isotropic scattering are not too unrealistic.

Dickinson, Robert E.↗

NASA NTRS1992

Analysis of an RNG based turbulence model for separated flows

A two-equation turbulence model of the K-epsilon type was recently derived by using Renormalization Group (RNG) methods. It was later reported that this RNG based model yields substantially better predictions than the standard K-epsilon model for turbulent flow over a backward facing step - a standard test case used to benchmark the performance of turbulence models in separated flows. The improvements obtained from the RNG K-epsilon model were attributed to the better treatment of near wall turbulence effects. In contrast to these earlier claims, it is shown in this paper that the original version of the RNG K-epsilon model substantially underpredicts the reattachment point in the backstep problem. This is a deficiency that is traced to the modeling of the production of dissipation term. However, with the most recent improvements in the RNG K-epsilon model, excellent results for the backstep problem are now obtained.

Speziale, C. G.↗