Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Performance benchmark”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 775 records · Page 43
All sourcesNASA NTRSDOE OSTI
DOE OSTI2020

Report on G4-Med, a Geant4 benchmarking system for medical physics applications developed by the Geant4 Medical Simulation Benchmarking Group

Geant4 is a Monte Carlo code extensively used in medical physics for a wide range of applications, such as dosimetry, micro- and nanodosimetry, imaging, radiation protection, and nuclear medicine. Geant4 is continuously evolving, so it is crucial to have a system that benchmarks this Monte Carlo code for medical physics against reference data and to perform regression testing. In this work, to respond to these needs, we developed G4-Med, a benchmarking and regression testing system of Geant4 for medical physics. G4-Med currently includes 18 tests. They range from the benchmarking of fundamental physics quantities to the testing of Monte Carlo simulation setups typical of medical physics applications. Both electromagnetic and hadronic physics processes and models within the prebuilt Geant4 physics lists are tested. The tests included in G4-Med are executed on the CERN computing infrastructure via the use of the geant-val web application, developed at CERN for Geant4 testing. The physical observables can be compared to reference data for benchmarking and to results of previous Geant4 versions for regression testing purposes. This paper describes the tests included in G4-Med and shows the results derived from the benchmarking of Geant4 10.5 against reference data.

60 APPLIED LIFE SCIENCES↗
DOE OSTI2025

Depletion Benchmark of the AFIP-7 Experiment in the Advanced Test Reactor

Reactor physics depletion benchmarks for low-enriched uranium fuel are limited in number. In particular, there is very limited data for LEU benchmarks for U-10Mo (Uranium-10% Molybdenum) plate fuel developed for use in U.S. high-performance research reactors (USHPRR). USHPRR includes the Advanced Test Reactor (ATR), Advanced Test Reactor Critical Facility (ATR-C), High Flux Isotope Reactor (HFIR), University of Missouri Research Reactor (MURR), Massachusetts Institute of Technology Reactor (MITR), and National Bureau of Standards Reactor (NBSR) at the National Institute of Science and Technology. These reactors are fueled with high-enriched uranium dispersed fuel in a silicon/aluminum matrix. In support of conversion to a HALEU fuel, qualification of U-10Mo formed into a monolithic foil is being performed. Fuel qualification involves irradiated fueled specimens in the ATR. The irradiation tests provide an opportunity to benchmark depletion capabilities of reactor physics codes in support of the ATR operation, as well as develop benchmarks that can be used by other institutions to benchmark other reactor physics codes. This report documents the development of a benchmark model of the irradiation of the ATR Full -size plate In center flux trap Position 7 (AFIP-7) experiment.

11 - NUCLEAR FUEL CYCLE AND FUEL MATERIALS↗
DOE OSTI2025

High-Fidelity CFD Assessments of Flow Resistance in a 61-Pin Wire-Wrapped Assembly with Partially Blocked Channels

The examination of thermal-hydraulic behaviors in wire-wrapped rod bundles continues to be an active area of research. The sodium fast reactor, a prominent candidate in next-generation nuclear designs, utilizes a hexagonal configuration of wire-wrapped fuel pins. Here, the potential for channel blockage within this compact arrangement poses a significant safety challenge, spurring a number of recent experimental and computational investigations to evaluate its impact on flow and heat transfer. The present work aims to benchmark the high-fidelity NekRS computational fluid dynamics (CFD) solver in predicting the pressure drops associated with substantial blockages, using available experimental data as a reference. A 61-pin wire-wrapped fuel assembly with two flow blockage configurations has been simulated and investigated at a range of low to moderate Reynolds numbers (487 ≤ Re ≤ 14 600). The NekRS solver demonstrates an exponential reduction of spatial discretization error with increasing polynomial order. The high level of agreement between the numerical results and measured data confirms the accuracy and consistency of the present numerical approach. This benchmark study establishes the capability of NekRS to perform reliable hydrodynamic simulations for sodium fast reactor applications and supports its use in design, licensing, and safety analyses.

CFD Benchmarking↗
DOE OSTI2020

Benchmark probabilistic solar forecasts: Characteristics and recommendations

We illustrate and compare commonly used benchmark, or reference, methods for probabilistic solar forecasting that researchers use to measure the performance of their proposed techniques. A thorough review of the literature indicates wide variation in the benchmarks implemented in probabilistic solar forecast studies. To promote consistent and sensible methodological comparisons, we implement and compare ten variants from six common benchmark classes at two temporal scales: intra-hourly forecasts and hourly resolution forecasts. Using open-source Surface Radiation Budget Network (SURFRAD) data from 2018, these benchmark methods are compared using proper probabilistic metrics and common diagnostic tools. Practical implementation issues, such as the impact of missing data and applicability for operational forecasting, are also discussed. Furthermore, we make recommendations for practitioners on the appropriate selection of benchmark methods to properly showcase state-of-the-art improvements in forecast reliability and sharpness. All code and open-source data are available on Github for reproducibility and for other researchers to apply the same benchmark methods to their own data.

14 SOLAR ENERGY↗
DOE OSTI2022

IER-501 CED-3b: Experiment Execution Summary for the Pulsed-Neutron Die-Away Experimental Testbed for Thermal Scattering Law Benchmarks (PNDA)

This report summarizes the experiments done for IER-501, a testbed for pulsed-neutron die-away (PNDA) experiments at Lawrence Livermore National Laboratory (LLNL). The PNDA experiments were conducted in two separate two-day campaigns: one campaign for high-density polyethylene and another for Lucite. All experiments were performed at Lawrence Livermore Laboratory. They will become high-quality benchmarks that serve to optimize and validate thermal neutron scattering laws (TSLs), which are high priority nuclear data for the Department of Energy’s Nuclear Criticality Safety Program (NCSP). The report presents the PNDA design and its equipment, and it documents the experimental die-away curves. It presents the characterization measurements that have been performed to-date. Importantly, mass spectrometry measurements to determine sample impurities were not yet completed. These will be included in the final benchmark. The report gives the fitted decay constants for the die-away curves of each target sample. It also provides the data for the die-away curves in the appendices. The HDPE experiments examined twelve targets of varying size. The Lucite measurements included ten targets of varying size. Experimenters in the campaign were Daniel Siefman, William Zywiec, and Ruby Araj.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗
DOE OSTI2025

A quantitative figure of merit for battery SEI films and their use as functional solid-state electrolytes

As a key passivation film that governs battery operation, the solid electrolyte interphase (SEI) has long been credited for enabling high-performance batteries or blamed for their eventual death. However, qualitative descriptions of the SEI often found in the literature (e.g., “conductive,” “passivating”) highlight our incomplete understanding of this layer, where even the most basic properties foundational to SEI function remain difficult to measure. Here, we quantify SEI conductivities and SEI transference numbers using a separator-free Cu|SEI|Li architecture that treats the SEI as a functional solid-state electrolyte (SSE). We find that while any SEI property alone (e.g., electronic conductivity) is weakly correlated (R 2 < 0.67) with battery performance (e.g., Coulombic efficiency), a strong correlation (R 2 > 0.99) can be achieved by defining the “SEI cT number” as a product between the SEI transference number (T) and the ratio of SEI conductivities (c). Analogous to the thermoelectric figure of merit (i.e., zT ), SEI cT quantitatively benchmarks the holistic impact of SEI properties on battery performance and underscores the pitfalls of citing such properties in isolation. Perhaps most strikingly, we demonstrate that Li metal deposition and stripping at room temperature is possible in our separator-free Cu|SEI|Li cell, confirming that the SEI can function precisely as an SSE. Together, these results enrich our understanding of the SEI, not just as a passivation layer but as a functional structure that can potentially have important implications for solid-state batteries.

Science & Technology - Other Topics↗
DOE OSTI2022

Runtime Systems for Energy Efficiency in Advanced Computing Systems

As heterogeneous systems become increasingly popular for both mobile and high-performance computing, conventional efficiency techniques such as dynamic voltage and frequency scaling (DVFS) fail to account for the tightly coupled and varied nature of systems on a chip (SoCs). In this work, we explore the impact of system unaware DVFS techniques on a mobile SoC under three benchmark suites: Chai, Rodinia, and Antutu. We then analyze performance trends across the suites to identify a set of consistent operating points that optimally balance power and performance across the system. The consistent operating points are then constructed into a dependency graph which can be leveraged to produce a more effective, SoC-wide governor.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2020

Effectively Using Remote I/O For Work Composition in Distributed Workflows

Distributed scientific workflows are becoming more important with the interest in incorporating AI into their loops. A critical programming and performance question is how to compose workflow tasks when data is produced on one system but must be consumed on another. Since the dominant technique is composition with remote I/O, this paper explores its performance expectations. We describe BigFlowSim, a workflow I/O simulator that captures key implementation choices for remote I/O, including intensity, reuse, locality, access pattern, and data movement.With BigFlowSim, we generate a synthetic benchmark. We quantify the effects of each parameter with a performance sensitivity study. We explain trends in terms of data movement reduction and show that, under certain conditions, it is possible to establish a total order among most parameters. We apply these insights to a high energy physics workflow, Belle II Monte Carlo and simulate several I/O optimizations. Speedups range from 5% to 2×, without changing compute time.

Friese, Ryan D.↗
DOE OSTI2020

Benchmarking materials property prediction methods: the Matbench test set and Automatminer reference algorithm

Abstract We present a benchmark test suite and an automated machine learning procedure for evaluating supervised machine learning (ML) models for predicting properties of inorganic bulk materials. The test suite, Matbench, is a set of 13 ML tasks that range in size from 312 to 132k samples and contain data from 10 density functional theory-derived and experimental sources. Tasks include predicting optical, thermal, electronic, thermodynamic, tensile, and elastic properties given a material’s composition and/or crystal structure. The reference algorithm, Automatminer, is a highly-extensible, fully automated ML pipeline for predicting materials properties from materials primitives (such as composition and crystal structure) without user intervention or hyperparameter tuning. We test Automatminer on the Matbench test suite and compare its predictive power with state-of-the-art crystal graph neural networks and a traditional descriptor-based Random Forest model. We find Automatminer achieves the best performance on 8 of 13 tasks in the benchmark. We also show our test suite is capable of exposing predictive advantages of each algorithm—namely, that crystal graph methods appear to outperform traditional machine learning methods given ~10 4 or greater data points. We encourage evaluating materials ML algorithms on the Matbench benchmark and comparing them against the latest version of Automatminer.

36 MATERIALS SCIENCE↗
DOE OSTI2024

SCITUNE: Aligning Large Language Models with Human-Curated Scientific Multimodal Instructions

Instruction finetuning is a popular paradigm to align large language models (LLM) with human intent. Despite its popularity, this idea is less explored in improving the LLMs to align existing foundation models with scientific disciplines, concepts and goals. In this work, we present SciTune as a tuning framework to improve the ability of LLMs to follow scientific multimodal instructions. To test our methodology, we use a human-generated scientific instruction tuning dataset and train a large multimodal model LLaMA-SciTune that connects a vision encoder and LLM for science-focused visual and language understanding. LLaMA-SciTune significantly outperforms the state-of-the-art models in the generated figure types and captions in multiple scientific multimodal benchmarks. In comparison to the models that are fine-tuned with machine generated data only, LLaMA-SciTune surpasses human performance on average and in many sub-categories on the ScienceQA benchmark.

• Artificial intelligence (AI) / machine learning ↗
DOE OSTI2022

Benchmarking Autonomous Scattering Experiments Illustrated on TAS

With the advancement of artificial intelligence and machine learning methods, autonomous approaches are recognized to have great potential for performing more efficient scattering experiments. In our view, it is crucial for such approaches to provide thorough evidence about respective performance improvements in order to increase acceptance within a scientific community. Therefore, we propose a benchmarking procedure designed as a cost-benefit analysis that is applicable to any scattering method sequentially collecting data during an experiment. For a given approach, the performance assessment is based on how much benefit, given a certain cost budget, it is able to acquire in predefined test cases. Different approaches thus get a chance for comparison and can make their advantages explicit and visible. Key components of the procedure, i.e., cost measures, benefit measures, and test cases, are made precise for the setting of three-axes spectrometry (TAS) as an illustration. Finally, we discuss neglected aspects and possible extensions for the TAS setting and comment on the procedure’s applicability to other scattering methods. A Python implementation of the procedure to simplify its utilization by interested researchers from the field is also provided.

36 MATERIALS SCIENCE↗
DOE OSTI2022

Assessment of four strain energy decomposition methods for phase field fracture models using quasi-static and dynamic benchmark cases

Abstract Strain energy decomposition methods in phase field fracture models separate strain energy that contributes to fracture from that which does not. However, various decomposition methods have been proposed in the literature, and it can be difficult to determine an appropriate method for a given problem. The goal of this work is to facilitate the choice of strain decomposition method by assessing the performance of three existing methods (spectral decomposition of the stress or the strain and deviatoric decomposition of the strain) and one new method (deviatoric decomposition of the stress) with several benchmark problems. In each benchmark problem, we compare the performance of the four methods using both qualitative and quantitative metrics. In the first benchmark, we compare the predicted mechanical behavior of cracked material. We then use four quasi-static benchmark cases: a single edge notched tension test, a single edge notched shear test, a three-point bending test, and a L-shaped panel test. Finally, we use two dynamic benchmark cases: a dynamic tensile fracture test and a dynamic shear fracture test. All four methods perform well in tension, the two spectral methods perform better in compression and with mixed mode (though the stress spectral method performs the best), and all the methods show minor issues in at least one of the shear cases. In general, whether the strain or the stress is decomposed does not have a significant impact on the predicted behavior.

Zhang, Shuaifang↗
DOE OSTI2026

Intercomparison of flood inundation models across land use types and hydrological flood stages

Flood Inundation Mapping (FIM) model selection is a key operational decision because accurate, rapid mapping underpins early warning and resource allocation. FIM performance is context-dependent and can vary with hydrograph phase, land-use/land-cover (LULC), and the evaluation benchmark. Intercomparison studies typically assess a single near-peak snapshot against one reference dataset. Here, we provide a context-stratified intercomparison across (i) multiple hydrograph phases, (ii) LULC classes, and (iii) benchmark types, for five FIM approaches spanning a wide range of physical complexity and operational cost (TRITON, LISFLOOD-FP, HEC-RAS 2D, ARC-Curve2Flood, and OWP HAND-FIM). We use the Hurricane Matthew flood (2016) in the Neuse River Basin, North Carolina, USA, as a case study. Using high-resolution remote sensing-derived flood inundation maps, hand-labeled points, and building footprints, we assess model skill across two rising and two falling hydrograph limbs and across major LULC types. Results show that model rankings shift systematically across contexts: LISFLOOD-FP ranks highest in three of four flood phases, while TRITON leads during one rising limb phase; LISFLOOD-FP performs best in vegetated areas, whereas HEC-RAS improves relative performance in agricultural and urban areas; and benchmark choice influences conclusions, with LISFLOOD-FP performing best for flooded-building detection in the late falling limb, while TRITON ranks highest against hand-labeled points. We also report representative wall-clock runtimes for each workflow to provide use-case context for operational feasibility. Together, these results offer transferable guidance for model selection and for designing large-scale, benchmark-aware FIM intercomparison studies.

Nikrou, Parvaneh [University of Alabama]↗
DOE OSTI2021

Nuclear Data and Cross Section Testing Using ENDF/B-VIII.0

With the release of the Evaluated Nuclear Data File (ENDF)/B-VIII.0 library, nuclear criticality safety practitioners and engineers have access to the latest cross section sets available for their analyses. However, these cross sections must be rigorously tested and validated to ensure that the nuclear data are responsive to the needs of the individuals responsible for developing, implementing, and maintaining computational tools for criticality safety applications. Thus, the ENDF/B-VIII.0 library is tested and validated with a large collection of experiments that were vetted by the International Criticality Safety Benchmark Evaluation Project and made available in the International Handbook of Evaluated Criticality Safety Benchmark Experiments. A selection of benchmark experiments for use within the criticality safety community were prepared and reviewed within the Verified, Archived Library of Inputs and Data (VALID), which is maintained by the Nuclear Energy and Fuel Cycle Division at Oak Ridge National Laboratory. The performance of the ENDF/B-VIII.0 library is assessed by using VALID models of benchmark experiments with the beta 12 version of SCALE 6.3 KENO V.a and KENO-VI Monte Carlo codes. The performance is compared with the results obtained from with the ENDF/B-VII.1 library. This report considers multigroup (MG) and continuous energy (CE) formats of the ENDF/B-VIII.0 and -VII.1 libraries. The benchmark experiments within VALID that validate the ENDF/B-VIII.0 library cover 15 broad system categories by using a range of fissile materials, uranium enrichments, plutonium isotopic vectors, and mixed uranium/plutonium systems. These forms are represented as metals, solutions, or various arrays of rods or plates that cover a variety of neutron energy spectra: thermal, fast, mixed, and intermediate. Over 600 cases were considered for use with the KENO V.a and KENO-VI codes with the ENDF/B-VIII.0 library. The results of the Monte Carlo comparison of ENDF/B-VIII.0 to ENDF/B-VII.1 with both KENO V.a and KENO-VI indicate that there is a less than 0.53% Δk difference between the bias of calculated k eff from the expected values. The CE ENDF/B-VIII.0 library results in smaller magnitude biases than the ENDF/B-VII.1 data for HEU-MET-FAST, HEU-SOL-THERM, IEU-MET-FAST, LEU-SOL-THERM, PU-SOL-THERM, and U233-MET-FAST systems, while the MG results yielded smaller magnitude biases for HEU-MET-FAST, HEU-SOL-THERM, IEU-MET-FAST, LEU-COMP-THERM, LEU-SOL THERM, MIX-COMP-FAST, and U233-MET-FAST systems. Most notable are the adjustments to the plutonium and 233 U cross section data, which has resulted in noticeably lower biases in the ENDF/B VIII.0 results for the mixed, plutonium, and 233 U systems. Results of the sensitivity data file comparison generated from TSUNAMI-3D for selected VALID cases for the ENDF/B-VIII.0 library indicate a very high level of agreement with correlation coefficients of the effect of nuclear data uncertainty on k eff (the c k integral parameter) all above 0.99. This indicates that cases with the ENDF/B-VIII.0 library would see very similar responses to any nuclear data errors or change as those with the ENDF/B-VII.1 library.

11 NUCLEAR FUEL CYCLE AND FUEL MATERIALS↗
DOE OSTI2023

CLC: A cross-level program characterization method

Characterization of program execution plays a key role in performance improvement. There are numerous transformations applied to each step that a program takes on its lowering from source code to a compiler intermediate representation to machine language to microarchitecture-specific execution. The unpredictable benefit of each transformation step could lead a notionally superior algorithm to exhibit inferior performance once actually run, and it can be hard to discern which step in the transformation path contradicted the code developer’s assumptions. Conventional approaches to program-execution characterization consider the behavior after only a single one of those steps, which limits the information that can be provided to the user. To help address the issue of myopic views of program execution, this paper presents a novel cross-level characterization approach for understanding the behavior of program execution at different levels in the process of writing, compiling, and running a program. We show that this approach provides a richer view of the sources of performance gains and losses and helps identify program execution in a more accurate manner.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2024

Bringing HPE Slingshot 11 support to Open MPI

The Cray HPE Slingshot 11 network is used on the new exascale systems arriving at the U.S. Department of Energy (DoE) laboratories (e.g., Frontier, Aurora, Perlmutter). As such, the support of this network is an important capability to meet the needs of exascale applications. Here, this article highlights recent work to develop supporting infrastructure to enable Open MPI to efficiently support these new platforms. A key component of this effort involves development of a new Open Fabrics Interface (OFI) provider, LinkX. We discuss the design and development of enhancements that take advantage of the new Slingshot 11 network and AMD GPUs. We include performance data from tests on the Frontier supercomputer using synthetic communication benchmarks, and the vendor provided MPI as a baseline for comparison. The tests demonstrate full functionality of Open MPI on the system and initial results show favorable performance when compared to the highly tuned vendor implementation.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2022

Matrix-Model Simulations Using Quantum Computing, Deep Learning, and Lattice Monte Carlo

Matrix quantum mechanics plays various important roles in theoretical physics, such as a holographic description of quantum black holes, and it underpins the only practical numerical approach to the study of complex high-dimensional supergravity theories. Understanding quantum black holes and the role of entanglement in a holographic setup is of paramount importance for the realization of a quantum theory of gravity. Moreover, a complete numerical understanding of the holographic duality and the emergence of geometric space-time features from microscopic degrees of freedom could pave the way for new discoveries in quantum information science. Euclidean lattice Monte Carlo simulations are the de facto numerical tool for understanding the spectrum of large matrix models and have been used to test the holographic duality. However, they are not tailored to extract dynamical properties or even the quantum wave function of the ground state of matrix models. Quantum computing and deep learning provide potentially useful approaches to study the dynamics of matrix quantum mechanics. If successful in the context of matrix models, these rapidly improving numerical techniques could become the new Swiss army knife of quantum gravity practitioners. In this paper, we perform the first systematic survey for quantum computing and deep-learning approaches to matrix quantum mechanics, comparing them to lattice Monte Carlo simulations. These provide baseline benchmarks before addressing more complicated problems. In particular, we test the performance of each method by calculating the low-energy spectrum.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
DOE OSTI2026

Influence of the drive design on ablation front hydrodynamic instability growth in a capsule implosion at the National Ignition Facility

A critical aspect of inertial confinement fusion experiments lies in the control of ablation front instabilities during the implosion process. The growth of Rayleigh–Taylor and Richtmyer–Meshkov instabilities, seeded by target defects, can significantly degrade the performance of the implosion. Here, this study explores the influence of drive design on ablation front stability using the hydro-growth radiography platform at the National Ignition Facility. This platform allows the measurement of the ablation front hydrodynamic instability growth. Because these measurements are restricted to early convergence, their primary role is to constrain the initial instability growth and benchmark the simulations used to infer shell integrity and performance at peak velocity and ignition. Three ignition designs, Hybrid-E (HyE), SQ-n (“S” for scaling and “Q” for quality), and High temperature, High thickness (HiT) were analyzed. The results demonstrated a high dependence of the growth factor on the choice of design, capsule scale as well as the hohlraum conditions, mainly the radiative temperature and the gold M-band emission. HyE and SQ-n display a similar growth factor range, but their evolution is different due to the design differences in hohlraum conditions. HiT has the lowest growth factor of the studied designs, which is a result of a higher radiative temperature. These results highlight the importance of design choices in controlling instability dynamics. These insights inform future drive design strategies to enhance stability and efficiency in fusion ignition experiments.

Physics↗