Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “BENCHMARKS”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 469 records · Page 26
All sourcesNASA NTRSDOE OSTI
DOE OSTI2025

The Risk Assessment Information System Compendium of Ecological Screening Benchmarks for Chemicals and Radionuclides (2025) (Volume V – Chemicals for Surface Water: General)

Volume V presents the chemical benchmarks for surface water: general. The benchmarks here are applicable to both fresh water and marine or where no differentiation was made by the source. These values are not repeated in the fresh water and marine categories. Likewise, values listed in the fresh water and marine categories are not repeated in the generic categories. Consult Volume I for information on the sources.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2025

ORNL Testing of Multiple Graphite Benchmarks [Slides]

Integral criticality safety and reactor physics benchmark experiments from the International Handbook of Evaluated Criticality Safety Benchmark Experiments (ICSBEP Handbook) and the International Handbook of Reactor Physics Experiments (IRPhE Handbook) are essential for nuclear data validation and testing

ENDF↗
DOE OSTI2025

Benchmark Exercise Report for Experimental Study of Bubble Scrubbing in Water Coolant Pool

Mechanistic assessments of radionuclide release during postulated accidents are expected to be included in advanced reactor license applications. The mechanistic source term (MST) provides an opportunity for vendors to realistically evaluate the radiological consequences of an incident, and may aid in justifying reduced emergency planning zones and plant sites. However, the development of MSTs for advanced nuclear reactors is challenging because there are numerous phenomena that can affect the transport and retention of radionuclides. As part of a trial MST assessment for a metal-fueled, pool-type sodium cooled fast reactor (SFR), led by Argonne National Laboratory, a simplified radionuclide transport code (SRT code) was developed, which includes models to estimate the quantity of fission product aerosols scrubbed in the sodium pool during postulated accident scenarios. In a pool-type SFR, when fission products are released into the coolant pool due to failure of fuel pins, most of the radionuclides are scrubbed by the coolant pool, but some have the potential to migrate to the cover gas region through entrainment within gas bubbles. The SRT code contains a model that evaluates this scrubbing behavior and calculates the fraction of fission product aerosols that reach the cover gas. Due to a lack of available validation data for sodium pool scrubbing, the U.S. Department of Energy funded an experiment at the University of Wisconsin-Madison to measure aerosol scrubbing by injecting air bubbles containing aerosol into a coolant pool. Prior to performing an experiment with liquid sodium, a water loop experiment was performed. Their experiment evaluated the effect of changing the aerosol size, aerosol density, aerosol concentration, bubble size, and pool depth on the aerosol scrubbing efficiency of the pool. In this benchmark experiment, the base tests were conducted by repeated tests of isolated bubbles. Afterwards, more prototypic tests with bubble swarms were performed to evaluate the interactions between the bubbles. The bubble swarm test was able to confirm that a larger amount of aerosol scrubbing occurred than the single bubble test. It was also confirmed that as the bubble size, aerosol density, and pool height increase, the extent of pool scrubbing also increases and does not change with the aerosol concentration. In addition, since the degree of scrubbing is the lowest at aerosol sizes between 0.01 and 1 μm, that is, the largest amount of aerosol is emitted, it was confirmed that the analysis of this size in MST is the most important. This benchmark experiment informs the direction of future sodium experiments.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗
DOE OSTI2025

Artificial Intelligence Benchmarking

AI benchmarking is a method for evaluating the effectiveness of an AI model using a set of standardized metrics, for example, high school-level math exams. These benchmarks and their results will enable ranking various AI models based on their effectiveness in performing a specific task.

Krishnan, Anjay [Fermilab]↗
DOE OSTI2026

Final Design for Thermal/Epithermal eXperiments (TEX) with Lithium Absorbers to Provide Validation Benchmarks for Y-12 Electrorefining Facility (IER 575, CED-2 Report)

One of the main goals of the Thermal/Epithermal eXperiments (TEX) project is to use existing Nuclear Criticality Safety Program (NCSP) assets to create critical experiment plutonium and uranium test beds for materials important to criticality safety that have insufficient benchmark evaluations. The plutonium test bed experiments were completed in 2018 and are published in the 2020 edition of the International Criticality Safety Benchmark Evaluation Project (ICSBEP) Handbook. The uranium test bed assemblies were completed in 2023 and accepted in the 2024 edition of the ICSBEP Handbook.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗
DOE OSTI2024

Benchmarking of ENDF/B-VIII.1 Thermal Scattering Library for Hydrogen

In correctly characterizing the energy and momentum transfer at thermal and cold energies between neutrons and its interacting medium, thermal scattering libraries, which details energy states due to the intra- and inter-molecular bond effects for the medium materials, are applied in place of free-gas cross section libraries in a particle transport simulation code. They are essential to the neutron performance of a neutron facility like SNS, where thermalized neutrons from 20 K liquid hydrogen and ambient (~300 K) water are transported to the beamlines for neutron scattering experiments in studying materials. Recently, a new version of thermal scattering library for parahydrogen and orthohydrogen at 14-20 K was developed and to be released in ENDF/B-VIII.1. It is, therefore, important to benchmark its impacts on the prediction of moderator performance due to the updates in the thermal scattering library. In this study, the recent ENDF/B-VIII.1 thermal scattering library was compared to the current ENDF/B-VII.1 one in the neutron performance calculations for the decoupled and coupled hydrogen moderators at SNS under theorized and real working conditions. In addition, the predictions using both thermal scattering libraries were benchmarked to the measurements of moderator performance. The consistency between the libraries was observed mostly for parahydrogen and the difference in orthohydrogen at cold neutron energies was noted.

Lu, Wei↗
DOE OSTI2023

Unifying and benchmarking state-of-the-art quantum error mitigation techniques

Error mitigation is an essential component of achieving a practical quantum advantage in the near term, and a number of different approaches have been proposed. In this work, we recognize that many state-of-the-art error mitigation methods share a common feature: they are data-driven, employing classical data obtained from runs of different quantum circuits. For example, Zero-noise extrapolation (ZNE) uses variable noise data and Clifford-data regression (CDR) uses data from near-Clifford circuits. We show that Virtual Distillation (VD) can be viewed in a similar manner by considering classical data produced from different numbers of state preparations. Observing this fact allows us to unify these three methods under a general data-driven error mitigation framework that we call UNIfied Technique for Error mitigation with Data (UNITED). In certain situations, we find that our UNITED method can outperform the individual methods (i.e., the whole is better than the individual parts). Specifically, we employ a realistic noise model obtained from a trapped ion quantum computer to benchmark UNITED, as well as other state-of-the-art methods, in mitigating observables produced from random quantum circuits and the Quantum Alternating Operator Ansatz (QAOA) applied to Max-Cut problems with various numbers of qubits, circuit depths and total numbers of shots. We find that the performance of different techniques depends strongly on shot budgets, with more powerful methods requiring more shots for optimal performance. For our largest considered shot budget (10 10 ), we find that UNITED gives the most accurate mitigation. Hence, our work represents a benchmarking of current error mitigation methods and provides a guide for the regimes when certain methods are most useful.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
DOE OSTI2024

HamLib: A library of Hamiltonians for benchmarking quantum algorithms and hardware

In order to characterize and benchmark computational hardware, software, and algorithms, it is essential to have many problem instances on-hand. This is no less true for quantum computation, where a large collection of real-world problem instances would allow for benchmarking studies that in turn help to improve both algorithms and hardware designs. To this end, here we present a large dataset of qubit-based quantum Hamiltonians. The dataset, called HamLib (for Hamiltonian Library), is freely available online and contains problem sizes ranging from 2 to 1000 qubits. HamLib includes problem instances of the Heisenberg model, Fermi-Hubbard model, Bose-Hubbard model, molecular electronic structure, molecular vibrational structure, MaxCut, Max- k -SAT, Max- k -Cut, QMaxCut, and the traveling salesperson problem. The goals of this effort are (a) to save researchers time by eliminating the need to prepare problem instances and map them to qubit representations, (b) to allow for more thorough tests of new algorithms and hardware, and (c) to allow for reproducibility and standardization across research studies.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2023

Posiform planting: generating QUBO instances for benchmarking

We are interested in benchmarking both quantum annealing and classical algorithms for minimizing quadratic unconstrained binary optimization (QUBO) problems. Such problems are NP-hard in general, implying that the exact minima of randomly generated instances are hard to find and thus typically unknown. While brute forcing smaller instances is possible, such instances are typically not interesting due to being too easy for both quantum and classical algorithms. In this contribution, we propose a novel method, called posiform planting , for generating random QUBO instances of arbitrary size with known optimal solutions, and use those instances to benchmark the sampling quality of four D-Wave quantum annealers utilizing different interconnection structures (Chimera, Pegasus, and Zephyr hardware graphs) and the simulated annealing algorithm. Posiform planting differs from many existing methods in two key ways. It ensures the uniqueness of the planted optimal solution, thus avoiding groundstate degeneracy, and it enables the generation of QUBOs that are tailored to a given hardware connectivity structure, provided that the connectivity is not too sparse. Posiform planted QUBOs are a type of 2-SAT boolean satisfiability combinatorial optimization problems. Our experiments demonstrate the capability of the D-Wave quantum annealers to sample the optimal planted solution of combinatorial optimization problems with up to 5, 627 qubits.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2023

Validation of time-dependent shift using the pulsed sphere benchmarks

The detailed behavior of neutrons in a rapidly changing time-dependent physical system is a challenging computational physics problem, particularly when using Monte Carlo methods on heterogeneous high-performance computing architectures. A small number of algorithms and code implementations have been shown to be performant for time-independent (fixed source and k-eigenvalue) Monte Carlo, and there are existing simulation tools that successfully solve the time-dependent Monte Carlo problem on smaller computing platforms. To bridge this gap, a time-dependent version of ORNL’s Shift code has been recently developed. Shift’s history-based algorithm on CPUs, and its event-based algorithm on GPUs, have both been observed to scale well to very large numbers of processors, which motivated the extension of this code to solve time-dependent problems. The validation of this new capability requires a comparison with time-dependent neutron experiments. Lawrence Livermore National Laboratory’s (LLNL) pulsed sphere benchmark experiments were simulated in Shift to validate both the time-independent as well as new time-dependent features recently incorporated into Shift. A suite of pulsed-sphere models was simulated using Shift and compared to the available experimental data and simulations with MCNP. Overall results indicate that Shift accurately simulates the pulsed sphere benchmarks, and that the new time-dependent modifications of Shift are working as intended. Validated exascale neutron transport codes are essential for a wide variety of future multiphysics applications.

Palmer, Camille J.↗
DOE OSTI2025

Benchmarking a Tunable Quantum Neural Network on Trapped-Ion and Superconducting Hardware

We implement a quantum generalization of a neural network on trapped-ion and IBM superconducting quantum computers to classify MNIST images, a common benchmark in computer vision. The network feedforward involves qubit rotations whose angles depend on the results of measurements in the previous layer. The network is trained via simulation, but inference is performed experimentally on quantum hardware. The classical-to-quantum correspondence is controlled by an interpolation parameter, $a$, which is zero in the classical limit. Increasing $a$ introduces quantum uncertainty into the measurements, which is shown to improve network performance at moderate values of the interpolation parameter. We then focus on particular images that fail to be classified by a classical neural network but are detected correctly in the quantum network. For such borderline cases, we observe strong deviations from the simulated behavior. We attribute this to physical noise, which causes the output to fluctuate between nearby minima of the classification energy landscape. Such strong sensitivity to physical noise is absent for clear images. We further benchmark physical noise by inserting additional single-qubit and two-qubit gate pairs into the neural network circuits. Our work provides a springboard toward more complex quantum neural networks on current devices: while the approach is rooted in standard classical machine learning, scaling up such networks may prove classically non-simulable and could offer a route to near-term quantum advantage.

FOS: Physical sciences↗
DOE OSTI2023

Panta Rhei benchmark dataset: socio-hydrological data of paired events of floods and droughts

As the adverse impacts of hydrological extremes increase in many regions of the world, a better understanding of the drivers of changes in risk and impacts is essential for effective flood and drought risk management and climate adaptation. However, there is currently a lack of comprehensive, empirical data about the processes, interactions, and feedbacks in complex human–water systems leading to flood and drought impacts. Here we present a benchmark dataset containing socio-hydrological data of paired events, i.e. two floods or two droughts that occurred in the same area. The 45 paired events occurred in 42 different study areas and cover a wide range of socio-economic and hydro-climatic conditions. The dataset is unique in covering both floods and droughts, in the number of cases assessed and in the quantity of socio-hydrological data. The benchmark dataset comprises (1) detailed review-style reports about the events and key processes between the two events of a pair; (2) the key data table containing variables that assess the indicators which characterize management shortcomings, hazard, exposure, vulnerability, and impacts of all events; and (3) a table of the indicators of change that indicate the differences between the first and second event of a pair. The advantages of the dataset are that it enables comparative analyses across all the paired events based on the indicators of change and allows for detailed context- and location-specific assessments based on the extensive data and reports of the individual study areas. The dataset can be used by the scientific community for exploratory data analyses, e.g. focused on causal links between risk management; changes in hazard, exposure and vulnerability; and flood or drought impacts. The data can also be used for the development, calibration, and validation of socio-hydrological models. The dataset is available to the public through the GFZ Data Services (Kreibich et al., 2023, https://doi.org/10.5880/GFZ.4.4.2023.001).

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2026

The AWAKEN wind farm benchmark, Part 2: Modeling results

Accurately modeling wind farm performance in complex atmospheric flows remains a challenge. This paper presents the modeling results of the American WAKE experimeNt (AWAKEN) wind farm benchmark, a collaborative effort involving 16 research groups from academia and industry within the International Energy Agency Wind Technology Collaboration Programme Task 57. The study evaluates a diverse suite of simulation tools, ranging from fast-running engineering wake models to high-fidelity large-eddy simulations, against a diurnal case study observed during the AWAKEN campaign. The benchmark utilized a three-phase structure to progressively assess model performance as observational data availability increased. Initial blind predictions showed that higher-fidelity models did not uniformly outperform simpler simulation tools. A distinct spatial bias was observed where models struggled to resolve the interplay between a low-level jet, wakes, and terrain-induced flow acceleration. In subsequent phases, leveraging additional measurements for model improvement led to a reduction in mean absolute error across the model ensemble; however, this effect was most pronounced in engineering wake models, where targeted calibration reduced error by up to 40~\%. Overall, the study demonstrates that inflow characterization remains a primary prerequisite for accuracy, particularly for models relying on coarse forcing datasets. While the limited ability to resolve local terrain-flow interactions under single-day conditions represent a recognized constraint, the overall findings on wake modeling and real-world validation still provide valuable guidance for model application and for mitigating this limitation.

Bodini, Nicola↗
DOE OSTI2025

Q1-2024 Solar Cost Benchmarks

Each year, the U.S. Department of Energy’s (DOE) Solar Energy Technologies Office (SETO) and its national laboratory partners develop cost benchmarks for U.S. solar photovoltaic (PV) systems. These benchmarks track progress toward reducing solar costs and guide R&D priorities. Unlike typical studies that report only $/W, SETO uses intrinsic units (e.g., $/m² for mounting structures) to better capture how technology improvements such as module efficiency would impact system costs. This allows flexible modeling where inputs can vary significantly to assess cost sensitivity. Costs are reported in two ways: Minimum Sustainable Price (MSP): Long term, financially viable price under stable market conditions. Modeled Market Price (MMP): Actual market price, influenced by short term distortions such as tariffs or subsidies. Three national labs collect cost data from industry stakeholders, ensuring no duplication in outreach to stakeholders. Data reflects real transactions (primarily from Q1) and is weighted based on the number of sources per cost element. The PV System Cost Model (PVSCM) divides total installed system cost into eight categories: 1. Module (PV) 2. Inverter 3. Energy Storage System (ESS) 4. Structural BOS (SBOS) 5. Electrical BOS (EBOS) 6. Fieldwork 7. Office work 8. Other (developer/EPC costs) The first five are hardware costs, while the last three are soft costs. Each category includes fixed and variable cost components, where “size” depends on context (e.g., manufacturing capacity for modules vs. system capacity for installation costs). Variable costs are expressed using appropriate intrinsic units. The model reflects the owner’s upfront overnight capital cost, excluding tax credits. Tariffs and subsidies are treated as temporary market distortions affecting MMP but not MSP. PVSCM is implemented in Excel, where cost elements are aggregated into total system cost. Additional sheets handle unit conversions and operation & maintenance (O&M), with O&M costs levelized over the system’s lifetime.

14 SOLAR ENERGY↗
NASA NTRS1985

The NAS kernel benchmark program

A collection of benchmark test kernels that measure supercomputer performance has been developed for the use of the NAS (Numerical Aerodynamic Simulation) program at the NASA Ames Research Center. This benchmark program is described in detail and the specific ground rules are given for running the program as a performance test.

Bailey, D. H.↗
NASA NTRS1986

Benchmarks of programming languages for special purposes in the space station

Although Ada is likely to be chosen as the principal programming language for the Space Station, certain needs, such as expert systems and robotics, may be better developed in special languages. The languages, LISP and Prolog, are studied and some benchmarks derived. The mathematical foundations for these languages are reviewed. Likely areas of the space station are sought out where automation and robotics might be applicable. Benchmarks are designed which are functional, mathematical, relational, and expert in nature. The coding will depend on the particular versions of the languages which become available for testing.

Knoebel, Arthur↗
NASA NTRS1986

Benchmarking hypercube hardware and software

It was long a truism in computer systems design that balanced systems achieve the best performance. Message passing parallel processors are no different. To quantify the balance of a hypercube design, an experimental methodology was developed and the associated suite of benchmarks was applied to several existing hypercubes. The benchmark suite includes tests of both processor speed in the absence of internode communication and message transmission speed as a function of communication patterns.

Grunwald, Dirk C.↗
NASA NTRS1986

Toward real-time performance benchmarks for Ada

Benchmarks are developed to measure the Ada notion of time, the Ada features believed important to real-time performance, and other time-related features that are not part of the language, but are part of the run-time system; these benchmarks are then applied to the language and run-time system, and the results evaluated.

Clapp, Russell M.↗