Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Benchmarks”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 343 records · Page 19

OpenSHMEM Implementation of IOR Benchmark

The benchmarking effort within the Computational Research & Development Programs at the Oak Ridge National Laboratory (ORNL) seeks to design and enable High Performance Computing (HPC) benchmarks and test suites. The work described in this report is a part of the effort focusing on the comparison and analysis of OpenSHMEM implementations using the Interleave Or Random (IOR) software for benchmarking parallel file system using POSIX, MPIIO, or HDF5 interfaces. We describe the effort to emulate the MPIIO parallel collective capabilities in the IOR benchmark using OpenSHMEM communication. One development effort was in emulating the MPI derived datatype used in the read/write operations and in setting the file view. Another effort was in implementing an internal cache in OpenSHMEM distributed shared memory to facilitate global collective I/O operations. Experiments comparing collective I/O in MPIIO implementations with the OpenSHMEM implementations were performed on the SGI Turing Cluster and the Cray XK7 Titan supercomputer at the Oak Ridge Leadership Computing Facility (OLCF). The preliminary results suggest that on the Cray XK7 Titan, the MPIIO implementations obtained higher write performance and the OpenSHMEM version obtained slightly higher read performance. On the SGI Turing Cluster, the MPIIO implementations obtained slightly higher performance over the OpenSHMEM implementations on large files.

97 MATHEMATICS AND COMPUTING↗

IAEA Coordinated Research Project on HTGR Physics, Thermal-Hydraulics, and Depletion Uncertainty Analysis Prismatic HTGR Benchmark Specification: Phase II

In February 2009, the Technical Working Group on Gas Cooled Reactors of the International Atomic Energy Agency (IAEA) recommended that the proposed Coordinated Research Project (CRP) on the High-Temperature Gas Cooled Reactor (HTGR) Uncertainty Analysis in Modeling (UAM) be implemented. This CRP is a continuation of the previous IAEA and Organisation for Economic Cooperation and Development (OECD)/Nuclear Energy Agency (NEA) international activities on verification and validation of available analytical capabilities for HTGR simulation for design and safety evaluations. Within the framework of these activities different numerical and experimental benchmark problems were performed, and insight was gained about specific physics phenomena and the adequacy of analysis methods. Because the prismatic design specification included in this report is based directly on the OECD/NEA Modular HTGR (MHTGR) 350 MW benchmark, participants in both activities can leverage their core models developed for the OECD/NEA benchmark for this CRP benchmark with only minor changes. The CRP on the HTGR UAM Phase II specifications is also directly linked to the Phase I specification document, since the core models required for Phase II exercises are constructed using the lattice models and cross-section libraries developed for Phase 1. It was decided at the Third Research Coordination Meeting (RCM) in May 2017 that the initial scope of CRP activities will be reduced to match the formally approved remaining contract period (June 2019), as well as limited funding and resource availability. A summary of the revised Phases I–IV exercise definitions is provided here.

42 ENGINEERING↗

Metrics and Benchmarks for Quantum Processors: State of Play

A compelling narrative has taken hold as quantum computing explodes into the commercial sector: Quantum computing in 2018 is like classical computing in 1965. In 1965 Gordon Moore wrote his famous paper about integrated circuits, saying: "At present, [minimum cost] is reached when 50 components are used per circuit. But... the complexity for minimum component costs has increased at a rate of roughly a factor of two per year... by 1975, the number of components per integrated circuit for minimum cost will be 65,000." This narrative is both appealing (we want to believe that quantum computing will follow the incredibly successful path of classical computing!) and plausible (2018 saw IBM, Intel, and Google announce 50-qubit integrated chips). But it is also deeply misleading. Here is an alternative: Quantum computing in 2018 is like classical computing in 1938. In 1938, John Atanasoff and Clifford Berry built the very first electronic digital computer. It had no program, and was not Turing-complete. Vacuum tubes — the standard "bit" for 20 years — were still 5 years in the future. ENIAC and the achievement of "computational supremacy" (over hand calculation) wouldn't arrive for 8 years, despite the accelerative effect of WWII. Integrated circuits and the information age were more than 20 years away. Neither of these analogies is perfect. Quantum computing technology is more like 1938, while the level of funding and excitement suggest 1965 (or later!). But the point of the cautionary analogy to 1938 is simple: Quantum computing in 2018 is a research field. It is far too early to establish metrics or benchmarks for performance. The best role for neutral organizations like IEEE is to encourage and shape research into metrics and benchmarks, so as to be ready when they become necessary. This white paper presents the evidence and reasoning for this claim. We explain what it means to say that quantum computing is a "research field", and why metrics and benchmarks for quantum processors also constitute a research field. We discuss the potential for harmful consequences of prematurely establishing standards or frameworks. We conclude by suggesting specific actions that IEEE or similar organizations can take to accelerate the development of good metrics and benchmarks for quantum computing.

97 MATHEMATICS AND COMPUTING↗

Benchmark Evaluation of TREAT Reactor M2 and M3 Transient Experiments

The objective of this project is to evaluate and produce TREAT reactor benchmarks under steady state and transient conditions. The benchmarks support the TREAT restart effort and are to be contributed to the International Reactor Physics Experiment Evaluation Project (IRPhEP) for inclusion in the International Handbook of Evaluated Reactor Physics Benchmark Experiments. The benchmark evaluation is performed using data generated during the “Power Calibration Experiment for test M2” and “IFR Safety Tests M2 and M3”. The evaluations are conducted using the high fidelity Serpent-OpenFOAM code system. Both the steady state and transient conditions of the M2 and M3 tests were investigated. Furthermore, TREAT has a graphitic core. Historical work has shown that the predictions of the outcomes of TREAT tests are highly sensitive to the graphite thermal neutron scattering cross sections. Therefore, in this project, thermal neutron scattering (i.e., S(α,β)) libraries of graphite, tailored to the specifics of the TREAT core, were generated and utilized in the evaluation

21 SPECIFIC NUCLEAR REACTORS AND ASSOCIATED PLANTS↗

Three Analytic Benchmarks in COG

Analytic benchmarks are valuable tools to test the algorithms in Monte Carlo codes not related to geometry or interaction data. COG was used to calculate three analytic benchmarks: Kobayashi’s benchmarks for simple geometries and void regions; Shmakov’s benchmarks using the back and-forth approximation; and Thomas et al’s critical problem for an infinite cylinder.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗

Critical and Subcritical Experiment NEA Benchmark Possibilities for Measurements at ORCEF and Other USDOE Facilities

This report documents a wide variety of critical and subcritical measurements that could be Nuclear Energy Agency (NEA) or International Criticality Safety Benchmark Program (ICSBEP) benchmarks. The list has omitted some few measurements. The experiments are divided into four classes: critical experiments only, critical and subsequent subcritical experiments, subcritical experiments (all performed in critical experiment facilities, and fourth subcritical measurements performed in Material Balance Areas at the Y-12 National Security Complex (NSC) for Y-12 nuclear criticality safety. This report documents to number of critical facility days utilized used for the measurements. Measurement at critical and subsequently subcritical in critical facilities at Oak Ridge National Laboratory (ORCEF), Los Alamos National Laboratory, Pacific Northwest National Laboratory, and Babcock and Wilcox at Lynchburg, Va, utilized ~350 operational days of critical facility time, Measurement only at subcritical at ORCEF utilized ~200 operational days in this critical facility, and subcritical measurements in Material Balance Areas of the Y-12 National Security Complex used 350 operational days, many were unattended measurements. In view of the high cost of critical facility time and the limitation of only one facility available for measurements in the United States, it is very cost-effective to mine the existing data to produce benchmarks such as those already in NEA and ICSBEP. For measurements at ORCEF the material descriptions from Y-12 NSC and the accuracy of the configurations should (bases on experience with existing NEA or ICSBEP of ORCEF measurement benchmarks) result in uncertainties in measured k eff values lower than ±0.001 and may be as low as ±0.0002; thus, allowing more accurate comparisons between measurements and calculations and thus more accurate verification of calculations.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗

Metallic Fuel Benchmark Simulations Based on the X430 Experiments

To gauge the accuracy of current BISON metallic fuel performance models and identify models that require further development, Oak Ridge National Laboratory has developed benchmark problems based on two pins of the X430 experiment. These benchmarks recreate the experimental conditions of the X430 series of experiments with as much accuracy as possible given the available data. The benchmarks were implemented in the BISON nuclear fuel performance code and the simulation results were compared with the experimental results to quantify the simulation accuracy. Findings in this work were consistent with previous benchmark problems. The temperature profiles, burnup, and fission gas behavior were all as accurate as can reasonably be expected. The cladding radial growth was accurate enough to not affect the results of other measurements. The fuel axial growth was accurate for U-Zr fuel but not for U-Pu-Zr fuel, which appears to be caused by a lack of U-Pu-Zr data. Swelling correlations have not been developed for U-Pu-Zr fuel because there were not enough data on this fuel type, so BISON used the U-Zr correlations. Finally, recommendations were made for near-term BISON development.

11 NUCLEAR FUEL CYCLE AND FUEL MATERIALS↗

Benchmark Testing on the IBM-Q Network

The goal of this project is to evaluate a proposed set of candidate benchmarks being developed by the Standards and Performance Metrics Technical Advisory Group (TAG) of the Quantum Economic Development Consortium (QED-C), of which LANL is a member. The QED-C Standards and Performance Metrics TAG has developed implementations of several benchmark codes with the hope and expectation that these codes will be helpful to QED-C members interested in investigating various quantum computer platforms. The benchmark set will be run on LANL’s access to the IBM-Q system with the intent to evaluate the set for scalability, correct execution, and coverage of the application space. The goal is for the QED-C Standards and Performance Metrics TAC to be able to produce a coherent, consistent, scalable set of benchmarks that will run on multiple quantum computing platforms and will be available to members of QED-C, including LANL.

97 MATHEMATICS AND COMPUTING↗

Plutonium Production Reactor Progression Problems: Magnox Neutronics Benchmarks

The goal of this document is to provide a set of comprehensive benchmarking neutronics problems for gas-cooled, graphite-moderated reactors. The problems are designed to be representative problems that increase in complexity to test performance of various codes. For the simulation of the benchmarking problems, the Shift Monte Carlo code and MPACT deterministic code are chosen because of their state-of-the-art capabilities, which are optimized for various reactor designs. Because these codes have been developed primarily for light water reactor applications, their application to graphite-moderated, gas-cooled reactors has not been extensively explored. This report attempts to evaluate benchmarks for a Magnox-style reactor, one that is graphite-moderated and gas-cooled. Parameters that are calculated and presented in this benchmark include the eigenvalue, peaking factors, and isotope concentrations. Results show good agreement between Shift and MPACT, and demonstrate the capability of MPACT to model graphite-moderated systems well.

21 SPECIFIC NUCLEAR REACTORS AND ASSOCIATED PLANTS↗

Questionable Benchmarks [Slides]

An investigation and initial review of ‘questionable benchmarks’ was performed at LANL. Multiple methods were used, including machine learning techniques, to identify 'questionable benchmarks' and/or benchmarks with low uncertainties. Those benchmarks were then reviewed for obvious errors. This is not a recommendation to ICSBEP, but can be used as a starting point for a more comprehensive review. A journal paper is being written on this work.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗

International Benchmarking: A Global Perspective of Workplace Drug and Alcohol Testing

Workplace drug and alcohol testing has been widely implemented in diverse industries and is designed to detect the use of and deter the influence of legal and illegal substances that would impair the ability to perform work that has significant security and safety implications. Across the globe, patterns of drug and alcohol use and abuse and the regulations controlling drug and alcohol use vary by location (e.g., regions, countries, etc.). The prevalence of workplace drug and alcohol testing and protection of individuals’ rights in the workplace similarly tend to exhibit contextual differences. With the advancements in drug-testing technologies and the recent issuance of guidance for additional testing matrices (e.g., oral fluid, hair, etc.) by the U.S. Department of Health and Human Services, gaining a global perspective on the current state of drug and alcohol testing practices and regulations is important for leveraging such advancements to improve the effectiveness of drug and alcohol testing. The U.S. Nuclear Regulatory Commission (NRC) tasked the Pacific Northwest National Laboratory (PNNL) fitness for duty (FFD) team to conduct a benchmarking analysis of international drug and alcohol testing regulations and programs to identify similarities and differences in workplace testing programs and the basis for any differences. The benchmarking study focuses on a policy analysis and benchmarking metrics comparison among different countries and regions. The policy analysis includes the following three types of relevant international and national policies relevant to drug and alcohol testing in the nuclear industry: 1. National drug and alcohol policy 2. Workplace drug and alcohol testing 3. FFD regulations for Nuclear Power Plants The international drug and alcohol benchmarking metrics focusses on the use of blood, breath, urine, hair, and oral fluid in testing; drug and drug metabolite cutoff values; point-of-collection testing devices; roadside testing methodologies; innovations in drug testing; and the extent of availability of such data. Gaining an understanding of workplace drug and alcohol testing in other countries as well as information about new testing methods, matrices, tools, and technologies that other countries have adopted provides NRC with international comparative information regarding the technical basis for such programs and methods as it relates to the protection of worker and public health and safety.

98 NUCLEAR DISARMAMENT, SAFEGUARDS, AND PHYSICAL P↗

The Risk Assessment Information System Compendium of Ecological Screening Benchmarks for Chemicals and Radionuclides (2025) (Volume I – Text, Radionuclides for all Media, Chemicals for Air and Biota)

The Risk Assessment Information System Ecological Benchmark Tool compiles screening level benchmark values for assessing potential ecological risks posed by chemical and radionuclide contaminants. Drawing from a wide array of federal, state, and international sources, it consolidates benchmarks across environmental media, including soil, water, sediment, air, and biota, and presents values for a broad range of receptors and exposure pathways. The tool includes both effect-based and no-effect thresholds, with media- and organism-specific guidance derived from peer-reviewed literature and regulatory frameworks. By integrating diverse benchmark types into a unified reference, the tool supports consistent, scientifically grounded evaluations in ecological risk assessments and environmental management practices.

54 ENVIRONMENTAL SCIENCES↗

Demonstration of Optimal Benchmark Selection Website and Validation of the q c Coverage Metric Using HEU-SOL-THERM-013-003 Experiment

In the work documented in this interim report, the experiment selection toolkit web site was demonstrated and q C coverage metric methodology was validated for IEU-MET-FAST-002-001, MIX-COMP-THERM 004-004, and HEU-SOL-THERM-013-003 experiments. 𝑞 𝐶 is an information-theoretic measure based on mutual information that quantifies the ability of candidate benchmark experiments to reduce the bias and uncertainty of a target criticality safety application. The metric and an accompanying open-source Python toolkit with a web-based interface were tested against a benchmark set of 425 experiments drawn from the International Criticality Safety Benchmark Evaluation Project Handbook. The interface is hosted at https://edim.covdef.com. It accepts sensitivity data files produced by the TSUNAMI-IP module of the SCALE code system and supports both (i) deterministic analysis using the ENDF/B-VII.0 covariance library and (ii) stochastic analysis based on user-supplied keff samples. Demonstrations on representative applications across a range of material composition, spectrum, and form show that q C -guided benchmark selection achieves greater uncertainty reduction with fewer experiments and yields more stable posterior bias and uncertainty estimates than traditional similarity coefficient ( c k )–based selection, while also capturing valuable low-ck experiments that one-to-one metrics overlook.

Abdel-khalik, Hany S. [Indiana Univ.-Purdue Univ. ↗

Nuclear Data Sensitivity Study for the EBR-II Fast Reactor Benchmark Using SCALE with ENDF/B-VII.1 and ENDF/B-VIII.0

The EBR-II benchmark, which was recently included in the International Handbook of Evaluated Reactor Physics Benchmark Experiments, served as a basis for assessing the performance of the SCALE code system for fast reactor analyses. A reference SCALE model was developed based on the benchmark specifications. Great agreement was observed between the eigenvalue calculated with this SCALE model and the benchmark eigenvalue. To identify potential gaps and uncertainties of nuclear data for the simulation of various quantities of interest in fast spectrum systems, sensitivity and uncertainty analyses were performed for the eigenvalue, reactivity effects, and the radial power profile of EBR-II using the two most recent ENDF/B nuclear data library releases. While the nominal results are consistent between the calculations with the different libraries, the uncertainties due to nuclear data vary significantly. The major driver of observed uncertainties is the uncertainty of the 235 U ( n,γ ) reaction. Since the uncertainty of this reaction is significantly reduced in the ENDF/B-VIII.0 library compared to ENDF/B-VII.1, the obtained output uncertainties tend to be smaller in ENDF/B-VIII.0 calculations, although the decrease is partially compensated by increased uncertainties in 235 U fission and ν ¯ .

42 ENGINEERING↗

Benchmarking soil moisture and its relationship to ecohydrologic variables in Earth System Models

Soil moisture (SM) is a key regulator of ecosystem biogeophysics, influencing plant water relations and land-atmosphere energy exchanges. We evaluate the representation of SM in 16 Earth System Models from the Coupled Model Intercomparison Project Phase 6 (CMIP6) using the International Land Model Benchmarking (ILAMB) framework, focusing on surface (0–5, 0–10 cm) and rootzone (0–100 cm) depths, as well as key ecohydrological variables like gross primary productivity (GPP), leaf area index (LAI), and evapotranspiration (ET), and their coupling. Models are benchmarked against multiple observational and assimilated datasets to assess both state variables and cross-variable relationships. Surface SM is generally well represented (r > 0.87), while rootzone SM variability is systematically overestimated (normalized standard deviation > 1). ET shows strong agreement with observations (r > 0.9), whereas GPP and LAI exhibit larger inter-model spread. Skill in individual variables does not guarantee realistic SM–ecohydrology coupling, which varies strongly across models and depends on the reference dataset. Köppen-based regional analyses reveal strong regime dependence, with several models performing well in Tropical and Temperate regions but degrading in Continental (high-latitude) zones. Across both global and regional benchmarks, models cluster by land surface framework, indicating that structural choices in soil hydrology and soil–plant coupling exert a first-order control on performance. These results provide process-relevant benchmarks and suggest that improving the representation of vertical soil structure, rooting depth distributions, and soil–plant hydraulic coupling will be central to advancing soil moisture realism in next-generation Earth system models.

CMIP6↗

Tools for Simulation and Benchmark Generation at Exascale

The path to exascale high-performance computing (HPC) poses several challenges related to power, performance, resilience, productivity, programmability, data movement, and data management. Investigating the performance of parallel applications at scale on future architectures and the performance impact of different architecture choices is an important component of HPC hardware/software co-design. Simulations using models of future HPC systems and communication traces from applications running on existing HPC systems can offer an insight into the performance of future architectures. This work targets technology developed for scalable application tracing of communication events and memory profiles, but can be extended to other areas, such as I/O, control flow, and data flow. It further focuses on extreme-scale simulation of millions of Message Passing Interface (MPI) ranks using a lightweight parallel discrete event simulation (PDES) toolkit for performance evaluation. Instead of simply replaying a trace within a simulation, the approach is to generate a benchmark from it and to run this benchmark within a simulation using models to reflect the performance characteristics of future-generation HPC systems. This provides a number of benefits, such as eliminating the data intensive trace replay and enabling simulations at different scales. The presented work utilizes the ScalaTrace tool to generate scalable trace files, the ScalaBenchGen tool to generate the benchmark, and the xSim tool to run the benchmark within a simulation.

Lagadapati, Mahesh↗

PHISICS/RELAP5-3D RESULTS FOR EXERCISES II-1 AND II-2 OF THE OECD/NEA MHTGR-350 BENCHMARK

The Idaho National Laboratory (INL) Advanced Reactor Technologies (ART) High-Temperature Gas-Cooled Reactor (HTGR) Methods group currently leads the Modular High-Temperature Gas-Cooled Reactor (MHTGR) 350 benchmark. The benchmark consists of a set of lattice-depletion, steady-state, and transient problems that can be used by HTGR simulation groups to assess the performance of their code suites. The paper summarizes the results obtained for the first two transient exercises defined for Phase II of the benchmark. The Parallel and Highly Innovative Simulation for INL Code System (PHISICS), coupled with the INL system code RELAP5-3D, was used to generate the results for the Depressurized Conduction Cooldown (DCC) (exercise II-1a) and Pressurized Conduction Cooldown (PCC) (exercise II-2) transients. These exercises require the time-dependent simulation of coupled neutronics and thermal-hydraulics phenomena, and utilize the steady-state solution previously obtained for exercise I-3 of Phase I. This paper also includes a comparison of the benchmark results obtained with a traditional system code “ring” model against a more detailed “block” model that include kinetics feedback on an individual block level and thermal feedbacks on a triangular sub-mesh. The higher spatial fidelity that can be obtained by the block model is illustrated with comparisons of the maximum fuel temperatures, especially in the case of natural convection conditions that dominate the DCC and PCC events. Differences up to 125 K (or 10%) were observed between the ring and block model predictions of the DCC transient, mostly due to the block model’s capability of tracking individual block decay powers and more detailed helium flow distributions. In general, the block model only required DCC and PCC calculation times twice as long as the ring models, and it therefore seems that the additional development and calculation time required for the block model could be worth the gain that can be obtained in the spatial resolution

Advanced Reactor Technologies↗

High-Fidelity Measurements for Flattop-HEU Benchmark Reevaluation

Flattop was first built in the 1950’s at Los Alamos National Laboratory. Flattop-HEU is composed of a sphere of highly enriched uranium (HEU) surrounded by a thick spherical natural uranium (NU) reflector. The reflector is composed of three parts: a stationary hemisphere and two movable quarter spheres. For fine control of the reactivity of the system, there are three control rods of natural uranium located in voids in the stationary hemisphere. The final components that make Flattop a useful critical assembly are the glory hole and mass adjustment pieces. These pieces can be loaded in various configurations into the glory hole and the core pedestal to control the known worth of the system. The glory hole and mass adjustment pieces are mostly small pieces of HEU with some mass adjustment pieces fabricated from NU. This allows for the irradiation of samples to a specified level. To better document the system, Flattop was evaluated and included in the International Criticality Safety Benchmark Evaluation Project (ICSBEP) handbook. The original benchmark evaluation of Flattop-HEU was written in 1999 based on an experiment completed in the 1960’s. This original evaluation was written to provide a single diameter that defined critical mass; however, as computational capabilities have increased, the focus for benchmark evaluations has shifted to include detailed modelswith all physical dimensions. Thus, as Flattop is a lynchpin in critical experiment work, the benchmark is being reevaluated at current standards. This summary discusses some of the largest known uncertainties from the evaluation and the high-fidelity measurements taken to reduce these uncertainties.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗