SEARCH · Engineering Papers

Results for “BENCHMARKS”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 181 records · Page 10

All sources NASA NTRS DOE OSTI

DOE OSTI2020

Assessment of Critical Experiment Benchmark Applicability to a Large-Capacity HALEU Transportation Package Concept

This work presents an assessment of the applicability of existing benchmark critical experiments to the criticality safety code validation for a large-capacity high-assay low-enriched uranium (HALEU) transportation package concept. Numerous next-generation nuclear reactor designs require HALEU fuel, which is characterized by an enrichment between 5 and 20 wt% 235 U. The U.S. Department of Energy (DOE) has proposed to recover and downblend highly enriched uranium from DOE-owned used nuclear fuel to accelerate the demonstration of commercially viable microreactor technologies. One element of the infrastructure needed to demonstrate HALEU-fueled reactors is the ability to safely transport enriched product to be used for fuel fabrication. There is uncertainty as to whether existing critical benchmark experiment data are sufficient to support criticality safety code validation for HALEU transportation applications. The anticipated chemical form of the HALEU in the proposed transportation concept is UO 2 with 20 wt% 235 U/U. The concept uses a combination of an existing transportation packaging design and a novel basket design, including borated aluminum flux traps. The basket provides space for 18 reusable, stainless steel canisters that contain the HALEU. In 10 CFR 71, normal conditions of transport (NCTs) and hypothetical accident conditions (HACs) are defined for fissile material transportation packages. NCT and HAC KENO-VI models of the transportation package were developed using the Standardized Computer Analyses for Licensing Evaluation (SCALE) 6.2.3 computer code package, and optimum moderation conditions were determined using the SCALE SAMPLER sequence. The SCALE Tools for Sensitivity and Uncertainty Analysis Methodology Implementation (TSUNAMI) sequences were then used to compare the neutronic characteristics of 1584 International Criticality Safety Benchmark Evaluation Project benchmark critical experiments with the NCT and HAC HALEU transportation models. The TSUNAMI integral correlation coefficient c k was the criterion used to rank neutronic similarity. Thirty-four experiments were identified as similar (c k ≥ 0.9) to the NCT model, and 55 experiments were identified as similar to the HAC model. Hundreds of experiments were also identified as at least marginally similar (c k ≥ 0.8) to both models. The results indicate that additional critical experiments are unlikely to be needed to support HALEU transportation criticality safety analyses for package concepts similar to the concept package analyzed.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗

DOE OSTI2023

Code Benchmark of the HTTF Pressurized Conduction Cooldown Test Using SAM

The High Temperature Test Facility (HTTF) at Oregon State University is an integral system test facility to simulate postulated reactor transients of prismatic high-temperature gas-cooled reactors(HTGRs). A series of test campaigns was launched, providing abundant test data that could be used to benchmark reactor system analysis codes like the System Analysis Module (SAM). In this study, a SAM model of the facility is developed based on the two-dimensional (2D) ring model approach. All components including the ceramic matrix, graphite heaters, helium coolant channels, core barrel, upcomer, pressure vessel, and reactor cavity cooling system are modeled as concentric cylindrical rings. The model is used to simulate one of the benchmark problems-Pressurized Conduction Cooldown (PCC)-within the scope of the Organisation for Economic Co-operation and Development Nuclear Energy Agency International HTTF Benchmark. The simulations consist of two parts. In the first part, operating and boundary conditions as well as thermophysical properties of materials are specified for the benchmark problem. In this work, results from the first part will be used in code-to-code comparison. In the second part, the SAM model is used to simulate Test PG-27, which is the first PCC test carried out in the HTTF, with only two of the ten heater banks activated. The results in the second part are used for code-to-data comparison. Because the helium coolant flow rate is not measured in this facility, it is estimated using the input power and inlet/outlet coolant temperatures. Additionally, radial heat flow in the ceramic blocks is complicated by hundreds of cylindrical coolant channels and heater rods embedded in them. As such, it is necessary to deduce an effective thermal conductivity for the ceramic to analyze the core thermal behavior. SAM predictions of the helium coolant and ceramic temperatures are compared with test data measured in three equivalent sectors. Overall, the SAM results agree reasonably well with test data within the variation of data among the three sectors, which demonstrates SAM's capability in capturing transient effects in HTGR using the simplified 2D ring model.

21 SPECIFIC NUCLEAR REACTORS AND ASSOCIATED PLANTS↗

DOE OSTI2024

Multicode benchmark on simulated Ti K-edge x-ray absorption spectra of Ti-O compounds

X-ray absorption spectroscopy (XAS) is an element-specific materials characterization technique that is sensitive to structural and electronic properties. First-principles simulated XAS has been widely used as a powerful tool to interpret experimental spectra and draw physical insights. Recently, there has also been growing interest in building computational XAS databases to enable data analytics and machine learning applications. However, there are non-trivial differences among commonly used XAS simulation codes, both in underlying theoretical formalism and in technical implementation. Reliable and reproducible computational XAS databases require systematic benchmark studies. In this work, we benchmarked Ti K-edge XAS simulations of ten representative Ti-O binary compounds, which we refer to as the Ti-O-10 dataset, using three state-of-the-art codes: XSPECTRA, OCEAN and exciting. We systematically studied the convergence behavior with respect to the input parameters and developed a workflow to automate and standardize the calculations to ensure converged spectra. Our benchmark comparison considers a 35 eV spectral range starting from the K-edge onset, representative of widely used near-edge spectra. Quantitative comparison over this range is based on Spearman’s rank correlation score (r sp ). Our results show: (1) the two Bethe-Salpeter equation (BSE) codes (OCEAN and exciting) have excellent agreement with an average $r$ $sp$ of 0.998; (2) good agreement is obtained between the core-hole potential code (XSPECTRA) and BSE codes (OCEAN and exciting) with an average $r$ $sp$ of 0.990, and this smaller $r$ $sp$ reflects the noticeable differences in the main edge spectral shape that can be primarily attributed to the difference in the strength of the screened core-hole potential; (3) simulations from both methods overall reproduce well the main experimental spectral features of rutile and anatase, and the different treatments of the screened core-hole potential have visible impact on pre-edge intensities and the peak ratio of the main edge; (4) there exist moderate differences in the relative edge alignment of the three codes with a standard deviation of about 0.2 eV, which arise from multiple contributions including the frozen core approximation, final state effects, and different approximations used for the self-energy correction. In conclusion, our benchmark study provides important standards for first-principles XAS simulations with broad impact in data-driven XAS analysis.

36 MATERIALS SCIENCE↗

DOE OSTI2024

SRBench++: Principled Benchmarking of Symbolic Regression With Domain-Expert Interpretation

Symbolic regression searches for analytic expressions that accurately describe studied phenomena. The main promise of this approach is that it may return an interpretable model that can be insightful to users, while maintaining high accuracy. The current standard for benchmarking these algorithms is SRBench, which evaluates methods on hundreds of datasets that are a mix of real-world and simulated processes spanning multiple domains. At present, the ability of SRBench to evaluate interpretability is limited to measuring the size of expressions on real-world data, and the exactness of model forms on synthetic data. In practice, model size is only one of many factors used by subject experts to determine how interpretable a model truly is. Furthermore, SRBench does not characterize algorithm performance on specific, challenging sub-tasks of regression such as feature selection and evasion of local minima. In this work, we propose and evaluate an approach to benchmarking SR algorithms that addresses these limitations of SRBench by 1) incorporating expert evaluations of interpretability on a domain-specific task, and 2) evaluating algorithms over distinct properties of data science tasks. We evaluate 12 modern symbolic regression algorithms on these benchmarks and present an in-depth analysis of the results, discuss current challenges of symbolic regression algorithms and highlight possible improvements for the benchmark itself.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2024

HamPerf: A Hamiltonian-Oriented Approach to Quantum Benchmarking

Quantum computing technologies are undergoing rapid development. The different qubit modalities being considered for quantum computing each have their strengths and weaknesses, making it challenging to compare their performance relative to each other and the state-of-the-art in classical high-performance computing. To better understand the utility of a given quantum processor and to assess when and how it will be able to advance the frontiers of computational science, researchers need a robust approach to quantum benchmarking. A variety of approaches have been proposed, many of which characterize the presence of noise in current quantum devices. These efforts include component-level performance metrics, such as randomized benchmarking and gate set tomography; high-level application-dependent metrics; and devicelevel metrics, such as the Quantum Volume. However, it remains unclear how low-level metrics, such as fidelities and decoherence times, and global device metrics, such as Quantum Volume, relate to the computational utility and practical limitations of quantum processors to solve useful problems. In this paper, we describe our Hamiltonian-oriented approach to quantum benchmarking called HamPerf. Where previous application-dependent approaches specify a suite of benchmarking circuits inspired by applications, we place the problem Hamiltonian at the center. Our strategy allows us to probe the computational performance of a quantum processor on standardized and relevant problem sets, agnostic of the algorithms and hardware used to solve them; it also provides fundamental insights into how device characteristics correlate with computational utility.

Butko, Anastasiia↗

DOE OSTI2023

Update to the Microcontroller Benchmark for Radiation Testing

LANL developed a benchmark of software code for radiation testing of microprocessors several years ago, and it was published under an open-source license on GitHub. Publishing the software is necessary for other researchers to adopt and implement this benchmark for radiation testing of other microprocessors to standardize test practices so that test data can be compared across different microprocessors. The original codes have been used several times by other organizations to test a wide range of microcontrollers and microprocessors. After several years of research, LANL is ready to update the benchmark. Changes include: 1. Addition of new codes that allow common software codes to be tested, 2. Addition of new codes that instrument more microprocessor circuitry, 3. Addition of input patterns that allow for a more compressive understanding of how the memory layout affects the sensitivity to radiation-induced faults and better use of automated test pattern generation standards, and 4. Modification of current codes for faster and more resilient detection, reporting and correction of radiation-induced faults. These codes have been tested by LANL researchers over the last few years, which has been published in the open literature. As the code base for the new benchmarks are stable, it is time to release the update to the GitHub repository, where the original codes were released.

Quinn, Heather↗

DOE OSTI2020

Status of the KRUSTY Benchmark Modeling and Uncertainty Analysis

The U.S. National Aeronautics and Space Administration’s (NASA) desire for a small nuclear reactor for space applications led to a prototype assembly, Kilopower Reactor Using Stirling Technology (KRUSTY). The KRUSTY experiments included four phases: component critical, cold critical, warm critical, and high temperature. The KRUSTY benchmark described in this paper includes five configurations from the component critical experiments that were performed at the National Criticality Experiments Research Center (NCERC) beginning in November 2017. Preliminary results have been reported. The benchmark is now nearing completion (although it is still preliminary until formally accepted by the International Criticality Safety Benchmark Evaluation Project [ICSBEP] working group). In this paper, we present the updated benchmark and its uncertainties. These results were computed using MCNP6.2 ® with ENDF/B-VIII.0 cross sections.

21 SPECIFIC NUCLEAR REACTORS AND ASSOCIATED PLANTS↗

DOE OSTI2020

Metallic Fuel Benchmark Simulations Based on the X430 Experiments

To gauge the accuracy of current BISON metallic fuel performance models and identify models that require further development, Oak Ridge National Laboratory has developed benchmark problems based on two pins of the X430 experiment. These benchmarks recreate the experimental conditions of the X430 series of experiments with as much accuracy as possible given the available data. The benchmarks were implemented in the BISON nuclear fuel performance code and the simulation results were compared with the experimental results to quantify the simulation accuracy. Findings in this work were consistent with previous benchmark problems. The temperature profiles, burnup, and fission gas behavior were all as accurate as can reasonably be expected. The cladding radial growth was accurate enough to not affect the results of other measurements. The fuel axial growth was accurate for U-Zr fuel but not for U-Pu-Zr fuel, which appears to be caused by a lack of U-Pu-Zr data. Swelling correlations have not been developed for U-Pu-Zr fuel because there were not enough data on this fuel type, so BISON used the U-Zr correlations. Finally, recommendations were made for near-term BISON development.

11 NUCLEAR FUEL CYCLE AND FUEL MATERIALS↗

DOE OSTI2021

Benchmark Testing on the IBM-Q Network

The goal of this project is to evaluate a proposed set of candidate benchmarks being developed by the Standards and Performance Metrics Technical Advisory Group (TAG) of the Quantum Economic Development Consortium (QED-C), of which LANL is a member. The QED-C Standards and Performance Metrics TAG has developed implementations of several benchmark codes with the hope and expectation that these codes will be helpful to QED-C members interested in investigating various quantum computer platforms. The benchmark set will be run on LANL’s access to the IBM-Q system with the intent to evaluate the set for scalability, correct execution, and coverage of the application space. The goal is for the QED-C Standards and Performance Metrics TAC to be able to produce a coherent, consistent, scalable set of benchmarks that will run on multiple quantum computing platforms and will be available to members of QED-C, including LANL.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2021

Plutonium Production Reactor Progression Problems: Magnox Neutronics Benchmarks

The goal of this document is to provide a set of comprehensive benchmarking neutronics problems for gas-cooled, graphite-moderated reactors. The problems are designed to be representative problems that increase in complexity to test performance of various codes. For the simulation of the benchmarking problems, the Shift Monte Carlo code and MPACT deterministic code are chosen because of their state-of-the-art capabilities, which are optimized for various reactor designs. Because these codes have been developed primarily for light water reactor applications, their application to graphite-moderated, gas-cooled reactors has not been extensively explored. This report attempts to evaluate benchmarks for a Magnox-style reactor, one that is graphite-moderated and gas-cooled. Parameters that are calculated and presented in this benchmark include the eigenvalue, peaking factors, and isotope concentrations. Results show good agreement between Shift and MPACT, and demonstrate the capability of MPACT to model graphite-moderated systems well.

21 SPECIFIC NUCLEAR REACTORS AND ASSOCIATED PLANTS↗

DOE OSTI2021

Questionable Benchmarks [Slides]

An investigation and initial review of ‘questionable benchmarks’ was performed at LANL. Multiple methods were used, including machine learning techniques, to identify 'questionable benchmarks' and/or benchmarks with low uncertainties. Those benchmarks were then reviewed for obvious errors. This is not a recommendation to ICSBEP, but can be used as a starting point for a more comprehensive review. A journal paper is being written on this work.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗

DOE OSTI2023

International Benchmarking: A Global Perspective of Workplace Drug and Alcohol Testing

Workplace drug and alcohol testing has been widely implemented in diverse industries and is designed to detect the use of and deter the influence of legal and illegal substances that would impair the ability to perform work that has significant security and safety implications. Across the globe, patterns of drug and alcohol use and abuse and the regulations controlling drug and alcohol use vary by location (e.g., regions, countries, etc.). The prevalence of workplace drug and alcohol testing and protection of individuals’ rights in the workplace similarly tend to exhibit contextual differences. With the advancements in drug-testing technologies and the recent issuance of guidance for additional testing matrices (e.g., oral fluid, hair, etc.) by the U.S. Department of Health and Human Services, gaining a global perspective on the current state of drug and alcohol testing practices and regulations is important for leveraging such advancements to improve the effectiveness of drug and alcohol testing. The U.S. Nuclear Regulatory Commission (NRC) tasked the Pacific Northwest National Laboratory (PNNL) fitness for duty (FFD) team to conduct a benchmarking analysis of international drug and alcohol testing regulations and programs to identify similarities and differences in workplace testing programs and the basis for any differences. The benchmarking study focuses on a policy analysis and benchmarking metrics comparison among different countries and regions. The policy analysis includes the following three types of relevant international and national policies relevant to drug and alcohol testing in the nuclear industry: 1. National drug and alcohol policy 2. Workplace drug and alcohol testing 3. FFD regulations for Nuclear Power Plants The international drug and alcohol benchmarking metrics focusses on the use of blood, breath, urine, hair, and oral fluid in testing; drug and drug metabolite cutoff values; point-of-collection testing devices; roadside testing methodologies; innovations in drug testing; and the extent of availability of such data. Gaining an understanding of workplace drug and alcohol testing in other countries as well as information about new testing methods, matrices, tools, and technologies that other countries have adopted provides NRC with international comparative information regarding the technical basis for such programs and methods as it relates to the protection of worker and public health and safety.

98 NUCLEAR DISARMAMENT, SAFEGUARDS, AND PHYSICAL P↗

DOE OSTI2025

The Risk Assessment Information System Compendium of Ecological Screening Benchmarks for Chemicals and Radionuclides (2025) (Volume I – Text, Radionuclides for all Media, Chemicals for Air and Biota)

The Risk Assessment Information System Ecological Benchmark Tool compiles screening level benchmark values for assessing potential ecological risks posed by chemical and radionuclide contaminants. Drawing from a wide array of federal, state, and international sources, it consolidates benchmarks across environmental media, including soil, water, sediment, air, and biota, and presents values for a broad range of receptors and exposure pathways. The tool includes both effect-based and no-effect thresholds, with media- and organism-specific guidance derived from peer-reviewed literature and regulatory frameworks. By integrating diverse benchmark types into a unified reference, the tool supports consistent, scientifically grounded evaluations in ecological risk assessments and environmental management practices.

54 ENVIRONMENTAL SCIENCES↗

DOE OSTI2025

Demonstration of Optimal Benchmark Selection Website and Validation of the q c Coverage Metric Using HEU-SOL-THERM-013-003 Experiment

In the work documented in this interim report, the experiment selection toolkit web site was demonstrated and q C coverage metric methodology was validated for IEU-MET-FAST-002-001, MIX-COMP-THERM 004-004, and HEU-SOL-THERM-013-003 experiments. 𝑞 𝐶 is an information-theoretic measure based on mutual information that quantifies the ability of candidate benchmark experiments to reduce the bias and uncertainty of a target criticality safety application. The metric and an accompanying open-source Python toolkit with a web-based interface were tested against a benchmark set of 425 experiments drawn from the International Criticality Safety Benchmark Evaluation Project Handbook. The interface is hosted at https://edim.covdef.com. It accepts sensitivity data files produced by the TSUNAMI-IP module of the SCALE code system and supports both (i) deterministic analysis using the ENDF/B-VII.0 covariance library and (ii) stochastic analysis based on user-supplied keff samples. Demonstrations on representative applications across a range of material composition, spectrum, and form show that q C -guided benchmark selection achieves greater uncertainty reduction with fewer experiments and yields more stable posterior bias and uncertainty estimates than traditional similarity coefficient ( c k )–based selection, while also capturing valuable low-ck experiments that one-to-one metrics overlook.

Abdel-khalik, Hany S. [Indiana Univ.-Purdue Univ. ↗

DOE OSTI2021

Nuclear Data Sensitivity Study for the EBR-II Fast Reactor Benchmark Using SCALE with ENDF/B-VII.1 and ENDF/B-VIII.0

The EBR-II benchmark, which was recently included in the International Handbook of Evaluated Reactor Physics Benchmark Experiments, served as a basis for assessing the performance of the SCALE code system for fast reactor analyses. A reference SCALE model was developed based on the benchmark specifications. Great agreement was observed between the eigenvalue calculated with this SCALE model and the benchmark eigenvalue. To identify potential gaps and uncertainties of nuclear data for the simulation of various quantities of interest in fast spectrum systems, sensitivity and uncertainty analyses were performed for the eigenvalue, reactivity effects, and the radial power profile of EBR-II using the two most recent ENDF/B nuclear data library releases. While the nominal results are consistent between the calculations with the different libraries, the uncertainties due to nuclear data vary significantly. The major driver of observed uncertainties is the uncertainty of the 235 U ( n,γ ) reaction. Since the uncertainty of this reaction is significantly reduced in the ENDF/B-VIII.0 library compared to ENDF/B-VII.1, the obtained output uncertainties tend to be smaller in ENDF/B-VIII.0 calculations, although the decrease is partially compensated by increased uncertainties in 235 U fission and ν ¯ .

42 ENGINEERING↗

DOE OSTI2026

Benchmarking soil moisture and its relationship to ecohydrologic variables in Earth System Models

Soil moisture (SM) is a key regulator of ecosystem biogeophysics, influencing plant water relations and land-atmosphere energy exchanges. We evaluate the representation of SM in 16 Earth System Models from the Coupled Model Intercomparison Project Phase 6 (CMIP6) using the International Land Model Benchmarking (ILAMB) framework, focusing on surface (0–5, 0–10 cm) and rootzone (0–100 cm) depths, as well as key ecohydrological variables like gross primary productivity (GPP), leaf area index (LAI), and evapotranspiration (ET), and their coupling. Models are benchmarked against multiple observational and assimilated datasets to assess both state variables and cross-variable relationships. Surface SM is generally well represented (r > 0.87), while rootzone SM variability is systematically overestimated (normalized standard deviation > 1). ET shows strong agreement with observations (r > 0.9), whereas GPP and LAI exhibit larger inter-model spread. Skill in individual variables does not guarantee realistic SM–ecohydrology coupling, which varies strongly across models and depends on the reference dataset. Köppen-based regional analyses reveal strong regime dependence, with several models performing well in Tropical and Temperate regions but degrading in Continental (high-latitude) zones. Across both global and regional benchmarks, models cluster by land surface framework, indicating that structural choices in soil hydrology and soil–plant coupling exert a first-order control on performance. These results provide process-relevant benchmarks and suggest that improving the representation of vertical soil structure, rooting depth distributions, and soil–plant hydraulic coupling will be central to advancing soil moisture realism in next-generation Earth system models.

CMIP6↗

NASA NTRS1993

Developing scheduling benchmark tests for the Space Network

A set of benchmark tests were developed to analyze and measure Space Network scheduling characteristics and to assess the potential benefits of a proposed flexible scheduling concept. This paper discusses the role of the benchmark tests in evaluating alternative flexible scheduling approaches and defines a set of performance measurements. The paper describes the rationale for the benchmark tests as well as the benchmark components, which include models of the Tracking and Data Relay Satellite (TDRS), mission spacecraft, their orbital data, and flexible requests for communication services. Parameters which vary in the tests address the degree of request flexibility, the request resource load, and the number of events to schedule. Test results are evaluated based on time to process and schedule quality. Preliminary results and lessons learned are addressed.

Moe, Karen L.↗

NASA NTRS1994

Concept of using a benchmark part to evaluate rapid prototype processes

A conceptual benchmark part for guiding manufacturers and users of rapid prototyping technologies is proposed. This is based on a need to have some tool to evaluate the development of this technology and to assist the user in judiciously selecting a process. The benchmark part is designed to have unique product details and features. The extent to which a rapid prototyping process can reproduce these features becomes a measure of the capability of the process. Since rapid prototyping is a dynamic technology, this benchmark part should be used to continuously monitor process capability of existing and developing technologies. Development of this benchmark part is, therefore, based on an understanding of the properties required from prototypes and characteristics of various rapid prototyping processes and measuring equipment that is used for evaluation.

Cariapa, Vikram↗