Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “BENCHMARKS”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 325 records · Page 18
All sourcesNASA NTRSDOE OSTI
NASA NTRSDate not supplied

Investigations of an Aeroelastic Optimization Benchmark Problem with MPhys and FUN3D

High-fidelity aeroelastic optimization capabilities are becoming more common in recent years. In this field, researchers typically present results on the design problems of greatest interest to them. The lack of consistency with respect to baseline configurations, design variables, objectives and constraints, etc., makes it difficult to compare approaches and creates an obstacle for the community to work together to advance the state of the art. The High-Fidelity Aeroelastic Optimization Benchmark Working Group has been established to create and study a series of benchmark aeroelastic optimization problems that the community can utilize to compare methods on the same optimization problem. This paper presents optimization results from NASA Langley Research Center for the first benchmark case of this series with analysis based on MPhys, a multiphysics library for OpenMDAO, and FUN3D, an unstructured computational fluid dynamics suite of tools. For the optimization case presented, it is demonstrated that a progression of increasing aerodynamic model fidelity reduces the wall time required to complete the optimization.

CFD↗
NASA NTRS2026

Benchmarking Bayesian Optimization Frameworks and Acquisition Strategies for Materials Discovery and Autonomous Laboratories

Bayesian optimization (BO) can accelerate materials discovery by guiding expensive experiments toward the most promising processing conditions. We systematically compare five BO surrogate and framework combinations (Gaussian processes in Ax, Gaussian processes and Monte-Carlo neural networks in BayBE, random forests in Lolopy, and tree-structured Parzen (TPE) estimators in Hyperopt) on three benchmarks that mimic common materials design tasks (a discrete solid-electrolyte composition space, a hybrid discrete/continuous laminate-composite design problem solved with micromechanics modeling, and the continuous Ishigami analytic function which is a standard optimization benchmark). Each BO surrogate is paired with posterior mean, probability of improvement, and expected improvement acquisition functions and run for 100 trials from randomized initial samples with uniform random search providing a control. Across five random seeds per setting, BayBE’s Gaussian-process surrogate with expected improvement consistently reached ≥95 % of the known optimum in the fewest evaluations, while Lolopy’s random forest matched or exceeded GP performance on purely categorical or mixed spaces at a higher computational cost. Posterior mean alone often stagnated at local optima, underscoring the need for exploration, whereas probability and expected improvement balanced exploration and exploitation leading to better optimization in fewer trials. Execution times ranged from milliseconds for TPE to minutes for neural-network and random-forest surrogates. These results establish baseline expectations for BO in automated materials laboratories and highlight expected improvement with Gaussian processes as a reliable first choice, with random forests offering a strong alternative when categorical variables dominate. The benchmark suite and code are released to facilitate future surrogate, acquisition, and constraint-handling research in data-driven materials optimization.

Bayesian optimization↗
DOE OSTI2021

MULTIPHYSICS-MODELING OF FIRE-INDUCED URANIUM AEROSOL FORMATION – A-POSTERIORI BENCHMARKING OF EXPERIMENTS

The formulation and solution of a model that properly described the temperature profiles of uranium in fire conditions was prepared, verified and validated, to support a-posteriori benchmarking of historical experiments. Data from the multi-physics model combined with data regression from the experiments provides a useful tool for exploring sources of potential bias in historical experiments. The approaches used are augmented by visual observations and photographic evidence allowing for correlations between various phenomena, independent of whether or not that specific phenomena are part of the validated model. In this manner, the validated model is a tool for a-posteriori benchmarking, and not a model designed to replicate the experimental output exactly for any specific experiment. For experiments like the Clark (2015) experiments, this integrated model (including chemistry) demonstrated that the largest bias was associated in the metallurgy of the coupons. The model developed does serve as a basis for defending which of the historical alloys tested are the most appropriate for any other alloy being considered (e.g., one that has not yet been tested). The model identifies key reactions that impact uranium chemistry and can significantly bias experimental results. Through use of the model and energy balances, data from historical experiments by Elder and Tinkle (1980) were regressed and benchmarked. This study identifies that all uranium alloys are not equal and that there are distinct differences between the dominant alloys representing distinct metallurgical phases.

11 NUCLEAR FUEL CYCLE AND FUEL MATERIALS↗
DOE OSTI2021

Evaluation of Oak Ridge National Laboratory Health Physics Research Reactor Operation Data for Criticality Accident Alarm System Benchmark Creation

Over the last few decades, the International Criticality Safety Benchmark Evaluation Project (ICSBEP) Working Group has focused on gathering critical and subcritical benchmark experiment data from different facilities around the world for inclusion in the International Handbook of Evaluated Criticality Safety Benchmark Experiments (ICSBEP Handbook). The compiled data can be used by criticality safety analysts to help validate their calculation tools and cross-section libraries for a variety of applications.

21 SPECIFIC NUCLEAR REACTORS AND ASSOCIATED PLANTS↗
DOE OSTI2022

Open-source FPGA-ML codesign for the MLPerf Tiny Benchmark

We present our development experience and recent results for the MLPerf Tiny Inference Benchmark on field-programmable gate array (FPGA) platforms. We use the open-source hls4ml and FINN workflows, which aim to democratize AI-hardware codesign of optimized neural networks on FPGAs. We present the design and implementation process for the keyword spotting, anomaly detection, and image classification benchmark tasks. The resulting hardware implementations are quantized, configurable, spatial dataflow architectures tailored for speed and efficiency and introduce new generic optimizations and common workflows developed as a part of this work. The full workflow is presented from quantization-aware training to FPGA implementation. The solutions are deployed on system-on-chip (Pynq-Z2) and pure FPGA (Arty A7-100T) platforms. The resulting submissions achieve latencies as low as 20 $\mu$s and energy consumption as low as 30 $\mu$J per inference. We demonstrate how emerging ML benchmarks on heterogeneous hardware platforms can catalyze collaboration and the development of new techniques and more accessible tools.

Borras, Hendrik↗
DOE OSTI2022

FastML Science Benchmarks: Accelerating Real-Time Scientific Edge Machine Learning

Applications of machine learning (ML) are growing by the day for many unique and challenging scientific applications. However, a crucial challenge facing these applications is their need for ultra low-latency and on-detector ML capabilities. Given the slowdown in Moore's law and Dennard scaling, coupled with the rapid advances in scientific instrumentation that is resulting in growing data rates, there is a need for ultra-fast ML at the extreme edge. Fast ML at the edge is essential for reducing and filtering scientific data in real-time to accelerate science experimentation and enable more profound insights. To accelerate real-time scientific edge ML hardware and software solutions, we need well-constrained benchmark tasks with enough specifications to be generically applicable and accessible. These benchmarks can guide the design of future edge ML hardware for scientific applications capable of meeting the nanosecond and microsecond level latency requirements. To this end, we present an initial set of scientific ML benchmarks, covering a variety of ML and embedded system techniques.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗
DOE OSTI2023

MUSiC: Critical Experiment with Bare Highly Enriched Uranium Shells Benchmark

The measurements of the Uranium Subcritical and Critical (MUSiC) experiment were performed from December 2020 through April 2021. The purpose of the experiment was to test a variety of multiplicity detection systems for a large range of multiplication values and to see how well they perform as the k eff of the MUSiC series of experiments approached the delayed critical window. Out of the ten configurations that were studied, two of them were super critical and were recently approved as benchmarks and they will be included in the International Criticality Safety Benchmark Evaluation Project (ICBESP) handbook. For each of the two experiments, uncertainties were evaluated six categories: (1) critical measurement, (2) mass, (3) composition, (4) positioning, (5) dimensions, and (6) temperature. The largest contribution to the overall uncertainty in each of the cases was due to the uncertainties in the dimensions of the HEU Rocky Flats shells. Simplified detailed models were created with an average bias of -130.5 pcm due to the removal various components surrounding the experiment. For MCNP6.2 models using ENDF-VIII.0 neutron cross section data, the average difference between the calculated and benchmark k effs was -54 pcm for the detailed model and -102 pcm for the simplified model.

11 NUCLEAR FUEL CYCLE AND FUEL MATERIALS↗
DOE OSTI2023

DeepSurveySim: Simulation Software and Benchmark Challenges for Astronomical Observation Scheduling

Modern astronomical surveys have multiple competing scientific goals. Optimizing the observation schedule for these goals presents significant computational and theoretical challenges, and state-of-the-art methods rely on expensive human inspection of simulated telescope schedules. Automated methods, such as reinforcement learning, have recently been explored to accelerate scheduling. However, there do not yet exist benchmark data sets or user-friendly software frameworks for testing and comparing these methods. We present DeepSurveySim -- a high-fidelity and flexible simulation tool for use in telescope scheduling. DeepSurveySim provides methods for tracking and approximating sky conditions for a set of observations from a user-supplied telescope configuration. We envision this tool being used to produce benchmark data sets and for evaluating the efficacy of ground-based telescope scheduling algorithms, particularly for machine learning algorithms that would suffer in efficacy if limited to real data for training.We introduce three example survey configurations and related code implementations as benchmark problems that can be simulated with DeepSurveySim.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗
DOE OSTI2023

RELAP5-3D Solutions to Exercise 1 of the OECD-NEA HTTF Benchmark

Discussion of RELAP5-3D solutions to Exercise 1 of the OECD-NEA High Temperature Test Facility (HTTF) Benchmark. Results from ongoing benchmark activities supported by the Advanced Reactor Technologies - Gas-Cooled Reactor Program with use of the High Performance Computing Center at Idaho National Laboratory. Also discussion of Ultra Safe Nuclear Corporation and BWXT plans for the decade, and issues of modern thermal hydraulic tools for prismatic HTGR modeling. Open discussion of verification and validation and ART-GCR campaign to spearhead the development of an HTGR thermal hydraulics benchmark based on the HTTF.

11 NUCLEAR FUEL CYCLE AND FUEL MATERIALS↗
DOE OSTI2023

RELAP5-3D Modeling in the OECD-NEA HTTF Benchmark

Prismatic gas-cooled reactors are a technologically mature reactor concept of interest for near- to mid-term deployment. To accelerate the deployment of these reactors, the DOE's Advanced Reactor Technologies Gas-Cooled Reactor campaign is spearheading a thermal hydraulics code validation benchmark based on the High Temperature Test Facility at Oregon State University. This presentation provides an introduction to that benchmark and an overview of some of the RELAP5-3D validation activities stemming from that benchmark.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗
DOE OSTI2026

A Benchmarking Framework for Evaluating Large Language Model Capabilities in Nuclear Reactor Safety Applications

Large language models (LLMs) are increasingly capable of answering technical questions, synthesizing domain knowledge, and supporting engineering workflows. For nuclear science and engineering, these capabilities require careful, domain-specific evaluation before they can be credibly incorporated into safety-related activities, regulatory review, or technical decision support. This paper presents preliminary results from benchmarking framework for evaluating LLM capabilities in nuclear contexts. The framework is organized into three evaluation categories: nuclear fundamentals, general dual-use knowledge, and plant specific knowledge. These categories are intended to distinguish general nuclear engineering competence from broader technical reasoning and more context-dependent nuclear knowledge. Initial evaluations focus on nuclear fundamentals using questions representative of the knowledge expected of a nuclear professional engineer. Results indicate that contemporary frontier models perform at a high level and substantially exceed the performance of older model generations, with some models approaching saturation of the current benchmark. These findings suggest both the rapid improvement of LLM capabilities in specialized technical domains and the need for more discriminating evaluation methods. The paper presents the benchmark structure, preliminary model-comparison results, and ongoing work. This work supports development of verifiable, responsible, and safety-conscious methods for assessing AI systems in nuclear engineering applications.

21 - SPECIFIC NUCLEAR REACTORS AND ASSOCIATED PLAN↗
DOE OSTI2024

IER 480: TEX-Pu Benchmark (PU-MET-THERM-004) to Test Polyethylene and Lucite Thermal Scattering Laws [Slides]

This presentation finds that PMT-004 has four new benchmark cases highly sensitive to PE TSL (2 cases) and PMMA TSL (2 Cases). PE cases were well predicted using MCNP6.2 and ENDF/B-VIII.0. PMMA cases overpredicted by approximately 0.6-0.7% in k eff at 20°C. Accepted into 2023 version of the ICSBEP Handbook. Temperature had a large impact on reactivity of the critical configuration. Implications for validation work for thermal cases- need to adjust TSL data to correct temperature as it can have hundreds of pcm effects for a few °C. Future thermal experiments should try and measure reactivity at multiple temperatures to aid in data testing and benchmark adjustment.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗
DOE OSTI2023

Reproducible benchmark for the SNAP 8 experimental reactor at dry conditions

Here, this work provides fully reproducible benchmark models of the Systems for Nuclear Auxiliary Power (SNAP) 8 Experimental Reactor. Validation benchmarks of the criticality configuration experiments under dry conditions with no coolant are presented. In addition, relevant reactivity worth experiments, and measurement of power distributions are validated and presented here. Discrepancies between modeled and experimental results are at most 110 pcm for fuel and poison worths, and critical configurations that do not manipulate control elements are within 50 pcm. Larger discrepancies found in reflector element worth experiments are due to experimental methodology which was noted as being simplified due to limited calculational capabilities at the time, while discrepancy in the control element manipulated critical configuration is due the absence of structural components for modeling simplification. Power distributions closely follow what was observed in experiment with expected peaking in reflecting elements. All models are well documented with cited references describing and justifying assumptions, simplifications, and adjustments to reproduce the models with any nuclear safety codes; model inputs and outputs are stored in the snapReactors GitHub repository.

21 SPECIFIC NUCLEAR REACTORS AND ASSOCIATED PLANTS↗
DOE OSTI2026

Reproducible benchmark for the SNAP 8 experimental reactor at operating conditions

This work presents fully reproducible multiphysics benchmark models of the Systems for Nuclear Auxiliary Power (SNAP) 8 Experimental Reactor at operating conditions with coolant flow. Wet experiment (with coolant, at power) validation benchmarks are presented using both deterministic (Serpent-Griffin) and Monte-Carlo (OpenMC-Cardinal) multiphysics frameworks coupled with thermal-hydraulic solvers in MOOSE. Reactivity coefficient measurements including fuel temperature, isothermal temperature, and power coefficients show good agreement with experiments, with discrepancies within experimental uncertainty. Reactivity worth experiments for coolant, samarium, and xenon poisoning are reproduced with differences under 200 pcm. Comparison between Serpent-Griffin and OpenMC-Cardinal frameworks reveal multiphysics coupling introduces positive reactivity effects (100-200 pcm) compared to uniform temperature and density fields at nominal operating conditions. Comparison between Serpent-Griffin and reference Serpent solution shows that power distributions maintain consistent radial and axial peaking behavior. All models, assumptions, thermophysical and thermomechanical properties, and material definitions are thoroughly documented with cited references; model inputs and model generating scripts are stored in the snapReactors GitHub repository.

SNAP↗
DOE OSTI2021

Benchmarking the multiconfigurational Hartree method by the exact wavefunction of two harmonically trapped bosons with contact interaction

Highlights: • Comparison of exact and numerical single-particle density matrices. • Benchmark for multiconfigurational Hartree. • Convergence for large number of orbitals. We consider two bosons in a one-dimensional harmonic trap, interacting by a contact potential, and compare the exact solution of this problem to a self-consistent numerical solution by using the multiconfigurational time-dependent Hartree (MCTDH) method. We thereby benchmark the predictions of the MCTDH method with a few-body problem that has an analytical solution for the most commonly experimentally realized interaction potential in ultracold quantum gases. It is found that exact ground state energy and first order correlations are accurately reproduced by MCTDH up to the intermediate dimensionless coupling strengths corresponding to typical background scattering lengths of magnetically trapped ultracold dilute Bose gases. For larger couplings, established for example by (a combination of) Feshbach resonances and optical trapping, the MCTDH approach overestimates the depth of the trap-induced correlation dip of first order correlations in position space, as well as overestimates the fragmentation, defined as the average relative occupation of orbitals other than the energetically lowest one. We anticipate that qualitatively similar features in the correlation function may arise for larger particle numbers, paving the way for a quantitative assessment of the accuracy of MCTDH by experiments with ultracold atoms.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
DOE OSTI2025

Electrocatalytic benchmarking of ruthenium-based bimetallic anodes for the electrocatalytic oxidation of biomass-derived wastewater

In this paper, we report on the synthesis, characterization, and use of ruthenium oxide (RuO 2 ) doped with a secondary metal (M2) to enhance electrochemical activity and stability for the electrocatalytic oxidation (ECO) of biomass-derived wastewaters. We used different electrochemical methods such as cyclic voltammetry (CV), electrochemical surface area (ECSA), and Tafel analysis as well as physical characterization such as grazing incidence X-ray diffraction, X-ray photoelectron spectroscopy, and scanning electron microscopy to understand how the introduction of M2 affects electrochemical performance. Our results show that including an M2 improves the ECO performance regardless of the composition of the electrolyte. Specifically, we saw increase in ECSA, which could be due to enhanced charge transfer for the pH ranges evaluated. Furthermore, the introduction of organic compounds in wastewater generated during the hydrothermal liquefaction of food waste affected the ECO performance differently, depending on M2, the electrolyte composition, and anodic half-cell potential, highlighting the need to properly control the reaction conditions when testing and characterizing the electrocatalysts under different reaction regimes. We developed an in situ electrocatalytic benchmarking protocol, Boruah – Lopez-Ruiz – Strange (BLoRS), to quickly assess if the presence of M2 improves the ECO performance; thus, saving time and resources in ex situ characterization, testing, and product analysis. This foundational work provides the basis for characterization and benchmarking of electrodes of the ECO of organic compounds.

Electrocatalysis↗
DOE OSTI2020

Phase field benchmark problems targeting fluid flow and electrochemistry

Here, we continue our development of phase field model benchmark problems with the addition of a third set, complimenting our previously developed problems for diffusion, precipitation, dendritic growth and linear elasticity. These benchmark problems are being jointly developed by the Center for Hierarchical Materials Design (CHiMaD) and the National Institute of Standards and Technology (NIST) along with input from the phase field community. The first problem in this third set targets Stokes flow, with a particular emphasis on flow around an obstruction placed asymmetrically in the domain. While Stokes flow is not traditionally in the canon of phase field problems, it is a class of problems gaining importance in areas such as filtration and water purification. The second problem deals with coupled Cahn-Hilliard diffusion and electrostatic forces, which is an important area in energy storage and battery sciences. We present our own solutions and discuss sources of numerical errors for the Stokes problem as well as simple checks to avoid fundamental issues in the coupled diffusion-electrostatics problem. The latter problem contains some subtleties that we expand on in an Appendix.

36 MATERIALS SCIENCE↗
DOE OSTI2026

Benchmarking of massively parallel phase-field codes for directional solidification

We present a detailed benchmark comparing two state-of-the-art phase-field implementations for simulating alloy solidification under experimentally relevant conditions. The study investigates the directional solidification of Al-3wt%Cu under high-velocity solidification conditions and SCN-0.46wt% camphor under microgravity conditions from National Aeronautics and Space Administration (NASA) DECLIC-DSI-R experiments. Both codes, one employing finite-difference discretization with uniform mesh and GPU-acceleration (GPU-PF) and the other one employing finite-element discretization with adaptive-mesh and CPU-parallelization (PRISMS-PF), solve the same quantitative phase-field formulation that incorporates an anti-trapping current for the solidification of dilute alloys. We evaluate the predictions of each code for dendritic morphology, primary spacing, and tip dynamics in both 2D and 3D, as well as their numerical convergence and computational performance. While existing benchmark problems have primarily focused on simplified or small-scale simulations, they do not reflect the computational and modeling challenges posed by employing experimentally relevant time and length scales. Our results provide a practical framework for assessing phase-field code performance as well as validating and facilitating their application in integrated computational materials engineering (ICME) workflows that require integration with realistic experimental data.

36 MATERIALS SCIENCE↗