Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Benchmark”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,603 records · Page 89
All sourcesNASA NTRSDOE OSTI
DOE OSTI2023

Verification and validation of the open-source plasma fluid code: Zapdos

Zapdos is an open-source finite element plasma fluid solver based on the MOOSE multiphysics framework. This paper outlines Zapdos verification, benchmarking, and validation efforts for 1D and 2D RF capacitively coupled plasma discharge models for mid-range pressures (0.1 - 1 Torr). The verification process involved using the method of manufactured solutions to assess Zapdos spatial and temporal error convergence. L2 errors ranged from 10 -2 to 10 -4 , while the convergence’s slope were in agreement with the predicted slopes for the tested variable and time integration orders. The benchmarking process involved comparisons to previously results from the validated finite element code, LSODI. These works included 1D and 2D simulations for a range of plasma parameters (densities, temperatures, voltage, etc.). For the 1D cases, Zapdos and LSODI results were in very good agreement. In the 2D cases, variable behaviors matched, with slight discrepancies in peak values. The validation process involved comparisons to experimental works including electron density measurements by microwave interferometry and metastable density measurements by planar laser-induced fluorescence imaging. Results shown reasonable agreement at higher pressure, with results starting to diverge at low pressures. Furthermore, probable causes for this diverges are the limitation of the fluid assumption for plasmas at low pressure, or the need for more robust boundary conditions. Overall, Zapdos shown reasonable results for the verification, benchmarking, and validation efforts, and Zapdos can be downloaded at https://github.com/shannon-lab/zapdos.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗
DOE OSTI2023

Comparative verification of hydro-mechanical fracture behavior: Task G of international research project DECOVALEX–2023

Numerical simulations become a necessity when experimental approaches cannot cover the required physical and time scale of interest. One of such area is a simulation of long-term host rock behaviors for nuclear waste disposal and simulation tools involved in the assessment must go through rigorous validation tests. The DECOVALEX project (Development of COupled models and their VAlidation against EXperiments) is dedicated to this purpose by international participants. This work is part of the ongoing phase DECOVALEX–2023 (D–2023, Task G) particularly aiming to simulate fracture behaviors under various conditions. Here, in this study, we cross-verified a variety of numerical methods including continuous and discontinuous approaches against four benchmark exercises with emphasis on numerical accuracy and parameterization of the various numerical approaches. The systematic inter-comparisons of test cases highlight advantages and disadvantages of the different numerical models. Numerical details on discretization effects (e.g. mesh density and orientation) and domain size were investigated in detail for practical applications. It became evident that meticulous attention to mesh resolution and domain size is imperative for achieving accurate numerical simulations, even for static cracks. Moreover, when comparing numerical methods to closed-form solutions for static cracks, all models successfully reproduced the maximum crack opening but encountered challenges near the crack tips. Finally, the paper discusses how to convert between and therefore compare parameters of various numerical approaches. Our benchmark studies reveal that each model necessitates a distinct number of parameters, even in simple scenarios like static crack aperture benchmarks. It is generally more practical to employ fewer parameters to mitigate model over-parameterization and enhance experimental feasibility.

58 GEOSCIENCES↗
DOE OSTI2025

Static Subspace Approximation for Random Phase Approximation Correlation Energies: Applications to Materials for Catalysis and Electrochemistry

Modeling complex materials using high-fidelity, ab initio methods at low cost is a fundamental goal for quantum chemical software packages. The GW approximation and random phase approximation (RPA) provide a unified description of both electronic structure and total energies using the same physics in a many-body perturbative approach that can be more accurate than generalized-gradient density functional theory (DFT) methods. However, GW/RPA implementations have historically been limited to either specific materials classes or application toward small chemical systems. Here, the static subspace approximation allows for reduced cost full-frequency GW/RPA calculations and has previously been benchmarked thoroughly for GW calculations. Here, we describe our approach to including partial occupations of electronic orbitals in full-frequency GW and RPA calculations for the study of electrocatalysts. We benchmarked RPA total energy calculations using the subspace approximation across a diverse test suite of materials for a variety of computational parameters. The benchmarking quantifies the impact of different extrapolation procedures for representing the static polarizability at infinite screened cutoff, and shows that using screened cutoffs above 20-25 Ryd result in diminishing accuracy returns for predicting RPA total energies. Additionally, for moderately sized electrocatalytic models, 2-3 times fewer computational resources are used to compute RPA total energies by representing the static polarizability with 20-30% of the static subspace basis, with an error of approximately 0.01 eV or better in RPA adsorption energy calculations. Finally, we show that for these electrochemical models RPA can shift DFT adsorption energy shifts by up to 0.5 eV and that GW can frequently shift DFT eigenvalues of surface and adsorbate states by approximately 0.5-1 eV.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2021

Measurements of Neutron Scattering from a Copper Sample Using a Quasi-Differential Method in the Region from 2 keV to 20 MeV

Neutron scattering from a copper sample was measured at Rensselaer Polytechnic Institute utilizing the quasi-differential method. The measurement spanned the energy range from 0.5 to 20 MeV using the high-energy scattering system and from 2 keV to 0.5 MeV using the new mid-energy scattering system. Copper was selected as a material of interest to measure due to large discrepancies between experiments and simulations of the Zeus benchmark. The Zeus benchmark consists of a copper reflected highly enriched uranium system, and the angular distribution of copper scattering was thought to potentially be the cause of the discrepancy. The copper measurements found differences in the scattering response particularly in the incident energy region from 1 to 2 MeV for the high-energy measurement and from 2 to 4 keV in the mid-energy system. These differences are particularly noticeable at angles near 90 deg in the high-energy system and back angles in the mid-energy system. Additionally, for ENDF/B-VIII.0 there is a large discrepancy at the forward angle in the energy range around 0.5 MeV. For these reasons, a new evaluation of copper scattering utilizing these results is recommended and perhaps could help to improve the agreement with the Zeus benchmarks.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗
DOE OSTI2022

Quantum advantages for Pauli channel estimation

We show that entangled measurements provide an exponential advantage in sample complexity for Pauli channel estimation, which is both a fundamental problem and a practically important subroutine for benchmarking near-term quantum devices. The specific task we consider is to simultaneously learn all the eigenvalues of an n-qubit Pauli channel to ±ε precision. We give an estimation protocol with an n-qubit ancilla that succeeds with high probability using only O(n/ε 2 ) copies of the Pauli channel, while proving that any ancilla-free protocol (possibly with adaptive control and channel concatenation) would need at least Ω(2 n/3 ) rounds of measurement. We further study the advantages provided by a small number of ancillas. For the case that a k-qubit ancilla (k≤n) is available, we obtain a sample complexity lower bound of Ω(2 (n-k)/3 ) for any nonconcatenating protocol, and a stronger lower bound of Ω(n 2n-k ) for any nonadaptive, nonconcatenating protocol, which is shown to be tight. We also show how to apply the ancilla-assisted estimation protocol to a practical quantum benchmarking task in a noise-resilient and sample-efficient manner, given reasonable noise assumptions. Our results provide a practically interesting example for quantum advantages in learning and also bring insights for quantum benchmarking.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
DOE OSTI2024

Characterizing non-Markovian and coherent errors in quantum simulation

Quantum simulation of many-body systems, particularly using ultracold atoms and trapped ions, presents a unique form of quantum control—it is a direct implementation of a multi-qubit gate generated by the Hamiltonian. As a consequence, it also faces a unique challenge in terms of benchmarking, because the well-established gate benchmarking techniques are unsuitable for this form of quantum control. Here we show that the symmetries of the target many-body Hamiltonian can be used not only to benchmark but to characterize experimental errors in the quantum simulation. We use our results to develop protocols to characterize these errors, which can be implemented using state-of-the-art technology. We consider two forms of errors: (i) unitary errors arising out of systematic errors in the applied Hamiltonian and (ii) canonical non-Markovian errors arising out of random shot-to-shot fluctuations in the applied Hamiltonian. We show that the dynamics of the expectation value of the target Hamiltonian itself, which is ideally constant in time, can be used to characterize these errors. In the presence of errors, the expectation value of the target Hamiltonian shows a characteristic thermalization dynamics, when it satisfies the operator thermalization hypothesis (OTH). That is, an oscillation in the short time followed by relaxation to a steady-state value in the long time limit. We show that while the steady-state value can be used to characterize the coherent errors, the amplitude of the oscillations can be used to estimate the non-Markovian errors. We prove a sandwich theorem to establish a linear relation between the amplitude of the oscillations and the magnitude of the non-Markovian errors. Moreover, by varying the initial state, we show that the steady state values can be used to completely construct the generator of the coherent errors. Using these results, we develop two experimental protocols to characterize the unitary errors based on these results, one of which requires single-qubit addressing and the other one doesn't. We also develop a protocol to characterize non-Markovian errors. Published by the American Physical Society 2024

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
DOE OSTI2025

Demography, dynamics and data: building confidence for simulating changes in the world's forests

Vegetation demographic models (VDMs) are advanced tools for simulating forest responses to climate and land-use changes, and are essential for projecting carbon cycling and large-scale forest management strategies. Despite their increasing incorporation into Earth System Models, VDMs differ in their demographic assumptions, with no prior quantitative comparison of their performance. We benchmarked nine VDMs against observational data from boreal, temperate and tropical sites, assessing their accuracy in predicting tree growth, carbon turnover, biomass stocks and size distributions. Models were simulated under consistent climate conditions with postdisturbance recovery monitored for at least 420 yr. Postdisturbance carbon recovery trajectories showed significant variability while remaining within observational ranges. Initial regrowth rates varied substantially (0.03-0.60, 0.18-0.70 and 0.35-1.10 kgCm-2 yr-1 for boreal, temperate and tropical sites, respectively), influenced by each model's initial forest state. Models captured mature forest carbon content but showed compensating effects between overestimated growth and underestimated mortality rates. This first multi-model benchmarking identifies growth and mortality rates as critical calibration targets and highlights the need to refine postdisturbance establishment conditions for model development. We outline specific benchmarking variables needed to improve predictions of forest responses to environmental change.

demographic vegetation model benchmarking↗
DOE OSTI2022

Neutronics and thermal-hydraulics simulation of generic pebble-bed fluoride-salt-cooled high-temperature reactor (gFHR)

The fluoride-salt-cooled high-temperature reactor (FHR) is one type of the advanced reactors and has been attracting great interest from the research institutes and commercial companies in the recent years. However, currently the technology is relatively immature. To facilitate the design and safety analysis of FHRs, Kairos Power published the generic FHR (gFHR) benchmark. In this paper, a hybrid method combining the stochastic code and deterministic code is developed to simulate the gFHR benchmark. Serpent 2 is employed as a few-group cross section generator and the cross sections are applied to the finite difference neutron diffusion code AGREE. A good consistency with the gFHR benchmark is achieved. The agreement between the Serpent 2 results and the AGREE results shows that the hybrid method is applicable to FHRs. The thermal-hydraulics are coupled to neutronics for the steady-state calculations and good agreement between AGREE and SAM is achieved. (authors)

21 SPECIFIC NUCLEAR REACTORS AND ASSOCIATED PLANTS↗
DOE OSTI2023

PMT-004 Validation Testing for Thermal Scattering Laws and Some Interesting Temperature-Dependent Implications [Slides]

This presentation finds that PMT-004 has four new benchmark cases highly sensitive to PE TSL (2 cases) and PMMA TSL (2 Cases). Additionally discovered was that the PE cases were well predicted using MCNP6.2 and ENDF/B-VIII.0. PMMA cases overpredicted by approximately 0.6-0.7% in k eff likely due to the lack of integral benchmarks sensitive to PMMA. The NCSU ENDF/B-VIII.1 PMMA TSL is an improvement over VIII.0. This was Accepted at last ICSBEP meeting pending comment resolution, should be published in 2024 version of the Handbook. It was found that temperature had a large impact on reactivity of the critical configuration. This has Implications for validation work for thermal cases- need to adjust TSL data to correct temperature as it can have hundreds of pcm effects for a few °C. Future thermal experiments should try and measure reactivity at multiple temperatures to add in data testing and benchmark adjustment.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗
DOE OSTI2024

Evaluating Operators in Deep Neural Networks for Improving Performance Portability of SYCL

SYCL is a portable programming model for heterogeneous computing, so it is important to obtain reasonable performance portability of SYCL. Towards the goal of better understanding and improving performance portability of SYCL for machine learning workloads, we have been developing benchmarks for basic operators in deep neural networks (DNNs). These operators could be offloaded to heterogeneous computing devices such as graphics processing units (GPUs) to speed up computation. In this work, we introduce the benchmarks, evaluate the performance of the operators on GPU-based systems, and describe the causes of the performance gap between the SYCL and Compute Unified Device Architecture (CUDA) kernels. We find that the causes are related to the utilization of the texture cache for read-only data, optimization of the memory accesses with strength reduction, shared local memory accesses, and register usage per thread. We hope that the efforts of developing benchmarks for studying performance portability will stimulate discussion and interactions within the community.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2024

Perform Design Support with MCNP for New Measurements

This report incorporates our work carried out during our 5-month internship at LANL under an internship agreement with EAMEA (École des Applications Militaires de l’Énergie Atomique). After outlining the context in which we worked, we present our work as aid to modeling and predicting the neutronic behavior of nuclear systems, with a view to carrying out criticality experiments qualifying the MCNP code as part of innovative projects. Fourth generation reactors will enable to tackle a lot of issues such as environmental crisis, affordable energy access, and nuclear waste management. They seem to be one of the keys for a sustainable future. Most of the projects that emerge nowadays include the use of HALEU (high assay low enriched uranium) or MOX recycled fuels. Our projects are part of this dynamic and addresses concrete scientific research issues in the nuclear field. Studies of HALEU package are essential to anticipate the need, therefore the Optimus L (OPTImal Modular Universal Shipping cask technology) designed by NAC (Nuclear Assurance Corporation) international but filled with 20 % enriched uranium dioxide (UO 2 ) will be studied to support safe transportation. However, it seems there is no benchmark with a high correlation with the combination of this fuel and this package to validate MCNP simulations. As such, the study will focus on the development of new criticality safety benchmarks for this case. On the other hand, there is a great need for critical benchmarks in the intermediate energy range with MOX fuel. An IER (Integral Experiment Request) has then be requested to answer it through a partnership between French institution IRSN and U.S. Department of Energy's Nuclear Criticality Safety Program (NCSP). This experience planned for 2025 requires to gather calculated data through a MCNP model to be realized safely. Finally, a presentation of our one-week experience at the DAF as part of our discovery of criticality experiments will be introduced in Appendix 1: Week at the DAF (Device Assembly Facility)

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗
DOE OSTI2025

Critical Simulation Pipeline for COG Suites [Poster]

The CRItical Simulation Pipeline (CRISP) is a Python package for automating validation of reactor criticality benchmarks. CRISP supplies COG—a multi-particle radiation transport code maintained by the Nuclear Criticality Safety Division—with a pipeline to calculate k eff performance for 400+ benchmark experiments with 3,400+ configurations from the International Criticality Safety Benchmark Evaluation Project (ICSBEP). The pipeline includes four stages: materials configuration, input card templating, cluster submission, and results analysis. CRISP includes a command-line interface to facilitate user interaction.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗
DOE OSTI2021

Neutronic Characteristics of ENDF/B-VIII.0 Compared to ENDF/B-VII.1 for Light-Water Reactor Analysis

The Evaluated Nuclear Data File (ENDF)/B-VIII.0 data library was released in 2018. To assess the new data during development and shortly after release, many validation calculations were performed with static, room-temperature benchmarks. Recently, when performing validation of ENDF/B-VIII.0 for pressurized water reactor depletion calculations, a regression in performance compared to ENDF/B-VII.1 was observed. This paper documents extensive benchmark calculations for light-water reactors performed using continuous energy Monte Carlo code with ENDF/B-VII.1 and -VIII.0 and neutronic characteristics of ENDF/B-VIII.0 are discussed and compared to those of ENDF/B-VII.1. It is our recommendation that ENDF/B data library assessment should include reactor-specific benchmark assessments, including depletion cases, such that these types of regressions may be caught earlier in the data development cycle.

21 SPECIFIC NUCLEAR REACTORS AND ASSOCIATED PLANTS↗
DOE OSTI2023

Future of the MUSiC Experiment Data

The Measurements of Uranium Subcritical and Critical (MUSiC) experiment was a highly enriched uranium (HEU) experiment performed at the National Criticality Experiments Research Center (NCERC) executed between December 2020 and April 2021. The experiment intended to measure criticality and reactor kinetics parameters in a bare HEU system. The experiment concurrently measured radiation signatures from the system while utilizing different neutron source types. This was an attempt to benchmark both detectors and analysis techniques against one another for identical measurements. The experiment consisted of the Rocky Flats HEU hemi-shells constructed into ten configurations spanning between deeply subcritical (about 14 kgs) to supercritical (about 60 kgs). Two of the ten configurations were supercritical and the other eight were subcritical. The two supercritical configurations, often referred to as the critical configurations, were documented into an International Criticality Safety Benchmark Evaluation Project (ICSBEP) benchmark evaluation and submitted to the technical review group (TRG). The subcritical configurations of the MUSiC experiment were examined using three different detection systems. The systems include: the NoMAD He-3 neutron detector which consists of 15 He-3 tubes surrounded by a polyethylene matrix, a liquid scintillator system called the Rossi-α Measurement Rapid Organic Discriminating Detector (RAM-RODD), and a trans-stilbene organic scintillator array (OSCAR) provided by the University of Michigan. The NoMAD and RAM-RODD data are planned to be evaluated in two separate ICSBEP evaluations utilizing neutron noise methods such as Feynman Variance-to-Mean, Rossi-α and the pulsed neutron source method. Each of the subcritical configurations were examined using three different source types including: a Cf-252 source in the center, with only the intrinsic neutron source in the HEU, and with an external D-T neutron generator.

11 NUCLEAR FUEL CYCLE AND FUEL MATERIALS↗
DOE OSTI2024

Summary Report of the FY24 DOE Contributions to the GIF VHTR CMVB

The Generation-IV Forum (GIF) Very-High-Temperature Reactor-Computational Methods Validation and Benchmark (VHTR-CMVB) initiative, involving organizations from Korea Atomic Energy Research Institute (KAERI) (South Korea), Institute of Nuclear and New Energy Technology of Tsinghua University (INET) (China), U.S. Department of Energy (DOE) (U.S.), Joint Research Centre (JRC) (Europe), and Japan Atomic Energy Agency (JAEA) (Japan), is dedicated to the verification and validation of tools for High-Temperature Gas-Cooled Reactors (HTGRs) analysis, using data shared by Computational Methods Validation and Benchmark (CMVB) signatories. For FY24, the US DOE CMVB has committed to several critical activities. Under WP1, led by the US, the integration of the High Temperature Gas Cooled Reactor - Pebble-Bed Module (HTR-PM) Phenomena Identification and Ranking Table (PIRT) into the comparative PIRT is progressing, with a new draft of the comparison tables issued earlier this year and currently being utilized by INET for their contribution. Neutronic validation efforts under WP3 include the preparation of the burnup analysis benchmark, preliminary calculations, and the development of reference models and results. In WP2, a validation exercise for hot gas mixing in the lower plenum of HTR-PM is in progress, using experimental data from INET (China) to validate modeling approaches. A model of the experimental facility has been developed using StarCCM+, with initial calculations slated for presentation at the GIF CMVB meeting this fall. Another WP2 activity focuses on validating numerical models for air-cooled Reactor Cavity Cooling System (RCCS) with experimental data from the Wisconsin Madison RCCS facility. A high-fidelity model, developed using NEK-RS, is currently being validated with available data from a low power forced convection test. These efforts are aimed at enhancing and confirming the accuracy of HTGR analysis tools, ensuring their alignment with experimental data and regulatory requirements.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗
DOE OSTI2026

How efficiently can AI recognize Wireless Devices?

This poster presents a hardware benchmarking methodology for a 3-layer CNN waveform classifier deployed using ONNX Runtime on an NVIDIA Jetson AGX Orin. The dataset consist of 9 signal types, -30 to +30 dB SNR with 5dB increments. Benchmarking on the Jetson AGX Orin gave an accuracy of 91.9% and GPU throughput of 107,120 predictions/sec (23× faster than CPU). The Jetson GPU reached approximately 27M samples/sec with stable performance but fell below the 40 MHz rate needed for real-time radio feeds. Sustained testing of 5 minutes confirmed stable performance with no memory leaks, establishing a reproducible benchmarking baseline for future edge-deployment optimization.

99 - GENERAL AND MISCELLANEOUS↗
DOE OSTI2020

Multimodel validation of single wakes in neutral and stratified atmospheric conditions

Previous research has revealed the need for a validation study that considers several wake quantities and code types so that decisions on the trade-off between accuracy and computational cost can be well informed and appropriate to the intended application. In addition to guiding code choice and setup, rigorous model validation exercises are needed to identify weaknesses and strengths of specific models and guide future improvements. Here, we consider 13 approaches to simulating wakes observed with a nacelle-mounted lidar at the Scaled Wind Technology Facility (SWiFT) under varying atmospheric conditions. We find that some of the main challenges in wind turbine wake modeling are related to simulating the inflow. In the neutral benchmark, model performance tracked as expected with model fidelity, with large-eddy simulations performing the best. In the more challenging stable case, steady-state Reynolds-averaged Navier–Stokes simulations were found to outperform other model alternatives because they provide the ability to more easily prescribe noncanonical inflows and their low cost allows for simulations to be repeated as needed. Dynamic measurements were only available for the unstable benchmark at a single downstream distance. These dynamic analyses revealed that differences in the performance of time-stepping models come largely from differences in wake meandering. This highlights the need for more validation exercises that take into account wake dynamics and are able to identify where these differences come from: mesh setup, inflow, turbulence models, or wake-meandering parameterizations. In addition to model validation findings, we summarize lessons learned and provide recommendations for future benchmark exercises.

17 WIND ENERGY↗
DOE OSTI2023

Experimental Characterization of OpenMP Offloading Memory Operations and Unified Shared Memory Support

The OpenMP specification recently introduced support for unified shared memory, allowing implementation to leverage underlying system software to provide a simpler GPU offloading model where explicit mapping of variables is optional. Support for this feature is becoming more available in different OpenMP implementations on several hardware platforms. A deeper understanding of the different implementation’s execution profile and performance is crucial for applications as they consider the performance portability implications of adopting a unified memory offloading programming style. This work introduces a benchmark tool to characterize unified memory support in several OepnMP compilers and runtimes, with emphasis on identifying discrepancies between different OpenMP implementations as to how they various memory allocation strategies interact with unified shared memory. The benchmark tool is used to characterize OpenMP compilers on three leading High Performance Computing platforms supporting different CPU and device architectures. The benchmark tool is used to assess the impact of enabling unified shared memory on the performance of memory-bound code, highlighting implementation differences that should be accounted for when applications consider performance portability across platforms and compilers.

Elwasif, Wael↗