Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “BENCHMARKS”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 163 records · Page 9
All sourcesNASA NTRSDOE OSTI
DOE OSTI2022

Development and Evaluation of Neutron Thermalization Integral Benchmarks for Advanced Reactor Applications (Final Report)

In this project, integral benchmarks that aim to examine thermal neutron scattering data for graphite (ideal and nuclear), light water, and molten salt FLiBe will be analyzed. In particular, the project aims to: 1) Develop the slowing-down-time experiment that was performed at the OREAL facility of ORNL as a graphite benchmarks; 2) Identify and examine a slowing-down-time experiment in light water as a potential benchmark; 3) Reexamine the molten salt MSRE-MSR-EXP-001 benchmark to explicitly capture the impact of TSL variations.

11 NUCLEAR FUEL CYCLE AND FUEL MATERIALS↗
DOE OSTI2026

OptiBench: An Optimization Benchmark Tool for Renewable Energy Problems

We propose a benchmark framework and visualization tool, OptiBench, for analyzing the performance of state-of-the-art optimization solvers across a variety of optimization problems in renewable energy research. Our framework is designed from the ground up in the Julia programming language and enables analysis at scale on high performance computing (HPC) systems. Our visualization tool allows effortless evaluation of optimization solver performance, robustness, and accuracy through intuitive plots, e.g., performance profiles, heat maps, and distribution plots. We have tested three benchmark suites relevant to the modeling of renewable energy systems, viz., CUTEst, PGLib-OPF, and WaterTAP water treatment optimization problems. We illustrate benchmarking of CUTEst using OptiBench on the National Laboratory of the Rockies's (NLR) HPC Kestrel. Our findings indicate that MA57 HSL linear solver demonstrated the best overall performance for an experimental IPOPT implementation. Our work is ongoing and we intend to add support for more optimization solvers and benchmark test suites in the future.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2020

Call for Participation: Collaborative Benchmarking of Functional-Structural Root Architecture Models. The Case of Root Water Uptake

Three-dimensional models of root growth, architecture and function are becoming important tools that aid the design of agricultural management schemes and the selection of beneficial root traits. However, while benchmarking is common in many disciplines that use numerical models, such as natural and engineering sciences, functional-structural root architecture models have never been systematically compared. The following reasons might induce disagreement between the simulation results of different models: different representation of root growth, sink term of root water and solute uptake and representation of the rhizosphere. Presently, the extent of discrepancies is unknown, and a framework for quantitatively comparing functional-structural root architecture models is required. We propose, in a first step, to define benchmarking scenarios that test individual components of complex models: root architecture, water flow in soil and water flow in roots. While the latter two will focus mainly on comparing numerical aspects, the root architectural models have to be compared at a conceptual level as they generally differ in process representation. Therefore, defining common inputs that allow recreating reference root systems in all models will be a key challenge. In a second step, benchmarking scenarios for the coupled problems are defined. We expect that the results of step 1 will enable us to better interpret differences found in step 2. This benchmarking will result in a better understanding of the different models and contribute toward improving them. Improved models will allow us to simulate various scenarios with greater confidence and avoid bugs, numerical errors or conceptual misunderstandings. This work will set a standard for future model development.

Plant Sciences↗
DOE OSTI2020

The ESIF-HPC-2 Benchmark Suite: Preprint

We describe the development of the ESIF-HPC-2 benchmark suite, a collection of kernel and application benchmark codes for measuring computational and I/O performance from single nodes to full HPC systems that was used for acceptance testing in our recent HPC procurement. The configurations of the benchmarks used for our system is presented. We also describe a set of 'dimensions' that can be used to classify benchmarks and assess coverage of a suite systematically. The collection is offered cost-free as a GitHub repository for general usage and further development.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2022

Progress Update on the MUSIC Critical Benchmark

The Measurement of Uranium Subcritical and Critical (MUSiC) experiment was carried out from December 2020 through April 2021 at the National Criticality Experiments Research Center (NCERC). This measurement campaign featured bare configurations of the Rocky Flats highly-enriched uranium (HEU) shells, with each configuration having different numbers of these shells. The goal of the experiment was to test multiple neutron multiplicity detectors and measurement methods for a large range of neutron multiplication values. The large range of multiplications allows researchers to see when the combination of detectors and methods break down as the configurations reach the delayed supercritical window. The critical configurations were the extreme end of the multiplication range in MUSIC configurations. A critical benchmark in the International Criticality Safety Benchmark Evaluation Project (ICSBEP) Handbook is planned to help further validate nuclear data. Even though there are many benchmarks focusing on the fast spectrum for highly-enriched uranium, an additional benchmark that is well documented and up to the modern standard of the handbook would be a welcome addition. Given that there are no other materials such as moderators or significant reflectors, and its similarity to Lady Godiva, it is possible that this could be very useful for validation of 235 U nuclear data in the future.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗
DOE OSTI2022

VERA neutronics high-fidelity benchmark for a modern PWR core design

Nearly 20 years since the origin of the infamous 'Kord Smith Challenge', and after 10 years and hundreds of millions of dollars invested by the US Department of Energy into the development of the Virtual Environment for Reactor Applications (VERA), the capability to accurately simulate a modern pressurized water reactor (PWR) fuel cycle at the fuel rod level with high-fidelity Monte Carlo (MC) stochastic transport methods still evades us. The deterministic methods in VERA make approximations that allow it to be successfully benchmarked against measured data from hundreds of PWR fuel cycles, but none of these benchmarks provided information at the local fuel rod level or allowed quantification of these approximations' impact for modern PWR reload designs. Using MPACT to establish the estimated isotopic distributions and thermal hydraulic conditions in a recent and challenging reactor design, a 2D quarter-core benchmark problem was established and solved consistently with both MPACT, the 51-energy group deterministic transport solver in VERA, and Shift, the continuous-energy MC transport solver in VERA. Impacts of energy groups, ray spacing, and scattering treatment were evaluated. Though MPACT has been previously shown to perform well compared to MC methods for initial Cycle 1 cores with only fresh fuel, this is the first benchmark of MPACT for a challenging modern reload core with depleted fuel and burnable absorbers. This work establishes additional credibility for the methods in MPACT and highlights the efficiency of these methods relative to those in use by MC-based tools. (authors)

21 SPECIFIC NUCLEAR REACTORS AND ASSOCIATED PLANTS↗
DOE OSTI2022

Development of the TREAT M2 experiment as a transient benchmark

A transient benchmark based on the Transient Reactor Test Facility (TREAT) M2 Calibration experiment (M2-CAL) is under development. TREAT, at Idaho National Laboratory, is a graphite moderated air-cooled research reactor which has been used extensively for fuel material testing under extreme and accident conditions. Accurate benchmark models are a beneficial component in the operations, experimental planning, and development of TREAT. In this work, we present a transient benchmark model for the M2-CAL experiment core loading. The benchmark model incorporates the coupling between the Monte Carlo Code SERPENT and the computational fluid dynamics code OpenFOAM to capture the temperature feedback mechanism. The M2-CAL transient 2580 was simulated in this work. A pre-transient analysis was performed to determine the optimum core composition and conditions before the beginning of the transient. The analysis was validated against historic TREAT kinetic measurements and the worth of the transient rod T-2. The axial power distribution in the flux wire for the M2-CAL experiment was determined and contrasted with the experiment. The model was then used to simulate the M2-CAL transient 2580 experiment based on the reported pre-transient and transient conditions. Several transient observables were calculated and compared to the experiment. The model shows good agreement with the experimental power traces, period, and average increase of power at the power ramp (time > 7.8 s). Inverse point kinetics analysis was performed during the period of the power ramp for the model and the experiment based on that, the temperature feedback component was isolated and contrasted with the experimental feedback. (authors)

21 SPECIFIC NUCLEAR REACTORS AND ASSOCIATED PLANTS↗
DOE OSTI2022

Reactor physics benchmark experiments at the JSI TRIGA MARK II reactor - Current status and future outlook

Full text of publication follows. With the development of new high-fidelity computational methods, improvement of nuclear data, and multiphysics modelling, there is an increased need for benchmark experiments to experimentally validate the models, methods and input data. Many of the nuclear facilities designed to perform reactor physics benchmark experiments have been shut down. Therefore, research reactors offer a great opportunity for benchmark experiments, if they are well designed and performed with great care and accuracy. In this presentation we provide an overview of the past and ongoing activities related to benchmark experiments at the Jozef Stefan Institute TRIGA Mark II research reactor. The following experiments have been performed: criticality with fresh fuel, {sup 197}Au(n,γ) and {sup 27}Al(n,α) reaction rates in irradiation channels, absolute and relative {sup 197}Au(n,γ), {sup 235}U(n,f) and {sup 238}U(n,f) reaction rates in the core, burnup, kinetic parameters, control rod worth, isothermal reactivity coefficient, self-shielding, slow and fast (pulse) transients, nuclear heating, delayed and prompt gamma ray production, temperature profiles for multi-physics. Since the existing fleet of research reactors is ageing very rapidly and new experiments are needed, new research reactors should be designed and built to meet the needs of future advanced reactors, education and training, and other technologies in the coming years. We will review planned activities at the JSI TRIGA reactors and plans for the new research reactor in Slovenia. (authors)

21 SPECIFIC NUCLEAR REACTORS AND ASSOCIATED PLANTS↗
DOE OSTI2023

Development of Shielding Benchmarks Using the Godiva IV Assembly

The Nuclear Criticality Safety Program (NCSP) is developing a shielding benchmark using the Godiva IV assembly as a source. Its present status is reviewed herein. Even eight decades into the nuclear era, substantial work remains to develop a database of shielding benchmarks to support future nuclear development. A nuclear simulation is only as good as its supporting data, inputs, and validation basis. Uncertainty in these areas is addressed using conservatism, which adds margin and, occasionally, cost. In many shielding situations, high accuracy is not necessary because additional material is not particularly expensive. After all, 1–2 cm of lead often reduces the gamma ray dose substantially. However, in certain areas, conservatism can add unnecessary cost. These areas include mobile shielding applications such as casks, ships, microreactors, and spacecraft, where weight and, thus, margin is expensive. Although these characteristics are side benefits for NCSP shielding benchmark development, the main driver is enabling more reliable placement of criticality accident alarm systems (CAASs) in nuclear material facilities, such as those dedicated to the production of advanced reactor fuels. CAAS placement relies on more than accurate data and code validation. It also relies on sufficiently accurate materials specifications, geometry specifications, and a well-defined, alarm-producing baseline accident. All these things require tacit knowledge and understanding of the problem being evaluated. Benchmarks can help ensure this understanding.

21 SPECIFIC NUCLEAR REACTORS AND ASSOCIATED PLANTS↗
DOE OSTI2024

RELAP5-3D in the OECD-NEA HTGR Thermal Hydraulics Benchmark

The OECD-NEA HTGR Thermal Hydraulics benchmark uses data from the High Temperature Test Facility for a series of code-to-code and code-to-data exercises aimed at improving the state of knowledge on existing thermal hydraulics modeling and simulation tools for prismatic HTGR applications. This benchmark includes problems representing hot gas mixing in the lower plenum, the depressurized conduction cooldown accident and the pressurized conduction cooldown accident. This presentation discusses the benchmark and RELAP5-3D's role in that benchmark, including as a tool for predicting behavior in the core and as a tool for providing boundary conditions to computational fluid dynamics analysis in the lower plenum.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗
DOE OSTI2026

Benchmarking Cylindrical Blast Wave Theory Against the OSIRIS-REx Sample Return Capsule Reentry

Weak shock theory based on cylindrical blast waves has been used to interpret meteor infrasound, but it has not been systematically benchmarked against a non-ablating hypersonic source with independently known parameters. The objective of this study is not to propose a new theoretical framework, but to evaluate the operational validity of the existing suite of blast radius formulations against a high-fidelity ground truth dataset. The OSIRIS-REx Sample Return Capsule reentry on 24 September 2023 provides such a benchmark because the capsule geometry, trajectory, and infrasound emission points are constrained from mission data and ray tracing, reducing source-side uncertainty associated with ablation. Using observations from 39 infrasound stations, this benchmarking study evaluates six published blast radius (${R}_{0}$ ) formulations and three weak-shock transition coefficients ( C ) within a stratified atmospheric propagation model to predict signal period and peak overpressure. The benchmarking identifies the Sakurai formulation as the best-performing formulation for non-ablating bodies, with the Jones/Plooster formulation performing comparably when a physically appropriate C is adopted. Sakurai and Jones/Plooster yield linear-period median absolute percentage residuals of 9% and 11%, respectively. The period predictions show only weak sensitivity to C at these propagation distances. The Mach-diameter approximation commonly used in meteor studies overestimates ${R}_{0}$ by more than a factor of 3 in the absence of ablation. Finally, these results establish a performance baseline for applying cylindrical blast wave theory to effectively non-ablating hypersonic bodies and demonstrate that the signal period is a robust observable for constraining ${R}_{0}$.

atmospheric entry↗
DOE OSTI2021

Monte Carlo neutronics benchmarks on nuclear fuel depletion: A review

Monte Carlo (MC) neutronics codes are used widely for academic and industrial needs. Several schemes of coupling MC neutronics code with isotope generation and depletion code exist, which are used for performing nuclear fuel depletion simulations. These simulations can estimate the inventory of isotopes in neutron irradiated nuclear reactor fuel. However, the accuracy of these simulations shall be validated through experiments. MC codes are seldom validated by isotopic benchmarks compared to criticality benchmarks. This work compiles and analyzes the fuel depletion benchmarks and validations used to analyze the performance of MC-based fuel depletion neutronics codes. Analyses of these benchmarks and validations showed that the computed concentrations of 133 Cs, 135 Cs, 137 Cs, 148 Nd, 239 Pu, 240 Pu, and 241 Pu in the irradiated fuel by the depletion codes agreed with the measured values within 10% error. However, the computed concentrations of 125 Sb, 242 Cm, 243 Cm, 244 Cm, 245 Cm, and 246 Cm had errors more than 15% compared to the measured values. As a result, Ventina depletion code showed the most accurate predictions for the greatest number of isotope concentrations compared to ORIGEN2 and CINDER90.

11 NUCLEAR FUEL CYCLE AND FUEL MATERIALS↗
DOE OSTI2026

SoK: What does it Mean to Benchmark Database Forensics?

Relational Database Management Systems are the backbone of modern enterprises and public-sector services, and are thus frequent targets of security incidents, insider threats, and thorough regulatory audits. Consequently, databases have become key sources of digital evidence, requiring investigators to reconstruct past activity from audit logs, transaction logs, and backups. Although benchmarking frameworks such as those developed by the Transaction Processing Performance Council (TPC) are widely used to evaluate database performance, they do not capture forensic requirements such as evidentiary completeness, tamper-evidence, chain of custody, or regulatory compliance under GDPR and CCPA. This survey examines the emerging domain of forensic database benchmarking. We gathered prior research on database forensics, secure logging, and tamper-evident data structures; we analyze modern forensic-ready features in commercial and open-source systems (SQL Server Ledger, Oracle Blockchain Tables, PostgreSQL pgAudit, Db2 Audit, Aurora Database Activity Streams, Oracle Real Application Security and IBM Guardium) and assess why existing benchmarks are insufficient. We propose forensic workloads, metrics, and methodologies that incorporate adversarial stressors, deleted-record recovery, and backup analysis. We also identify open research problems and call for a community-driven forensic benchmark suite. The result is an idea for evaluating not only database performance but also forensic soundness, bridging the gap between system engineering, compliance, and digital investigations.

Lenard, Ben↗
DOE OSTI2021

Development of a U-19Pu-10Zr fuel performance benchmark case based on the IFR-1 experiment

Metallic nuclear fuels are subject to research and development for use in advanced reactors. Robust, accurate metallic fuel performance models are important for the design, safety analysis, and licensing of these reactors. However, metallic fuel performance models require additional development; they are not as mature as UO 2 fuel performance models. A benchmark case based on the IFR-1 experiment was developed to better gauge the accuracy of existing models, identify models for high-priority development, and potentially quantify any future improvements made by further model development.This work collected publicly available information on the IFR-1 experiment and used it to develop the benchmark case. Fuel behavior during the IFR-1 irradiation was simulated by using the fuel performance code BISON, and the predicted results were compared with postirradiation examination data from the IFR-1 experiment. Furthermore, a sensitivity study and tuning studies were performed as a preliminary investigation into the causes of inaccurate temperature and dimensional change predictions.The benchmark predicted reasonably accurate values for the burnup and fission gas release. There was error in the predicted temperatures, which could be explained by uncertainty in the input parameters and legacy temperatures. BISON over-predicted dimensional changes in the fuel and cladding. The sensitivity study showed that the dimensional changes were most sensitive to the fuel swelling anisotropy and the cladding void swelling model. Future benchmark and model development should focus on cladding swelling behaviors to improve dimensional change predictions.

11 NUCLEAR FUEL CYCLE AND FUEL MATERIALS↗
DOE OSTI2020

Spin Splitting Energy of Transition Metals: A New, More Affordable Wave Function Benchmark Method and Its Use to Test Density Functional Theory

Accurately predicting the spin splitting energy of chemical species is important for understanding their reactivity and magnetic properties, but it is very challenging, especially for molecules containing transition metals. One impediment to progress is the scarcity of accurate benchmark data. Here we report a set of calculations designed to yield reliable benchmarks for simple transition-metal complexes that can be used to test density functional methods that are affordable for large systems of more practical interest. Various wave function methods are tested against experiment for Fe 2+ , Fe 3+ , and Co 3+ , including CASSCF, CASPT2, CASPT3, MRCISD, MRCISD+Q, ACPF, AQCC, CCSD(T), and CASPT2/CCSD(T) and also a new method called CASPT2.5, which is performed by taking the average of the CASPT2 and CASPT3 energies. Furthermore, we find that MRCISD+Q, ACPF, and AQCC require smaller active spaces for good accuracy than are required by CASPT2 and CASPT3, and this aspect may be important for calculations on larger molecules; here we find that CASPT2.5 extrapolated to a complete basis set is the most suitable method—in terms of computational cost and in terms of accuracy on monatomic systems—and therefore we chose this method for molecular benchmarks. Then Kohn–Sham density functional calculations with 60 exchange-correlation functionals are tested for FeF 2 , FeCl 2 , and CoF 2 . We find that MN15-L, M06-SX, and revM06 have very good agreement with CASPT2.5 benchmarks in terms of both the spin splitting energy and the optimized geometry for each spin state. In addition, we recommend def2-TZVP as the most suitable basis set to perform density functional calculations for molecular spin splitting energies; extra polarization functions in the basis set do not help to increase the accuracy of the spin splitting energy in KS calculations.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2022

Scientific machine learning benchmarks

Deep learning has transformed the use of machine learning technologies for the analysis of large experimental datasets. In science, such datasets are typically generated by large-scale experimental facilities, and machine learning focuses on the identification of patterns, trends and anomalies to extract meaningful scientific insights from the data. In upcoming experimental facilities, such as the Extreme Photonics Application Centre (EPAC) in the UK or the international Square Kilometre Array (SKA), the rate of data generation and the scale of data volumes will increasingly require the use of more automated data analysis. Furthermore, at present, identifying the most appropriate machine learning algorithm for the analysis of any given scientific dataset is a challenge due to the potential applicability of many different machine learning frameworks, computer architectures and machine learning models. Historically, for modelling and simulation on high-performance computing systems, these issues have been addressed through benchmarking computer applications, algorithms and architectures. Extending such a benchmarking approach and identifying metrics for the application of machine learning methods to open, curated scientific datasets is a new challenge for both scientists and computer scientists. Here, we introduce the concept of machine learning benchmarks for science and review existing approaches. As an example, we describe the SciMLBench suite of scientific machine learning benchmarks.

42 ENGINEERING↗
DOE OSTI2022

A critical comparison of CH∙∙∙π versus π∙∙∙π interactions in the benzene dimer: obtaining benchmarks at the CCSD(T) level and assessing the accuracy of lower scaling methods

In this study, we have established CCSD(T)/CBS (Complete Basis Set) limits for 3 stationary points on the benzene dimer potential energy surface, corresponding to the π∙∙∙π (parallel displaced or PD(C 2h ), minimum) and CH∙∙∙π (T-shaped or T(C 2v ), transition state) and tilted T-shaped (or TT(Cs), minimum) bonding scenarios considering both the structure and binding energy. The CCSD(T)/CBS binding energies are -2.65 ± 0.02 (PD), -2.74 ± 0.03 (T), and -2.83 ± 0.01 kcal mol -1 (TT). To this end, the CH∙∙∙π is ~0.2 kcal mol -1 stronger than the π∙∙∙π interaction, whereas the tilting of the CH donating benzene molecule with respect to the other benzene is worth 0.1 kcal mol -1 . As previously discussed in the literature, the MP2 level of theory does not provide a close match for either the energy or structure, yet the SCS-MP2 yields structures in excellent agreement with respect to the CCSD(T) result. It is found that the SCS-MI-MP2 also gives optimized structures very close to SCS-MP2 (within ~0.01 Å of the benchmark). Despite the closer match in structure, the spin-biased MP2 methods (SCS-, SCS-MI-, and SOS-MP2) incorrectly predict the relative stabilities of the isomers. That said, none of the spin biased MP2 methods offers a good compromise between energy and structure for the systems examined. Finally, the CCSD(T)/CBS benchmarks were used to assess the performance of 13 DFT functionals selected from different rungs of Jacob's ladder. Several functionals such as TPSS-D3, B3LYP-D3, B97-D, B97-D3, and B2PLYP-D3 provided a good description of the binding energies for both CH∙∙∙π and π∙∙∙π interactions, yielding values within 6% of the CCSD(T)/CBS benchmark values. Unlike the MP2 methods, these functionals correctly predict the relative stability of the PD(C 2h ) and T(C 2v ) dimers. Further, we find that there is no systematic improvement as Jacob's ladder is ascended (increased complexity of functional). The best functionals that result in a good compromise between structure and energy accuracy are B97-D3 and B2PLYP-D3 for both the CH∙∙∙π and π∙∙∙π interaction. Despite the impressive performance of these functionals, a challenge that remains is ensuring the transferability of these density functionals in accurately describing the interaction between dimers of larger aromatic molecules, the latter requiring high-level benchmarks for these systems.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2024

Availability of Critical Benchmark Experiments for the Pebble Tanker Transportation Model for Nuclear Criticality Safety Validation of TRISO Pebbles

This study addresses the need for comprehensive investigations into TRi-structural ISOtropic (TRISO) fuel pebble transportation validation. In this work, an exploratory model, the pebble tanker(PT), was developed with the aim of facilitating the validation of nuclear criticality safety calculations in the context of industrial-scale transportation of TRISO fuel. The PT model was designed to investigate the availability and applicability of critical benchmark experiments crucial for assessing the transportation of these pebbles. This work incorporated sensitivity/uncertainty (S/U) similarity studies to quantify the applicability of critical benchmark experiments and to address nuclear data uncertainties in the context of TRISO transportation. Two container models were investigated: one for the Hermes-type pebble and one for the Pebble Bed Modular Reactor (PBMR)–type pebble. The models were simplified, considering fuel, containment, and either water or air, to enable a focus on the underlying physics of applications involving TRISO fuel pebbles using the PT model. A crucial aspect under consideration was the capacity of the transport package to hold pebbles while ensuring subcriticality in the flooded state. An approach in the criticality validation process involves assessing the similarity between systems through an integral index parameter evaluation. This involves calculating a correlation coefficient (referred to as c k ) based on shared nuclear data–induced uncertainty between a benchmark experiment and the application of the PT model. To facilitate this analysis, the SCALE tools, particularly the CSAS6-Shift, TSUNAMI-3D-Shift, and TSUNAMI-IP sequences, were employed for comprehensive studies in neutronics and S/U analysis. Our findings showed that there are sufficient critical experimental benchmarks to perform this validation of the PT model in the most reactive state, i.e. when the tanker is flooded. This paper provides valuable insights into validating a transport package for Generation IV TRISO fuel pebbles.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗