Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Benchmarks”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 145 records · Page 8

Engagement opportunities in OECD NEA benchmark development

A myriad of opportunities is available to collaborate via international benchmark exercises and experimental data preservation activities. Many such opportunities abound under the auspices of the Nuclear Science Committee of the Organisation for Economic Co-operation and Development Nuclear Energy Agency (NEA). Key projects and activities of relevance to the development of advanced reactors design include the International Criticality Safety Benchmark Evaluation Project (ICSBEP), the International Reactor Physics Experiment Evaluation Project (IRPhEP), the International Assay Data of Spent Nuclear Fuel Database (SFCOMPO), the Shielding Integral Benchmark and Archive Database (SINBAD), and The International Experimental Thermal HYdraulicS Database (TIETHYS), and various cooperative benchmark exercises. Interested participants are encouraged to contact the leadership and secretariat of the various Technical Working Groups and Working Parties to become more engaged. This paper provides a summary of the current benchmark exercises and experimental databases available for international participation.

96 KNOWLEDGE MANAGEMENT AND PRESERVATION↗

Intrinsic value of the international benchmark projects, ICSBEP and IRPhEP, for advanced reactor development

The International Criticality Safety Benchmark Evaluation Project, a sanctioned program under the auspices of the Nuclear Energy Agency of the Organisation for Economic Co-operation and Development, has been a highly successful and productive collaboration, now encompassing over 5,000 evaluated experimental benchmarks trusted and relied upon throughout the international nuclear communities. The success of this project led to the development of the International Reactor Physics Experiment Evaluation Project, which is dedicated to the evaluation of benchmark experiment data to sustain current and future reactor physics validation needs. These exemplary programs, and their widely utilized handbooks, serve as gold standards to which other databases strive to emulate. The purpose of the two projects is to preserve modern and legacy experimental data and evaluate it in a standardized handbook format to provide quality benchmarks to support modern and future criticality safety and reactor physics validation. These two projects have often served as the mechanism through which historic and modern neutronics experiments are evaluated and shared across international borders, to best provide unique, high-quality peer-reviewed, and often otherwise unavailable, benchmark data. The contents of these handbooks are utilized not only in validating criticality safety, reactor physics, and advanced reactor calculations, but are used to validate neutronics calculations and nuclear data for most other nuclear applications. This paper discusses both international programs and available content to enable advanced reactor design validation.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗

Spacecraft Observatory Benchmark Problem for Optical Disturbance Rejection

Since mid-1980s NASA has launched several first-generation interplanetary observatory missions. A salient feature of these missions has been the mounting of the optical payload on top of a large flexible space structure. This imposes challenging control-structure interaction problems (CSI) that have been studied over several decades. The Dec. 2021 launch of the James Webb Telescope marks the latest example of such a mission. One way to encourage development of novel control methods is to provide a benchmark problem for researchers. By the mid-2000s, there were a few well-known benchmark problems developed by the control community with a focus on CSI applications. However, in the past 15 years, the control challenges in optical observation missions have shifted from CSI to line-of-sight (LOS) precision pointing. This is because popular designs of the second-generation (Gen II) space earth missions have eliminated the large structure interface by mounting the optical payload directly on top of a rigid bus platform. The focus now is to achieve accurate pointing with maximum rejection of the surrounding disturbances. The NASA Safety and Engineering Center (NESC) has tasked T he Aerospace Corporation to develop a Benchmark Problem to study the challenging control issues that Gen II observatory missions encounter. This software tool is to be released as a public domain platform aimed at government, academic, and industry researchers. The focus of this benchmark problem is for researchers to design a set of innovative payload control laws with the maximum Optical Disturbance Rejection (ODR) capability to meet a set of pre-defined μrad-level of LOS jitter requirements. This paper describes the development of the benchmark problem. Aerospace/NASA will provide an integrated LOS plant model and disturbance/command profiles for users to run their design and simulation with as well as a user’s guide describing the required interfaces. The Aerospace Corp is also working to develop a hardware fast steering mirror testbed for users to demonstrate their innovative design in real hardware, if so desired.

Spacecraft Observatory Benchmark Problem↗

Using HEP experiment workflows for the benchmarking and accounting of WLCG computing resources

Benchmarking of CPU resources in WLCG has been based on the HEP-SPEC06 (HS06) suite for over a decade. It has recently become clear that HS06, which is based on real applications from non-HEP domains, no longer describes typical HEP workloads. The aim of the HEP-Benchmarks project is to develop a new benchmark suite for WLCG compute resources, based on real applications from the LHC experiments. By construction, these new benchmarks are thus guaranteed to have a score highly correlated to the throughputs of HEP applications, and a CPU usage pattern similar to theirs. Linux containers and the CernVM-FS filesystem are the two main technologies enabling this approach, which had been considered impossible in the past. In this paper, we review the motivation, implementation and outlook of the new benchmark suite.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗

Validation of MCNP Critical Benchmarks Models of Highly Enriched Uranium Cylinders

A new centralized repository of high-quality MCNP models of critical benchmark experiments is currently under development at Los Alamos National Laboratory (LANL). The benchmark experiments are described in the International Criticality Safety Benchmark Evaluation Project (ICSBEP) Handbook, and the initial set of benchmark models are derived from the Whisper Suite provided with MCNP6.2. This effort is a collaboration among the Nuclear Criticality Safety, Nuclear Data, and Monte Carlo code development/application organization at LANL. The objective is to create a current single LANL benchmark collection that includes the latest ICSBEP revision that has a formal review and revision process, is contained in an open- source repository, and utilizes new Python tools for improved input and output file review.

11 NUCLEAR FUEL CYCLE AND FUEL MATERIALS↗

Evaluation of Oak Ridge National Laboratory Health Physics Research Reactor Operation Data for Critical Benchmark Creation [Abstract]

The Oak Ridge National Laboratory (ORNL) Health Physics Research Reactor (HPRR) was a research reactor designed and built at ORNL in 1961. The critical assembly was using a highly enriched uranium and molybdenum alloy as the fuel, and it could be operated in steady-state or burst modes. The reactor was used for about 25 years to produce a lot of publications related to dosimetry, radiobiology and radiation detectors testing before its decommissioning in 1987. In recent years, the idea of using legacy operation data from the to create a valuable critical accident alarm system shielding benchmark arose. Such a benchmark has been submitted to the International Criticality Safety Benchmark Experiment Project (ICSBEP) Technical Review Group for a potential inclusion in the 2022 version of the handbook. Another way to use the valuable data from the operation of the HPRR is to evaluate the feasibility of the creation of a subcritical or prompt supercritical benchmark for inclusion in the ICSBEP or the International Reactor Physics Experiments Evaluation Project (IRPhEP) handbooks. To initiate a burst, the HPRR had to be operated in a slightly subcritical state for a few minutes. Then, the insertion of the burst control rod would greatly increase the reactivity of the system and start the burst. No critical configuration of the HPRR critical assembly could be located. The only information available concerns stable subcritical and prompt supercritical states, found in a burst experiments’ logbook. In the recovered logbook pages, information about 8 different bursts is available. The information includes the rods positions before and during a burst, the recorded subcritical reactor period and reactivity, and the burst fission yield derived from the temperature elevation sulfur pellet irradiation analysis. By using the HPRR logbook information and the as-built drawings of the critical assembly, a highly detailed model of the HPRR was created with SCALE 6.2.4/KENO-VI. Eight KENO-VI models were created to replicate the sub-critical assembly configurations described in the eight bursts from the recovered logbook pages. KENO-VI calculates k eff and it can be linked to a reactivity value in cents by using the delayed neutron fraction B eff , also calculated by KENO-VI. KENO-VI can also be used to model the prompt super-critical configurations of the HPRR and to assess the similarity with the burst measurements by comparing the calculated k eff and the measured fission yields between each burst. Unfortunately, high uncertainty exist and the obtained discrepancies between experiments and calculation results are high, compromising the creation of a valuable critical benchmark from HPRR operation data. The reasons of the discrepancies and potential ways to solve them are explored.

21 SPECIFIC NUCLEAR REACTORS AND ASSOCIATED PLANTS↗

Status of the OECD/NEA Watts Bar unit 1 benchmark and calculations of local reactor core power data of the Zero Power physics tests

The paper aims to provide an update on the status of the Organization for Economic Cooperation and Development (OECD) / the Nuclear Energy Agency (NEA) Tennessee Valley Authority (TVA) Watts Bar 1 (WB1) Multi-Physics Multi-Cycle depletion benchmark and the results associated with Exercise 1. The benchmark relies on the set of benchmark progression problems developed by the Department of Energy (DOE) Consortium for the Advanced Simulation of Light Water Reactors (CASL) for the Virtual Environment for Reactor (VERA), which are based on real plant design and operational data. The OECD/NEA TVA WB1 benchmark is designed for validation of both traditional and novel high-fidelity multi-physics codes to analyze Pressurized Water Reactors (PWR) depletion cycles. The activities are conducted under the Expert Group on Reactor Systems Multi-Physics (EGMUP) at NEA/OECD. In this work, we analyze Exercise 1 of the benchmark: stand-alone Three-Dimensional (3D) neutronics at Start-up Zero Power Physics Test (ZPPT) at Hot Zero Power Conditions (HZP) and the resulting power maps. The code used to model the exercise is the continuous energy Monte Carlo code Serpent 2.1.31 along with the nuclear data library ENDF/B-VII.1. Serpent results are compared with the high-fidelity deterministic code MPACT, which is part of VERA. The paper presents three selected results from power calculations required output. The results compared are the normalized axially integrated radial core power maps, the normalized axial averaged core power shapes, and the normalized core hottest and coldest assemblies radial power maps. The Root Mean Square Deviation (RMSD) between Serpent and MPACT is 0.51 % for the axially integrated radial core power map, 1.41 % for the axial averaged core power shape, 2.42 % for the hottest assembly axial power shape, 0.88 % for the coldest assembly axial integrated hottest assembly radial pin power map, and 0.17 % for the axially integrated coldest assembly radial pin power map. (authors)

21 SPECIFIC NUCLEAR REACTORS AND ASSOCIATED PLANTS↗

Generation of group constants at GRS for the Rostov-2 benchmark

The OECD/NEA benchmark entitled 'Reactivity compensation of boron dilution by stepwise insertion of control rod cluster into the VVER-1000 core' aims at validating and assessing high fidelity multi-physics simulation code capabilities by comparison with the high-quality measurements performed at the nuclear power plant Rostov unit 2. The benchmark is divided into 2 phases: assembly wise and pin-by-pin resolution of the multi-physics problems. Multi-physics simulation requires the generation of an accurate parametrized few-group cross-section library, and so this benchmark offers an opportunity to validate the methodology for their generation. Based on the specifications of the benchmark, the core simulator KMACS developed at GRS was used to generate a library of cross-sections as well as the burnup distribution required to model the transient scenario of the benchmark at the assembly level. The assessment of the library is performed through comparison with measured values obtained at the initial state of the transient. Good agreement was obtained in terms of critical boron concentration, peaking factors and power profiles. (authors)

21 SPECIFIC NUCLEAR REACTORS AND ASSOCIATED PLANTS↗

25-Pin metallic fuel performance benchmark case based on the EBR-II X430 experiments series

A metallic fuel benchmark case was developed for the fuel performance code BISON based on 25 uranium-zirconium and uranium-plutonium-zirconium pins of the Experimental Breeder Reactor II X430 experiment series. Results of the benchmarks were compared with measurements and calculations made at the time of the experiment as well as subsequent measurements reported in 2019. The comparisons were used to quantify the accuracy of the BISON predictions and to identify patterns in the BISON differences.BISON predicted burnup, plenum pressure, and fission gas release fractions accurately. BISON temperature predictions were somewhat cooler than the temperatures determined at the time of the experiment but appeared to be reasonably accurate considering uncertainties in the legacy temperature calculations, uncertainties in the legacy linear heat rate calculations, and the high sensitivities of the BISON-predicted temperatures to BISON inputs. Fuel axial elongation predictions had errors correlated to fuel composition; BISON tended to underpredict the elongation of binary fuels and overpredict the elongation of ternary fuels. BISON cladding radial dilation predictions were also significantly lower than legacy PIE measurements.Recommendations were made to improve the BISON fuel gaseous swelling model to account for fuel composition, to add additional capabilities to the coolant channel temperature model to ease benchmark development, and to continue developing benchmark cases based on a wide range of experiments in several reactors. Once a wide array of benchmarks is developed, an attempt can be made to enhance or calibrate BISON models to improve the cladding dilation predictions.

11 NUCLEAR FUEL CYCLE AND FUEL MATERIALS↗

Extending quantum-mechanical benchmark accuracy to biological ligand-pocket interactions

Predicting the binding affinity of ligands to protein pockets is key in the drug design pipeline. The flexibility of ligand-pocket motifs arises from a range of attractive and repulsive electronic interactions during binding. Accurately accounting for all interactions requires robust quantum-mechanical (QM) benchmarks, which are scarce for ligand-pocket systems. Additionally, disagreement between “gold standard” Coupled Cluster (CC) and Quantum Monte Carlo (QMC) methods casts doubt on many benchmarks for larger non-covalent systems. We introduce the “QUantum Interacting Dimer” (QUID) benchmark framework containing 170 non-covalent (non-)equilibrium systems modeling chemically and structurally diverse ligand-pocket motifs. Symmetry-adapted perturbation theory shows that QUID broadly covers non-covalent binding motifs and energetic contributions. Robust binding energies are obtained using complementary CC and QMC methods, achieving agreement of 0.5 kcal/mol. The benchmark data analysis reveals that several dispersion-inclusive density functional approximations provide accurate energy predictions, though their atomic van der Waals forces differ in magnitude and orientation. Contrarily, semiempirical methods and empirical force fields require improvements in capturing non-covalent interactions (NCIs) for out-of-equilibrium geometries. The wide span of NCIs, highly accurate interaction energies, and analysis of molecular properties take QUID beyond the “gold standard” for QM benchmarks of ligand-protein systems.

Puleva, Mirela [University of Luxembourg, Luxembou↗

Quantum chemistry as a benchmark for near-term quantum computers

We present a quantum chemistry benchmark for noisy intermediate-scale quantum computers that leverages the variational quantum eigensolver, active-space reduction, a reduced unitary coupled cluster ansatz, and reduced density purification as error mitigation. We demonstrate this benchmark using 4 of the available qubits on the 20-qubit IBM Tokyo and 16-qubit Rigetti Aspen processors via the simulation of alkali metal hydrides (NaH, KH, RbH), with accuracy of the computed ground state energy serving as the primary benchmark metric. We further parameterize this benchmark suite on the trial circuit type, the level of symmetry reduction, and error mitigation strategies. Our results demonstrate the characteristically high noise level present in near-term superconducting hardware, but provide a relevant baseline for future improvement of the underlying hardware, and a means for comparison across near-term hardware types. We also demonstrate how to reduce the noise in post processing with specific error mitigation techniques. Particularly, the adaptation of McWeeny purification of noisy density matrices dramatically improves accuracy of quantum computations, which, along with adjustable active space, significantly extends the range of accessible molecular systems. We demonstrate that for specific benchmark settings and a selected range of problems, the accuracy metric can reach chemical accuracy when computing over the cloud on certain quantum computers.

97 MATHEMATICS AND COMPUTING↗

A robust benchmark for detection of germline large deletions and insertions

New technologies and analysis methods are enabling genomic structural variants (SVs) to be detected with ever-increasing accuracy, resolution and comprehensiveness. To help translate these methods to routine research and clinical practice, we developed a sequence-resolved benchmark set for identification of both false-negative and false-positive germline large insertions and deletions. To create this benchmark for a broadly consented son in a Personal Genome Project trio with broadly available cells and DNA, the Genome in a Bottle Consortium integrated 19 sequence-resolved variant calling methods from diverse technologies. The final benchmark set contains 12,745 isolated, sequence-resolved insertion (7,281) and deletion (5,464) calls ≥50 base pairs (bp). The Tier 1 benchmark regions, for which any extra calls are putative false positives, cover 2.51 Gbp and 5,262 insertions and 4,095 deletions supported by ≥1 diploid assembly. We demonstrate that the benchmark set reliably identifies false negatives and false positives in high-quality SV callsets from short-, linked- and long-read sequencing and optical mapping.

59 BASIC BIOLOGICAL SCIENCES↗

Estimating Code Biases for Criticality Safety Applications with Few Relevant Benchmarks

Criticality safety analyses rely on the availability of relevant benchmark experiments to determine justifiable margins of subcriticality. When a target application lacks neutronically similar benchmark experiments, validation studies must provide justification to the regulator that the impact of modeling and simulation limitations is well understood for the application and often must provide additional subcritical margin to ensure safe operating conditions. This study estimated the computational bias in the critical eigenvalue for several criticality safety applications supported by only a few relevant benchmark experiments. The accuracy of the following three methods for predicting computational biases was evaluated: the Upper Subcritical Limit STATisticS (USLSTATS) trending analysis method; the Whisper nonparametric method; and TSURFER, which is based on the generalized linear least-squares technique. Here, these methods were also applied to estimate computational biases and recommended upper subcriticality limits for several critical experiments with known biases and for several cases from a blind benchmark study. The methods are evaluated based on both the accuracy of their predicted computation bias and upper subcriticality limit estimates, as well as on the consistency of the methods’ estimates, as the model parameters, covariance data libraries, and set of available benchmark data were varied. Data assimilation methods typically have not been used for criticality safety licensing activities, and this study explores a methodology to address concerns regarding the reliability of such methods in criticality safety bias prediction applications.

98 NUCLEAR DISARMAMENT, SAFEGUARDS, AND PHYSICAL P↗

Performance Portability Evaluation of OpenCL Benchmarks across Intel and NVIDIA Platforms

We evaluate the capabilities of vendor-provided OpenCL implementations for performance portability across multiple computing platforms. The Rodinia benchmark suite is used for this evaluation. We apply the metric defined by Pennycook et al., and we use roofline efficiency from the Roofline performance model as the "performance efficiency" in the metric's definition. We found that the delivered performance portability is similar for several benchmarks, even if the roofline-based performance efficiencies across platforms are very different among the benchmarks. To help distinguish between these instances, we extend the metric by adding the standard deviation of the performance efficiencies for each benchmark. We argue that the standard deviation gives additional insight into performance portability assessment since it adds the performance variability across platforms. Additionally, we discuss the challenges to measure performance portability associated with algorithms and system software. In terms of algorithms, we need to carefully construct the benchmarks and appropriately use the concurrency available on a platform. In terms of system software, we depend on the vendor performance tools to support the desired programming model and runtime to be able to measure the metrics of interest.

GPU↗

miniVite: A Graph Analytics Benchmarking Tool for Massively Parallel Systems

Benchmarking of high performance computing sys-tems can help provide critical insights for ef?cient design of computing systems and software applications. Although a large number of tools for benchmarking exist, there is a lack of rep-resentative benchmarks for the class of irregular computations as exempli?ed by graph analytics. In this paper, we propose miniVite as a representative graph analytics benchmark tool to test a variety of distributed-memory systems. Graph clustering, popularly known as community detection, is a prototypical graph operation used in numerous scienti?c computing and analytics applications. The goal of clustering is to partition a graph into clusters (or communities) such that each cluster consists of vertices that are densely connected within the cluster and sparsely connected to the rest of the graph. Modu-larity optimization is a popular technique for identifying clusters in a graph. Ef?cient parallelization of modularity optimization-based algorithms is challenging. One successful approach was conceived in Vite, a distributed-memory implementation of the Louvain algorithm that incorporates several heuristics. We introduce miniVite as a representative but simpli?ed variant of Vite, to serve as a prototypical graph analytics benchmarking tool. Unlike other graph-based methods such as breadth-?rst search and betweenness centrality, miniVite represents highly complex computational patterns stressing a variety of system features, which can provide crucial insight for co-design of future computing systems.

community detection, parallel, MPI↗

WfBench: Automated Generation of Scientific Workflow Benchmarks

The prevalence of scientific workflows with high computational demands calls for their execution on various distributed computing platforms, including large-scale leadership-class high-performance computing (HPC) clusters. To handle the deployment, monitoring, and optimization of workflow executions, many workflow systems have been developed over the past decade. There is a need for workflow benchmarks that can be used to evaluate the performance of workflow systems on current and future software stacks and hardware platforms.We present a generator of realistic workflow benchmark specifications that can be translated into benchmark code to be executed with current workflow systems. Our approach generates workflow tasks with arbitrary performance characteristics (CPU, memory, and I/O usage) and with realistic task dependency structures based on those seen in production workflows. We present experimental results that show that our approach generates benchmarks that are representative of production workflows, and conduct a case study to demonstrate the use and usefulness of our generated benchmarks to evaluate the performance of workflow systems under different configuration scenarios.

Coleman, Taina↗

Assessment of four strain energy decomposition methods for phase field fracture models using quasi-static and dynamic benchmark cases

Abstract Strain energy decomposition methods in phase field fracture models separate strain energy that contributes to fracture from that which does not. However, various decomposition methods have been proposed in the literature, and it can be difficult to determine an appropriate method for a given problem. The goal of this work is to facilitate the choice of strain decomposition method by assessing the performance of three existing methods (spectral decomposition of the stress or the strain and deviatoric decomposition of the strain) and one new method (deviatoric decomposition of the stress) with several benchmark problems. In each benchmark problem, we compare the performance of the four methods using both qualitative and quantitative metrics. In the first benchmark, we compare the predicted mechanical behavior of cracked material. We then use four quasi-static benchmark cases: a single edge notched tension test, a single edge notched shear test, a three-point bending test, and a L-shaped panel test. Finally, we use two dynamic benchmark cases: a dynamic tensile fracture test and a dynamic shear fracture test. All four methods perform well in tension, the two spectral methods perform better in compression and with mixed mode (though the stress spectral method performs the best), and all the methods show minor issues in at least one of the shear cases. In general, whether the strain or the stress is decomposed does not have a significant impact on the predicted behavior.

Zhang, Shuaifang↗

Benchmark Analysis of the HTR-10 with the MAMMOTH Reactor Physics Application

This report documents a set of benchmarks used to validate the Serpent and MAMMOTH neutron transport models of the HTR-10 reactor developed at Idaho National Laboratory. The high-fidelity Serpent Monte Carlo models of the HTR-10 critical and full core configurations include both random, discrete distributions of TRISO particles in the pebbles and random, discrete pebble distributions in the pebble bed core. The Serpent results agree very well with the critical and control rod worth measurements provided in the International Re- actor Physics Experiment Evaluation Project (IRPhEP) report if the ENDF/B-VII.r1 data- set is used. These Serpent models are subsequently used to prepare cross sections and flux tallies in a 10 coarse energy group structure for the MAMMOTH Reactor Physics MOOSE- based application. We find that MAMMOTH can reproduce the Monte Carlo solution for the HTR-10 reactor by using the PJFNK-SPH equivalence method. In all cases studied, the MAMMOTH results are within 120 pcm of the Monte Carlo reference calculation and the maximum errors in neutron absorption and generation rates are within 0.536% and 0.215% from the reference calculation, respectively. A set of benchmark exercises from the International Atomic Energy Agency (IAEA) HTR-10 benchmark in IAEA-TECDOC-1382 were conducted including parts B1 (critical), B2 (temperature coefficient) and B3 (control rod worth). The results we obtain for the critical core are in excellent agreement with the expected value and we confirm that with ENDF/B-VII.r1 data we match the critical eigen- value keff=1 to within 100 pcm. Our temperature coefficient calculation is consistent with the VSOP results generated by the German participants. The Chinese and South African VSOP models produce lower temperature coefficients, a fact that was noted in the original IAEA benchmark, but for which no explanation was provided. Both a full and single control rod worth calculations with Serpent and MAMMOTH agree well with the results from the IAEA benchmark. Finally, and most importantly, this work demonstrates that MAM- MOTH can recover the Monte Carlo high-fidelity solutions to an excellent accuracy using the PJFNK-SPH method. This sets the stage for the ultimate goal of using MAMMOTH in transient, multi-physics scenarios where correction of the cross sections is obtained for steady-state conditions and then used in the transient scenario for which no Monte-Carlo solution is available.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗