Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Benchmarks”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 739 records · Page 41

Benchmark High-Fidelity EMT Models for Power Grid with PV Plants

In recent times electromagnetic transient (EMT) modeling tools have been identified as one of the most important requirements in replicating, analyzing, and investigating the dynamics of the power grid with photovoltaic (PV) plants. However, there are no benchmark models for power grid with PVs to investigate emerging challenges with higher penetration of PVs (like trips and momentary cessations during faults from a region far away). To this end, in this paper, synthetic benchmark high-fidelity EMT dynamic models of power grid with large-scale PV plants are presented. The models are developed in PSCAD and PSCAD/Fortran. Simulation results for different use cases (events) and scenarios are presented.

Marthi, Phani Ratna Vanamali↗

Benchmark Accelerated Aging of Harvested Hypalon/Epr and Cspe/Xlpe Power and I&C Cable in Nuclear Power Plants

As part of the Light Water Reactor and Sustainability (LWRS) program in the U.S. Department of Energy (DOE) Office of Nuclear Energy, material aging and degradation research is currently geared to support the long-term operation of existing nuclear power plants (NPPs) as they move beyond their initial 40 year licenses. The goal of this research is to provide information so that NPPs can develop aging management programs (AMPs) to address replacement and monitoring needs as they look to operate for 20 years, and in some cases 40 years, beyond their initial operating lifetimes. For cable insulation and jacket materials that support instrument, control, and safety systems, accelerated aging data are needed to determine priorities in cable aging management programs. Before accelerated thermal and radiation aging of harvested, representative cable insulation and jacket materials, the benchmark performance of a new test capability at Oak Ridge National Laboratory (ORNL) was evaluated for temperatures between 70 and 135°C, dose rates between 100 and 500 Gy/h, and accumulated doses up to 20 kGy, Samples that were characterized and are representative of current materials in use were harvested from the Callaway NPP near Fulton, Missouri, and the San Onofre NPP north of San Diego, California. From the Callaway NPP, a multiconductor control rod cable manufactured by Boston Insulated Wire (BIW), with a Hypalon/ chorolosulfonated polyethylene (CSPE) jacket and ethylene-propylene rubber (EPR) insulation, was harvested from the auxiliary space during a planned outage in 2013. This cable was placed into service when the plant was started in 1984. From the San Onofre NPP, a Rockbestos Firewall III (FRIII) cable with a Hypalon/ CSPE jacket with cross-linked polyethylene (XLPE) insulation was harvested from an on-site, climate-controlled storage area. This conductor, which was never placed into service, was procured around 2007 in anticipation of future operation that did not occur. Benchmark aging for both jacket and insulation material was carried out in air at a temperature of 125°C or in uniform 140 Gy/h gamma irradiation over a period of 60 days. Their mechanical properties over the course of their exposures were compared with reference data from comparable cable jacket/insulation compositions and aging conditions. For both accelerated thermal and radiation aging, it was observed that the mechanical properties for the Callaway BIW control rod cable were consistent with those previously measured. However, for the San Onofre Rockbestos FRIII, there was an observable functional difference for accelerated thermal aging at 125°C. Details on possible sources for this difference and plans for resolving each source are given in this paper.

aging↗

Benchmarking of Monte Carlo simulation of bremsstrahlung from thick targets at radiotherapy energies

Several Monte Carlo systems were benchmarked against published measurements of bremsstrahlung yield from thick targets for 10-30 MV beams. The quantity measured was photon fluence at 1 m per unit energy per incident electron (spectra), and total photon fluence, integrated over energy, per incident electron (photon yield). Results were reported at 10-30 MV on the beam axis for Al and Pb targets and at 15 MV at angles out to 90 degree sign for Be, Al, and Pb targets. Beam energy was revised with improved accuracy of 0.5% using an improved energy calibration of the accelerator. Recently released versions of the Monte Carlo systems EGSNRC, GEANT4, and PENELOPE were benchmarked against the published measurements using the revised beam energies. Monte Carlo simulation was capable of calculation of photon yield in the experimental geometry to 5% out to 30 degree sign , 10% at wider angles, and photon spectra to 10% at intermediate photon energies, 15% at lower energies. Accuracy of measured photon yield from 0 to 30 degree sign was 5%, 1 s.d., increasing to 7% for the larger angles. EGSNRC and PENELOPE results were within 2 s.d. of the measured photon yield at all beam energies and angles, GEANT4 within 3 s.d. Photon yield at nonzero angles for angles covering conventional field sizes used in radiotherapy (out to 10 degree sign ), measured with an accuracy of 3%, was calculated within 1 s.d. of measurement for EGSNRC, 2 s.d. for PENELOPE and GEANT4. Calculated spectra closely matched measurement at photon energies over 5 MeV. Photon spectra near 5 MeV were underestimated by as much as 10% by all three codes. The photon spectra below 2-3 MeV for the Be and Al targets and small angles were overestimated by up to 15% when using EGSNRC and PENELOPE, 20% with GEANT4. EGSNRC results with the NIST option for the bremsstrahlung cross section were preferred over the alternative cross section available in EGSNRC and over EGS4. GEANT4 results calculated with the ''low energy'' physics list were more accurate than those calculated with the ''standard'' physics list.

62 RADIOLOGY AND NUCLEAR MEDICINE↗

TH-E-BRE-01: A 3D Solver of Linear Boltzmann Transport Equation Based On a New Angular Discretization Method with Positivity for Photon Dose Calculation Benchmarked with Geant4

Purpose: The Linear Boltzmann Transport Equation (LBTE) solved through statistical Monte Carlo (MC) method provides the accurate dose calculation in radiotherapy. This work is to investigate the alternative way for accurately solving LBTE using deterministic numerical method due to its possible advantage in computational speed from MC. Methods: Instead of using traditional spherical harmonics to approximate angular scattering kernel, our deterministic numerical method directly computes angular scattering weights, based on a new angular discretization method that utilizes linear finite element method on the local triangulation of unit angular sphere. As a Result, our angular discretization method has the unique advantage in positivity, i.e., to maintain all scattering weights nonnegative all the time, which is physically correct. Moreover, our method is local in angular space, and therefore handles the anisotropic scattering well, such as the forward-peaking scattering. To be compatible with image-guided radiotherapy, the spatial variables are discretized on the structured grid with the standard diamond scheme. After discretization, the improved sourceiteration method is utilized for solving the linear system without saving the linear system to memory. The accuracy of our 3D solver is validated using analytic solutions and benchmarked with Geant4, a popular MC solver. Results: The differences between Geant4 solutions and our solutions were less than 1.5% for various testing cases that mimic the practical cases. More details are available in the supporting document. Conclusion: We have developed a 3D LBTE solver based on a new angular discretization method that guarantees the positivity of scattering weights for physical correctness, and it has been benchmarked with Geant4 for photon dose calculation.

97 MATHEMATICS AND COMPUTING↗

Lifetimes of excited states in $^{16}$C as a benchmark for ab initio developments

Lifetimes of higher-lying states ($2_2^+$ and $4_1^+$) in 16 C have been measured, employing the Gammasphere and Microball detector arrays, as key observables to test and refine ab initio calculations based on interactions developed within chiral Effective Field Theory. The presented experimental constraints to these lifetimes of $\tau ({2_2^+}) = [244, 446]\,~\textrm{fs}$ and $\tau ({4_1^+}) = [1.8, 4]\,~\textrm{ps}$, combined with previous results on the lifetime of the $2_1^+$ state of 16 C, provide a rather complete set of key observables to benchmark the theoretical developments. We present No-Core Shell-Model calculations using state-of-the-art chiral 2- (NN) and 3-nucleon (3N) interactions at next-to-next-to-next-to-leading order for both the NN and the 3N contributions and a generalized natural-orbital basis (instead of the conventional harmonic-oscillator single-particle basis) which reproduce, for the first time, the experimental findings remarkably well. The level of agreement of the new calculations as compared to the CD-Bonn meson-exchange NN interaction is notable and presents a critical benchmark for theory.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗

Benchmarking black hole heat engines, I

We present the results of initiating a benchmarking scheme that allows for cross-comparison of the efficiencies of black holes used as working substances in heat engines. We use a circular cycle in the [Formula: see text] plane as the benchmark engine. We test it on Einstein–Maxwell, Gauss–Bonnet and Born–Infeld black holes. Also, we derive a new and surprising exact result for the efficiency of a special “ideal gas” system to which all the black holes asymptote.

Astronomy & Astrophysics↗

A Sparse Tensor Benchmark Suite for CPUs and GPUs

Tensor computations present significant performance chal- lenges that impact a wide spectrum of applications ranging from machine learning, healthcare analytics, social network analysis, data mining to quantum chemistry and signal processing. Efforts to improve the perfor- mance of tensor computations include exploring data layout, execution scheduling, and parallelism in common tensor kernels. This work presents a benchmark suite for arbitrary-order sparse tensor kernels using state- of-the-art tensor formats: coordinate (COO) and hierarchical coordinate (HiCOO) on CPUs and GPUs. It presents a set of reference tensor kernel implementations that are compatible with real-world tensors and power law tensors extended from synthetic graph generation techniques. We also propose Roofline performance models for these kernels to provide insights of computer platforms from sparse tensor view. This benchmark suite along with the synthetic tensor generator is publicly available.

Li, Jiajia↗

wa-hls4ml: A Benchmark and Surrogate Models for hls4ml Resource and Latency Estimation

As machine learning (ML) is increasingly implemented in hardware to address real-time challenges in scientific applications, the development of advanced toolchains has significantly reduced the time required to iterate on various designs. These advancements have solved major obstacles, but also exposed new challenges. For example, processes that were not previously considered bottlenecks, such as hardware synthesis, are becoming limiting factors in the rapid iteration of designs. To mitigate these emerging constraints, multiple efforts have been undertaken to develop an ML-based surrogate model that estimates resource usage of ML accelerator architectures. We introduce wa-hls4ml, a benchmark for ML accelerator resource and latency estimation, and its corresponding initial dataset of over 680,000 fully connected and convolutional neural networks, all synthesized using hls4ml and targeting Xilinx FPGAs. The benchmark evaluates the performance of resource and latency predictors against several common ML model architectures, primarily originating from scientific domains, as exemplar models, and the average performance across a subset of the dataset. Additionally, we introduce GNN- and transformer-based surrogate models that predict latency and resources for ML accelerators. We present the architecture and performance of the models and find that the models generally predict latency and resources for the 75% percentile within several percent of the synthesized resources on the synthetic test dataset.

Hawks, Benjamin [Fermilab] (ORCID:0000000157000288↗

A Programming Model Performance Study Using the NAS Parallel Benchmarks

Harnessing the power of multicore platforms is challenging due to the additional levels of parallelism present. In this paper we use the NAS Parallel Benchmarks to study three programming models, MPI, OpenMP and PGAS to understand their performance and memory usage characteristics on current multicore architectures. To understand these characteristics we use the Integrated Performance Monitoring tool and other ways to measure communication versus computation time, as well as the fraction of the run time spent in OpenMP. The benchmarks are run on two different Cray XT5 systems and an Infiniband cluster. Our results show that in general the three programming models exhibit very similar performance characteristics. In a few cases, OpenMP is significantly faster because it explicitly avoids communication. For these particular cases, we were able to re-write the UPC versions and achieve equal performance to OpenMP. Using OpenMP was also the most advantageous in terms of memory usage. Also we compare performance differences between the two Cray systems, which have quad-core and hex-core processors. We show that at scale the performance is almost always slower on the hex-core system because of increased contention for network resources.

Shan, Hongzhang↗

Application of Benchmarked Kinetic Resistive Wall Mode Stability Codes to ITER, Including Additional Physics

Leading resistive wall mode (RWM) stability codes MARS-K [Y. Liu, et al., Phys. Plasmas 15, 112503 (2008)] and MISK [B. Hu, et al., Phys. Plasmas 12, 057301 (2005)] have been previously benchmarked. The benchmarking has now been extended to include additional physics, and used to project the stability of ITER in a realistic operating space. Due to ITER's relatively low plasma rotation and collisionality, collisions and non-resonance rotational effects were both found to have little impact on stability, and these non-resonance rotational effects also will not self-consistently affect the ITER RWM eigenfunction. Resonances between thermal ions and electrons and the expected level of ITER toroidal rotation were found to be important to stability, as were alpha particles, which are not in rotational resonance. MISK calculations show that without alpha particles, ITER is projected to be unstable to the RWM, but the expected level of alphas is calculated to provide a sufficient level of stability.

Resistive Wall Mode Stability↗

Algebraic Multigrid Benchmark

AMG is a parallel algebraic multigrid solver for linear systems arising from problems on unstructured grids. It has been derived directly from the BoomerAMG solver in the hypre library, a large linear solvers library that is being developed in the Center for Applied Scientific Computing (CASC) at LLNL and is very similar to the AMG2013 benchmark with additional optimizations. The driver provided in the benchmark can build various test problems. The default problem is a Laplace type problem with a 27-point stencil, which can be scaled up and is designed to solve a very large problem. A second problem simulates a time dependent problem, in which successively various smnllcr systems are solved.

Yang, Ulrike↗

Sequoia Messaging Rate Benchmark

The purpose of this benchmark is to measure the maximal message rate of a single compute node. The first num_cores ranks are expected to reside on the 'core' compute node for which message rate is being tested. After that, the next num_nbors ranks are neighbors for the first core rank, the next set of num_nbors ranks are neighbors for the second core rank, and so on. For example, testing an 8-core node (num_cores = 8) with 4 neighbors (num_nbors = 4) requires 8 + 8 * 4 - 40 ranks. The first 8 of those 40 ranks are expected to be on the 'core' node being benchmarked, while the rest of the ranks are on separate nodes.

Friedley, Andrew↗

Benchmarking the performance of uncertainty quantification methods for neural network-based interatomic potentials

Machine-learned interatomic potentials (ML-IAPs) continue to gain popularity as accurate, computationally efficient replacements for traditional, physics-based interatomic potentials and expensive ab initio methods. Uncertainty quantification (UQ) of ML-IAPs is a growing area of research as UQ is critical in many applications of IAPs, such as developing curated datasets, active learning-based data augmentation, self-improving models, and estimating the uncertainty of molecular dynamics simulations. In this paper, we construct and benchmark a series of different neural network potentials (NNPs) with varying network architectures to determine the performance of these models with respect to both the mean and uncertainty calibration error. Each NNP method is specifically designed to predict either epistemic or aleatoric uncertainty with particular focus on the differences in behavior between the epistemic and aleatoric uncertainty estimates. We benchmark these methods using multiple datasets common in the ML-IAP literature. The results show that the aleatoric uncertainty from single-shot model architectures is a competitive alternative to ensemble-based epistemic uncertainty predictions in regions of sufficient data-density. However, in regions where the representative data is sparse, aleatoric uncertainty models tend to overpredict and epistemic methods tend to underpredict the actual model error. We conclude that the type of UQ is crucial when discussing performance of probabilistic model results as different methods have different performance characteristics depending on the regime in which they are evaluated. Therefore, the type of UQ method should be carefully evaluated against both the data characteristics and requirements for the intended application.

97 MATHEMATICS AND COMPUTING↗

Reference solutions for linear radiation transport: the Hohlraum and Lattice Benchmarks

Radiation transport describes the propagation of energetic particles through space as they interact with a surrounding material medium. In a kinetic description, radiation transport is modeled by a radiation transport equation (RTE) that prescribes the density of the radiation in position-momentum phase space. The purpose of this dataset is to provide highly resolved solutions to two benchmark problems. These two benchmarks do not possess exact solutions; moreover, the construction of a manufactured solution may require a non-physical source that is not desirable, especially if it spoils the physical nature of the solution. Thus the goal of this computational study is to provide a highly resolved reference solution for testing newer, more cost efficient methods that are currently being developed in the research community.

97 MATHEMATICS AND COMPUTING↗

Li1−xNiO2 Many-body DMC Benchmark Dataset

The dataset contains all numerical data generated in support of the manuscript “Many‑body Benchmark of Electronic Charge and Spin Densities for Li1–xNiO2​” (Journal of Chemical Theory and Computation, DOI: 10.1021/acs.jctc.5c02097, URL: https://pubs.acs.org/doi/10.1021/acs.jctc.5c02097). The materials included in this repository are: 1. Data files used to produce all figures and tables in the main manuscript and supporting information. 2. Benchmark density‑functional theory (DFT) datasets used for the charge‑ and spin‑density analyses. 3. Reference many‑body diffusion Monte Carlo (DMC) calculations and associated input/output files.

36 MATERIALS SCIENCE↗

Comparison of Spatial Discretization Methods for Solving the S N Equations Using a Three-Dimensional Method of Manufactured Solutions Benchmark Suite with Escalating Order of Nonsmoothness

For this study, a comparison of the accuracy and computational efficiency of spatial discretization methods of the three-dimensional S N equations is conducted, including discontinuous Galerkin finite element methods, the arbitrarily high-order transport method of nodal type (AHOTN), the linear-linear method, the linear-nodal (LN) method, and the higher-order diamond difference method. For this purpose, we have developed a suite of method of manufactured solutions benchmarks that provides an exact solution of the S N equations even in the presence of scattering. Most importantly, our benchmark suite permits the user to set an arbitrary level of smoothness of the exact solution across the singular characteristics. Our study focuses on the computational efficiency of the considered spatial discretization methods. Numerical results indicate that the best-performing method depends on the norm used to measure the discretization error. We employ discrete Lp norms and integral error norms in this work. For configurations with continuous exact angular flux, high-order AHOTNs perform best under L p error norms, while the LN method performs best when measured by integral error norms. Lastly, when the angular flux is discontinuous, a new singular-characteristic tracking method for three-dimensional geometries performs best among the considered methods.

97 MATHEMATICS AND COMPUTING↗

Comparison of the PHISICS/RELAP5-3D ring and block model results for phase I of the OECD/NEA MHTGR-350 benchmark

The PHISICS code system has been under development at INL since 2010. It consists of several modules providing improved coupled core simulation capability: INSTANT (3D nodal transport core calculations), MRTAU (depletion and decay heat generation) and modules performing criticality searches, fuel shuffling and generalized perturbation. Coupling of the PHISICS code suite to the thermal hydraulics system code RELAP5-3D was finalized in 2013, and as part of the verification and validation effort the first phase of the OECD/NEA MHTGR-350 Benchmark has now been completed. The theoretical basis and latest development status of the coupled PHISICS/RELAP5-3D tool are described in more detail in a concurrent paper. This paper provides an overview of the OECD/NEA MHTGR-350 Benchmark and presents the results of Exercises 2 and 3 defined for Phase I. Exercise 2 required the modelling of a stand-alone thermal fluids solution at End of Equilibrium Cycle for the Modular High Temperature Reactor (MHTGR). The RELAP5-3D results of four sub-cases are discussed, consisting of various combinations of coolant bypass flows and material thermophysical properties. Exercise 3 required a coupled neutronics and thermal fluids solution, and the PHISICS/RELAP5-3D code suite was used to calculate the results of two sub-cases. The main focus of the paper is a comparison of results obtained with the traditional RELAP5-3D “ring” model approach against a much more detailed model that include kinetics feedback on individual block level and thermal feedbacks on a triangular sub-mesh. The higher fidelity that can be obtained by this “block” model is illustrated with comparison results on the temperature, power density and flux distributions. Furthermore, it is shown that the ring model leads to significantly lower fuel temperatures (up to 10%) when compared with the higher fidelity block model, and that the additional model development and run-time efforts are worth the gains obtained in the improved spatial temperature and flux distributions.

11 NUCLEAR FUEL CYCLE AND FUEL MATERIALS↗

Benchmarking of ENDF/B-VIII.0 cross sections for LWR ex-core transport applications

ENDF/B-VIII.0 cross section data is benchmarked on a simple Monte-Carlo model representative of the ex-core geometry of a PWR used for reactor vessel neutron fluence calculations. The benchmarking indicates that the ENDF/B-VIII.0 results show large differences as compared to previous evaluations, which have been extensively compared to measured data. This exercise suggests that ENDF/B-VIII.0 nuclear data is not suitable for ex-core LWR radiation transport applications. Evaluators are aware of the problems with the ENDF/B-VIII.0 evaluation of iron and work is underway to improve the evaluations for an upcoming release.

21 SPECIFIC NUCLEAR REACTORS AND ASSOCIATED PLANTS↗