Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “BENCHMARKS”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 415 records · Page 23
All sourcesNASA NTRSDOE OSTI
DOE OSTI2025

Real singlet scalar benchmarks in the multi-TeV resonance regime

Scalar extensions of the Standard Model (SM) are of much interest at the Large Hadron Collider (LHC) and future colliders. In particular, these models can give rise to resonant di-Higgs production and alter the Higgs trilinear coupling. In this paper, we study di-Higgs production in the Standard Model extended by a real scalar singlet with no additional symmetries. We determine how large the resonant di-Higgs rate and variation in the Higgs trilinear coupling can be in four scenarios: current LHC results and projected results at the high luminosity LHC (HL-LHC), the HL-LHC combined with a circular 𝑒 − ⁢𝑒 + collider such as the Circular Electron Positron Collider or Future Circular Collider with electron-positron collisions, and the HL-LHC combined with a linear 𝑒 − ⁢𝑒 + collider such as the International Linear Collider. While these are updated results from a previous study by [I. M. Lewis and M. Sullivan, Benchmarks for double Higgs production in the singlet extended standard model at the LHC, Phys. Rev. D 96, 035037 (2017).] using current LHC data, we go further and find benchmark points in the multi-TeV resonance regime for future colliders beyond the HL-LHC. Considering current LHC results, the resonant di-Higgs rate can still be an order of magnitude larger than the SM predicted di-Higgs rate. In the HL-LHC scenario, the Higgs trilinear coupling can still be a factor of three larger than the SM prediction for resonance masses in the 1.5–3.5 TeV range, where resonant searches may have less reach. This enhancement is just at the projected 2⁢𝜎 sensitivity of the HL-LHC. We find there are resonance masses for which the change in the Higgs trilinear is maximized while the resonant rate is negligible. We provide an analytical understanding of these effects with a discussion on the interplay of various constraints on the parameter space and the Higgs trilinear coupling.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗
DOE OSTI2024

Benchmarking characterization methods for noisy quantum circuits

Effective methods for characterizing the noise in quantum computing devices are essential for programming and debugging circuit performance. Existing approaches vary in the information obtained as well as the amount of quantum and classical resources required, with more information generally requiring more resources. Here we benchmark the characterization methods of gate set tomography, Pauli channel noise reconstruction, and empirical direct characterization for developing models that describe noisy quantum circuit performance on a 27-qubit superconducting transmon device. We evaluate these models by comparing the accuracy of noisy circuit simulations with the corresponding experimental observations. Here, we find that the agreement of noise model to experiment does not correlate with the information gained by characterization and that the underlying circuit strongly influences the best choice of characterization approach. Empirical direct characterization scales best of the methods we tested and produced the most accurate characterizations across our benchmarks.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
DOE OSTI2021

Ab initio benchmarks of neutrinoless double- β decay in light nuclei with a chiral Hamiltonian

Here, we report ab initio benchmark calculations of nuclear matrix elements (NMEs) for neutrinoless double-β (0νββ) decays in light nuclei with mass numbers ranging from A = 6 to A = 22. We use the transition operator derived from light-Majorana neutrino exchange and evaluate the NME with three different methods: two variants of in-medium similarity renormalization group (IMSRG) and importance-truncated no-core shell model (IT-NCSM). The same two plus three nucleon interaction from chiral effective field theory is employed, and both isospin-conserving (ΔT = 0) and isospin-changing (ΔT = 2) transitions are studied. We compare our resulting ground-state energies and NMEs to those of recent ab initio no-core shell model and coupled-cluster calculations, also with the same inputs. We show that the NMEs of ΔT = 0 transitions are in good agreement among all calculations, at the level of 10%. For ΔT = 2, relative deviations are more significant in some nuclei. The comparison with the exact IT-NCSM result allows us to analyze these cases in detail, and indicates the next steps toward improving the IMSRG-based approaches. The present study clearly demonstrates the power of consistent cross checks that are made possible by ab initio methodology. This capability is crucial for providing meaningful many-body uncertainties in the NMEs for the 0νββ decays in heavier candidate nuclei, where quasiexact benchmarks are not available.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗
DOE OSTI2021

Simulating and benchmarking neutron total scattering instrumentation from inception of events to reduced and fitted data

In the design and realization of modern neutron scattering instrumentation, particularly when designing beamline concepts from the ground up, it is desirable to fully benchmark against realistically simulated data. This is especially true for total scattering beamlines, where the future deliverable data is to be analysed in both reciprocal- and real-space representations, and needs must be carefully balanced to ensure sufficient range, resolution and flux of the instrument. An approach to optimize the design of neutron scattering instrumentation via a workflow including ray-tracing simulations, event-based data reduction, heuristic analysis and fitting against realistically simulated spectra is demonstrated here. The case of the DISCOVER beamline concept at the Spallation Neutron Source is used as an example. The results of the calculations are benchmarked through simulation of existing instrumentation and subsequent direct comparison with measured data. Finally, on the basis of the validated models, the ability to explore design characteristics for future beamline concepts or future instrument improvements is demonstrated through the examples of detector tube size and detector layout.

47 OTHER INSTRUMENTATION↗
DOE OSTI2026

A Benchmark Suite for Evaluating Scientific AI Workloads on GPUs

AI applications have been steadily increasing in the allocation portfolio among leadership computing facilities. These applications depend on deep learning frameworks with hardware acceleration and underlying software systems. With the rapid development of applications, software stacks, and hardware devices, it is essential to evaluate the performance of core operations in AI workloads for direction of optimizations and procurement of next-generation high-performance computing (HPC) infrastructures. Currently, most benchmarks lack scientific AI workloads. So, we present DeepKernelBench and the experimental results of evaluating the benchmark suite for early observations and performance comparisons on datacenter GPUs using representative workloads for scientific AI, including Attentions, General matrix multiplications, Geometrics and Fourier neural operations.

Jin, Zheming [Advanced Micro Devices (AMD)]↗
DOE OSTI2024

Benchmark High-Fidelity EMT Models for Power Grid with PV Plants

In recent times electromagnetic transient (EMT) modeling tools have been identified as one of the most important requirements in replicating, analyzing, and investigating the dynamics of the power grid with photovoltaic (PV) plants. However, there are no benchmark models for power grid with PVs to investigate emerging challenges with higher penetration of PVs (like trips and momentary cessations during faults from a region far away). To this end, in this paper, synthetic benchmark high-fidelity EMT dynamic models of power grid with large-scale PV plants are presented. The models are developed in PSCAD and PSCAD/Fortran. Simulation results for different use cases (events) and scenarios are presented.

Marthi, Phani Ratna Vanamali↗
DOE OSTI2024

Lifetimes of excited states in $^{16}$C as a benchmark for ab initio developments

Lifetimes of higher-lying states ($2_2^+$ and $4_1^+$) in 16 C have been measured, employing the Gammasphere and Microball detector arrays, as key observables to test and refine ab initio calculations based on interactions developed within chiral Effective Field Theory. The presented experimental constraints to these lifetimes of $\tau ({2_2^+}) = [244, 446]\,~\textrm{fs}$ and $\tau ({4_1^+}) = [1.8, 4]\,~\textrm{ps}$, combined with previous results on the lifetime of the $2_1^+$ state of 16 C, provide a rather complete set of key observables to benchmark the theoretical developments. We present No-Core Shell-Model calculations using state-of-the-art chiral 2- (NN) and 3-nucleon (3N) interactions at next-to-next-to-next-to-leading order for both the NN and the 3N contributions and a generalized natural-orbital basis (instead of the conventional harmonic-oscillator single-particle basis) which reproduce, for the first time, the experimental findings remarkably well. The level of agreement of the new calculations as compared to the CD-Bonn meson-exchange NN interaction is notable and presents a critical benchmark for theory.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗
DOE OSTI2020

A Sparse Tensor Benchmark Suite for CPUs and GPUs

Tensor computations present significant performance chal- lenges that impact a wide spectrum of applications ranging from machine learning, healthcare analytics, social network analysis, data mining to quantum chemistry and signal processing. Efforts to improve the perfor- mance of tensor computations include exploring data layout, execution scheduling, and parallelism in common tensor kernels. This work presents a benchmark suite for arbitrary-order sparse tensor kernels using state- of-the-art tensor formats: coordinate (COO) and hierarchical coordinate (HiCOO) on CPUs and GPUs. It presents a set of reference tensor kernel implementations that are compatible with real-world tensors and power law tensors extended from synthetic graph generation techniques. We also propose Roofline performance models for these kernels to provide insights of computer platforms from sparse tensor view. This benchmark suite along with the synthetic tensor generator is publicly available.

Li, Jiajia↗
DOE OSTI2026

wa-hls4ml: A Benchmark and Surrogate Models for hls4ml Resource and Latency Estimation

As machine learning (ML) is increasingly implemented in hardware to address real-time challenges in scientific applications, the development of advanced toolchains has significantly reduced the time required to iterate on various designs. These advancements have solved major obstacles, but also exposed new challenges. For example, processes that were not previously considered bottlenecks, such as hardware synthesis, are becoming limiting factors in the rapid iteration of designs. To mitigate these emerging constraints, multiple efforts have been undertaken to develop an ML-based surrogate model that estimates resource usage of ML accelerator architectures. We introduce wa-hls4ml, a benchmark for ML accelerator resource and latency estimation, and its corresponding initial dataset of over 680,000 fully connected and convolutional neural networks, all synthesized using hls4ml and targeting Xilinx FPGAs. The benchmark evaluates the performance of resource and latency predictors against several common ML model architectures, primarily originating from scientific domains, as exemplar models, and the average performance across a subset of the dataset. Additionally, we introduce GNN- and transformer-based surrogate models that predict latency and resources for ML accelerators. We present the architecture and performance of the models and find that the models generally predict latency and resources for the 75% percentile within several percent of the synthesized resources on the synthetic test dataset.

Hawks, Benjamin [Fermilab] (ORCID:0000000157000288↗
DOE OSTI2026

Benchmarking the performance of uncertainty quantification methods for neural network-based interatomic potentials

Machine-learned interatomic potentials (ML-IAPs) continue to gain popularity as accurate, computationally efficient replacements for traditional, physics-based interatomic potentials and expensive ab initio methods. Uncertainty quantification (UQ) of ML-IAPs is a growing area of research as UQ is critical in many applications of IAPs, such as developing curated datasets, active learning-based data augmentation, self-improving models, and estimating the uncertainty of molecular dynamics simulations. In this paper, we construct and benchmark a series of different neural network potentials (NNPs) with varying network architectures to determine the performance of these models with respect to both the mean and uncertainty calibration error. Each NNP method is specifically designed to predict either epistemic or aleatoric uncertainty with particular focus on the differences in behavior between the epistemic and aleatoric uncertainty estimates. We benchmark these methods using multiple datasets common in the ML-IAP literature. The results show that the aleatoric uncertainty from single-shot model architectures is a competitive alternative to ensemble-based epistemic uncertainty predictions in regions of sufficient data-density. However, in regions where the representative data is sparse, aleatoric uncertainty models tend to overpredict and epistemic methods tend to underpredict the actual model error. We conclude that the type of UQ is crucial when discussing performance of probabilistic model results as different methods have different performance characteristics depending on the regime in which they are evaluated. Therefore, the type of UQ method should be carefully evaluated against both the data characteristics and requirements for the intended application.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2025

Reference solutions for linear radiation transport: the Hohlraum and Lattice Benchmarks

Radiation transport describes the propagation of energetic particles through space as they interact with a surrounding material medium. In a kinetic description, radiation transport is modeled by a radiation transport equation (RTE) that prescribes the density of the radiation in position-momentum phase space. The purpose of this dataset is to provide highly resolved solutions to two benchmark problems. These two benchmarks do not possess exact solutions; moreover, the construction of a manufactured solution may require a non-physical source that is not desirable, especially if it spoils the physical nature of the solution. Thus the goal of this computational study is to provide a highly resolved reference solution for testing newer, more cost efficient methods that are currently being developed in the research community.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2026

Li1−xNiO2 Many-body DMC Benchmark Dataset

The dataset contains all numerical data generated in support of the manuscript “Many‑body Benchmark of Electronic Charge and Spin Densities for Li1–xNiO2​” (Journal of Chemical Theory and Computation, DOI: 10.1021/acs.jctc.5c02097, URL: https://pubs.acs.org/doi/10.1021/acs.jctc.5c02097). The materials included in this repository are: 1. Data files used to produce all figures and tables in the main manuscript and supporting information. 2. Benchmark density‑functional theory (DFT) datasets used for the charge‑ and spin‑density analyses. 3. Reference many‑body diffusion Monte Carlo (DMC) calculations and associated input/output files.

36 MATERIALS SCIENCE↗
DOE OSTI2022

Benchmarking of ENDF/B-VIII.0 cross sections for LWR ex-core transport applications

ENDF/B-VIII.0 cross section data is benchmarked on a simple Monte-Carlo model representative of the ex-core geometry of a PWR used for reactor vessel neutron fluence calculations. The benchmarking indicates that the ENDF/B-VIII.0 results show large differences as compared to previous evaluations, which have been extensively compared to measured data. This exercise suggests that ENDF/B-VIII.0 nuclear data is not suitable for ex-core LWR radiation transport applications. Evaluators are aware of the problems with the ENDF/B-VIII.0 evaluation of iron and work is underway to improve the evaluations for an upcoming release.

21 SPECIFIC NUCLEAR REACTORS AND ASSOCIATED PLANTS↗
DOE OSTI2020

Data Testing of Polyethylene Thermal Scattering Law with New Thermal Epithermal eXperiments (TEX) Plutonium Baseline Benchmark, PU-MET-MIXED-002

Five plutonium baseline experiments completed in September of 2018 from the Thermal Epithermal eXperiments (TEX) program have been accepted by the International Criticality Safety Benchmark Evaluation Project as PU-MET-MIXED-002 and will be included in the 2020 version of the handbook. The experiments were made up of stacked layers of plutonium Zero Power Physics Reactor (ZPPR) plates moderated by polyethylene (PE, chemical formula C2H4) to varying degrees to create configurations with five different neutron spectra, including one fast configuration, one thermal configuration, and three mixed configurations. Calculations of the benchmark show overprediction of k eff when using the most recent release of the Evaluated Nuclear Data File Version B (ENDF/B) library release (VIII.0) cross sections, particularly when compared to the previous ENDF/B-VII.1 results. One potential source of overprediction was hypothesized to be the new Molecular Dynamics (MD) generated PE Thermal Scattering Law (TSL). The overprediction was shown via calculation to not be caused by the new ENDF/B-VIII.0 thermal scattering law, and in fact the overprediction was mitigated by the new PE TSL.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗
DOE OSTI2023

Charged point defect benchmark of Hybrid and GGA-PBE

Data used by the publication "High-throughput calculations of charged point defect properties with semi-local density functional theory - performance benchmarks for materials screening applications." This work presented an in-depth benchmark analysis of automated, semi-local point defect calculations with a-posteriori corrections, compared to 245 “gold standard” hybrid calculations previously published. We considered three different a-posteriori correction sets for semi-local calculations, implemented in a fully automated workflow, and consider the qualitative and quantitative differences for four different categories of defect information: thermodynamic transition levels, formation energies, fermi levels, and dopability limits. We highlighted the type of qualitative information about point defect properties that can be extracted from high-throughput calculations based on semi-local DFT methods, while also demonstrating the limits of quantitative accuracy that can be achieved by these approaches.

Broberg, Danny↗
DOE OSTI2026

Wire-arc Additive Manufacturing Benchmark

This is the dataset associated with the 2022 SRP Additive Manufacturing Prediction Challenge, originally hosted on Github at https://github.com/SRP-AM/SRP_AM_Prediction_Challenge. The benchmark was designed for validating prediction for the temperature history, residual stress, and distortion of an additively manufactured metal part with relatively simple geometry. A calibration problem with the same as-built geometry is provided with measured quantities of interest; including temperature histories at selective locations, post-build residual stress at selective locations, and overall distortion measurements. The challenge problem is presented with a different build sequence (i.e. thermal history). In this dataset, we include the actual recorded calibration and challenge measurements, as well as benchmark template files for testing predictions without incorporating the challenge data. Supplementary files around the materials and setup are available for transparency and reproducibility.

Bachus, Nicholas [UC Davis, Davis, CA]↗
DOE OSTI2022

Benchmarking Demand Flexibility in Commercial Buildings and Flattening the Duck – Addressing Baseline and Commissioning Challenges

With the transition from our traditional electric grid to a cleaner grid with renewable power generation, there is a need to enable building loads to be flexible. Load shedding and shifting will be essential for flattening the “Duck” for decarbonization. This paper explored the trend in the timing of DR events as a reflection of the grid’s needs using recent four years of event data from 203 retail stores in 11 states. The events are becoming significantly shorter with 2-hour duration being the most popular; shifting to late afternoon and early evening is another trend beyond California. Benchmarking will be essential for accounting DF as a reliable grid resource. This paper addresses a challenging aspect of benchmarking – inaccuracies in counterfactual baseline methods can introduce significant DF metrics variations in addition to weather and building characteristics related factors. The conventional “10/10” with adjustment baseline method has inherent limitation by design for load shifting applications. Therefore, it is imperative to identify alternative methods. This study compared three hourly regression baseline methods with “10/10” methods using two groups of commercial buildings that participated in DR programs: (1) 121 big-box retail stores, and (2) 11 office buildings in CA. The 14-day hourly outdoor temperature regression method was found to produce least error in the tested datasets and is promising for load shifting. The paper also pointed out that commissioning issues can also be a significant barrier for achieving consistent DF performance, which building managers and utilities should be aware of.

Liu, Jingjing↗
DOE OSTI2022

The Dark Machines Anomaly Score Challenge: Benchmark Data and Model Independent Event Classification for the Large Hadron Collider

We describe the outcome of a data challenge conducted as part of the Dark Machines (https://www.darkmachines.org) initiative and the Les Houches 2019 workshop on Physics at TeV colliders. The challenged aims to detect signals of new physics at the Large Hadron Collider (LHC) using unsupervised machine learning algorithms. First, we propose how an anomaly score could be implemented to define model-independent signal regions in LHC searches. We define and describe a large benchmark dataset, consisting of >1 billion simulated LHC events corresponding to 10\, fb^{-1} 10 f b − 1 of proton-proton collisions at a center-of-mass energy of 13 TeV. We then review a wide range of anomaly detection and density estimation algorithms, developed in the context of the data challenge, and we measure their performance in a set of realistic analysis environments. We draw a number of useful conclusions that will aid the development of unsupervised new physics searches during the third run of the LHC, and provide our benchmark dataset for future studies at https://www.phenoMLdata.org. Code to reproduce the analysis is provided at https://github.com/bostdiek/DarkMachines-UnsupervisedChallenge.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗