Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “computer system benchmarking”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 397 records · Page 22
All sourcesNASA NTRSDOE OSTI
DOE OSTI2025

FAIR Surrogate Benchmarks Supporting AI and Simulation Research (Final Report)

Computational Science is being revolutionized by integrating AI and simulation and, in particular, by deep learning surrogate models that can replace all or part of traditional large‐scale HPC computations. Such surrogates can achieve remarkable performance improvements, as much as several orders of magnitude, and save both compute time and energy. The Surrogate Benchmark Initiative (SBI) project creates a community repository and FAIR (Findable, Accessible, Interoperable, and Reusable) data ecosystem for HPC application surrogate benchmarks. The SBI team comes from Argonne National Laboratory (ANL), Indiana University (IU), Rutgers University, the University of Tennessee, Knoxville (UTK), and the University of Virginia (UVA). SBI repositories include data, code, and all relevant collateral artifacts that the science and engineering community need to use and reuse these data sets and surrogates. SBI repositories generate active research from both the participants in SBI and the broad community of AI and domain scientists. This project develops surrogates that use several different neural nets to learn and quickly infer the results of simulations and data systems and captures them as surrogate benchmarks with a rich set of metadata covering: Data; Model; Metrics specification; Machine specification; and Science, Speed, and Power Results. We research FAIR metadata for these benchmarks. We develop application surrogate examples as benchmarks across many fields (ANL, UTK, IU, UVA). We also study non-Surrogate benchmarks that have many common features and similar issues as regards FAIRness. We work with MLCommons (UVA, UTK), which is a major machine learning benchmarking activity where we get metadata ontologies, software, and benchmarks, Benchmarks have datasets, models, and metadata and they need a technical framework developed by UTK and Rutgers and deployed by UVA. We study features of Surrogates including performance, training set size, and uncertainty quantification (Rutgers, UVA and IU).

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2023

A parallel, distributed memory implementation of the adaptive sampling configuration interaction method

The many-body simulation of quantum systems is an active field of research that involves several different methods targeting various computing platforms. Many methods commonly employed, particularly coupled cluster methods, have been adapted to leverage the latest advances in modern high-performance computing. Selected configuration interaction (sCI) methods have seen extensive usage and development in recent years. However, the development of sCI methods targeting massively parallel resources has been explored only in a few research works. Here, we present a parallel, distributed memory implementation of the adaptive sampling configuration interaction approach (ASCI) for sCI. In particular, we will address the key concerns pertaining to the parallelization of the determinant search and selection, Hamiltonian formation, and the variational eigenvalue calculation for the ASCI method. Load balancing in the search step is achieved through the application of memory-efficient determinant constraints originally developed for the ASCI-PT2 method. The presented benchmarks demonstrate near optimal speedup for ASCI calculations of Cr 2 (24e, 30o) with 10 6 , 10 7 , and 3 × 10 8 variational determinants on up to 16 384 CPUs. Importantly, to the best of the authors’ knowledge, this is the largest variational ASCI calculation to date.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2024

ACOPF Transmission Switching Using Open-Source MINLP Solvers

The optimal transmission switching (OTS) problem with AC physics represents a mixed integer non-linear non-convex optimization problem which can provide benefits to transmission level power system operations. In this paper we benchmark a set of open-source mixed integer non-linear programming (MINLP) solvers on the OTS problem with AC physics using the pglib set of power system test cases. Results characterizing the performance of the different solvers are reported and discussed.

ACOPF↗
DOE OSTI2021

Investigation of Delayed Neutron Sensitivities for Several ICSBEP Benchmarks using MCNP

The effective delayed neutron (β$_{eff}$) is a very important parameter for reactor and criticality applications. This parameter is equal to the difference in reactivity between delayed critical ($k_{eff}$ = 1, which requires both prompt and delayed neutrons to achieve criticality) and prompt critical ($k_p$ = 1, which requires only prompt neutrons to achieve criticality). This is often referred to as the delayed critical "window" (the region of criticality between delayed and prompt critical). β$_{eff}$ is a reactor kinetics parameters and depends on the nuclides in the system that undergo fission as well as the spectral characteristics of the system. Measurements of β$_{eff}$ have been performed for many criticality experiments. The EUCLID (Experiments Underpinned by Computational Learning for Improvements in nuclear Data) project at Los Alamos National Laboratory (LANL) aims to constrain nuclear data by using a suite of measurement types beyond $k_{eff}$. Our team has recently investigated the use of pulsed spheres for nuclear data validation. Several other measurement methods are also of interest, including β$_{eff}$ (investigated here) and reactivity coefficients (investigated in a separate work at this same meeting). One focus of our work is to determine if other methods are complimentary to the critical experiments already utilized for nuclear data validation. This is important because if a method has similar sensitivities then it will not be particularly useful for nuclear data validation as it will provide the same information as the critical experiments already being used. Here "similar" could refer to several characteristics, one being the shape of a sensitivity profile over energy. In the future, these methods will be utilized (with both existing and new experiments) in machine learning algorithms for nuclear validation, similar to what is currently done for criticality experiments. In order to use a measurement type for nuclear validation, it is necessary to obtain cross-section sensitivities for that parameter. This work looks at one approach to estimate β$_{eff}$ sensitivities by utilizing $k_{eff}$ sensitivities within Monte Carlo N-Particle ® Code Version 6.2. This is applied to several criticality benchmarks. Results are compared and the benefits and limitations of this approach are discussed.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗
DOE OSTI2022

Retargetable Optimizing Compilers for Quantum Accelerators via a Multi-Level Intermediate Representation

In this work, we present a multi-level quantum-classical intermediate representation (IR) that enables an optimizing, retargetable compiler for available quantum languages. Our work builds upon the Multi-level Intermediate Representation (MLIR) framework and leverages its unique progressive lowering capabilities to map quantum languages to the LLVM machine-level IR. We provide both quantum and classical optimizations via the MLIR pattern rewriting sub-system and standard LLVM optimization passes, and demonstrate the programmability, compilation, and execution of our approach via standard benchmarks and test cases. In comparison to other standalone language and compiler efforts available today, our work results in compile times that are 1000x faster than standard Pythonic approaches, and 5-10x faster than comparative standalone quantum language compilers. Our compiler provides quantum resource optimizations via standard programming patterns that result in a 10x reduction in entangling operations, a common source of program noise. We see this work as a vehicle for rapid quantum compiler prototyping.

43 PARTICLE ACCELERATORS↗
DOE OSTI2025

MS25: Materials Science-Focused Benchmark Data Set for Machine Learning Interatomic Potentials

Here, we present MS25, a benchmark data set for evaluating machine learning interatomic potentials (MLIPs) across diverse materials-relevant systems including MgO surfaces, liquid water, zeolites, a catalytic Pt surface reaction, high-entropy alloys (HEAs), and disordered Zr-oxides. Five MLIP architectures (MACE, NequIP, Allegro, MTP, and Torch-ANI) are trained and tested, focusing not only on traditional metrics (energies, forces, and stresses) but also explicitly validating derived physical observables such as lattice constants, volumes, and reaction barriers. We find that most models reach comparable accuracy on standard error metrics across the simple systems, although equivariant MLIPs offer 1.5–2× improvements over nonequivariant MLIPs in energy and force error for structurally complex or compositionally disordered environments such as HEAs and Zr–O systems. Our analysis highlights that low errors in energy and force predictions do not guarantee reliable observables, emphasizing the necessity of explicit validation. We demonstrate limitations in cross-framework transferability, as models trained on one zeolite framework (CHA) fail to reliably generalize to predictions of structurally distinct frameworks (e.g., MFI). Size-extensive tests show some dependence on system size for MgO, resulting from forced periodicity. The HEA and Zr–O data sets are identified as challenging tests for future benchmarks and MLIP model architecture developments as they show significant differentiation in error between MLIP architectures and are still relatively difficult at 1000 training images. Moving forward, we recommend that benchmarking efforts shift their focus from marginal accuracy improvements in energy and force errors toward identifying and understanding model failure modes, rigorously assessing transferability, and evaluating how their errors affect observable predictions. For researchers looking to choose an MLIP architecture, we suggest selecting equivariant MLIP architectures if the complexity of the system is a challenge. For simple materials problems, auxiliary features such as integration with molecular dynamics engines, trade-offs between computational data set generation cost vs MLIP inference speed, and framework integration may play a more important decision factor than small differences in error metrics that are unlikely to matter for production-level research.

chemical structure↗
DOE OSTI2025

Desmearing small-angle scattering data by central moment expansions of instrument resolution

This study introduces an enhanced numerical technique tailored specifically for refining 1D small-angle scattering (SAS) intensity profiles affected by smearing. Our primary objective is to address the resolution blurring commonly encountered in SAS data, particularly in systems with clearly defined correlation peaks whose spread aligns with the width of the resolution function at corresponding Q positions. Unlike previous approaches that expanded the SAS intensity using central moment expansion, the new method focuses on expanding the resolution function itself, thus eliminating artificial oscillations observed in smeared spectra due to limitations inherent in our earlier algorithm. This method is straightforward to implement, computationally efficient and consistently performs well in numerical benchmarking. To illustrate its effectiveness, we present a case study of a lamellar phase characterized by distinct peaks in its small-angle neutron scattering intensities.

central moment expansions↗
DOE OSTI2022

Error rate reduction of single-qubit gates via noise-aware decomposition into native gates

In the current era of Noisy Intermediate-Scale Quantum (NISQ) technology, the practical use of quantum computers remains inhibited by our inability to aptly decouple qubits from their environment to mitigate computational errors. In this paper, we introduce an approach by which knowledge of a qubit’s initial quantum state and the standard parameters describing its decoherence can be leveraged to mitigate the noise present during the execution of a single-qubit gate. We benchmark our protocol using cloud-based access to IBM quantum processors. On ibmq_rome, we demonstrate a reduction of the single-qubit error rate by 38%, from $1.6 \times 10 ^{-3}$ to $1.0 \times 10 ^{-3}$ , provided the initial state of the input qubit is known. On ibmq_bogota, we prove that our protocol will never decrease gate fidelity, provided the system’s T 1 and T 2 times have not drifted above 100 times their assumed values. The protocol can be used to reduce quantum state preparation errors, as well as to improve the fidelity of quantum circuits for which some knowledge of the qubits’ intermediate states can be inferred. This paper presents a pathway to using information about noise levels and quantum state distributions to significantly reduce error rates associated with quantum gates via optimized decomposition into native hardware gates.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
DOE OSTI2025

I/O in Machine Learning Applications on HPC Systems: A 360-degree Survey

Growing interest in Artificial Intelligence (AI) has resulted in a surge in demand for faster methods of Machine Learning (ML) model training and inference. This demand for speed has prompted the use of high performance computing (HPC) systems that excel in managing distributed workloads. Because data is the main fuel for AI applications, the performance of the storage and I/O subsystem of HPC systems is critical. In the past, HPC applications accessed large portions of data written by simulations or experiments or ingested data for visualizations or analysis tasks. ML workloads perform small reads spread across a large number of random files. This shift of I/O access patterns poses several challenges to modern parallel storage systems. In this paper, we survey I/O in ML applications on HPC systems, and target literature within a 6-year time window from 2019 to 2024. We define the scope of the survey, provide an overview of the common phases of ML, review available profilers and benchmarks, examine the I/O patterns encountered during offline data preparation, training, and inference, and explore I/O optimizations utilized in modern ML frameworks and proposed in recent literature. Lastly, we seek to expose research gaps that could spawn further R&D.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2026

Physics vs structure: A systematic benchmark of learning strategies for multi-zone building thermal dynamics

Recent advances in physics-informed and data-driven machine learning promise improved thermal models for advanced building control, yet there is limited quantitative evidence on when added physics structure and architectural complexity are beneficial. Here, this work presents a systematic benchmark of five representative system identification methods for modeling multi-zone building thermal dynamics: linear state-space models, multi-layer perceptrons, neural state-space models, neural ordinary differential equations, and physically-consistent neural networks. The methods are evaluated across multiple data regimes and zone coupling strategies. Using a high-fidelity multi-zone commercial building emulator, we examine short-term and long-term prediction accuracy, computational efficiency, and ease of development. Our results reveal critical trade-offs between prediction performance, model complexity, and physical consistency. We demonstrate that decoupled, nonlinear black-box models consistently outperform coupled physics-constrained architectures in both predictive accuracy and out-of-distribution robustness in majority of the test cases for the building type considered in the study. Our findings quantify the cost of complexity in building thermal modeling and provide concrete, actionable, scenario-based guidelines for selecting model classes for control-oriented applications.

Building thermal modeling↗
DOE OSTI2024

The ATLAS experiment software on ARM

With an increased dataset obtained during the Run 3 of the LHC at CERN and the even larger expected increase of the dataset by more than one order of magnitude for the HL-LHC, the ATLAS experiment is reaching the limits of the current data processing model in terms of traditional CPU resources based on x86_64 architectures and an extensive program for software upgrades towards the HL-LHC has been set up. The ARM architecture is becoming a competitive and energy efficient alternative. Some surveys indicate its increased presence in HPCs and commercial clouds, and some WLCG sites have expressed their interest. Chip makers are also developing their next generation solutions on ARM architectures, sometimes combining ARM and GPU processors in the same chip. Consequently it is important that the ATLAS software embraces the change and is able to successfully exploit this architecture. We report on the successful porting to ARM of the Athena software framework, which is used by ATLAS for both online and offline computing operations. Furthermore we report on the successful validation of simulation workflows running on ARM resources. For this we have set up an ATLAS Grid site using ARM compatible middleware and containers on Amazon Web Services (AWS) ARM resources. The ARM version of Athena is fully integrated in the regular software build system and distributed in the same way as other software releases. In addition, the workflows have been integrated into the HEPscore benchmark suite which is the planned WLCG wide replacement of the HepSpec06 benchmark used for Grid site pledges. In the overall porting process we have used resources on AWS, Google Cloud Platform (GCP) and CERN. A performance comparison of different architectures and resources will be discussed.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗
DOE OSTI2023

Advancing the Theory of Nuclear Data Evaluations [Abstract]

We present recent advances in the R-matrix formalism as well as the Bayesian evaluation framework for improved nuclear data evaluations. The advances in the R matrix formalism include: 1) direct processes, 2) doorway, as well as multistep, processes, and 3) various forms of the Reich-Moore approximation for eliminated capture channels. Furthermore, to address unreasonably small posterior uncertainties often encountered in nuclear data evaluations of large data sets using the conventional form of the Bayes’ theorem, we introduce imperfections (of the data or the model) as a formal evaluation tool for taming the evaluated uncertainties in harmony with Bayes’ theorem. These theoretical advances were motivated by the nuclear data evaluations of differential resolved resonance cross section data using the code SAMMY, as well as the integral benchmark experiments using the SCALE code system, being performed at Oak Ridge National Laboratory for the Nuclear Criticality Safety Program. Some pedagogical applications of the new formalism, as well as a snapshot of the SAMMY modernization efforts, will be presented.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗
DOE OSTI2022

High-order multirate explicit time-stepping schemes for the baroclinic-barotropic split dynamics in primitive equations

In order to treat the multiple time scales of ocean dynamics in an efficient manner, the baroclinic-barotropic splitting technique has been widely used for solving the primitive equations for ocean modeling. Based on the framework of strong stability-preserving Runge-Kutta approach, we propose two high-order multirate explicit time-stepping schemes (SSPRK2-SE and SSPRK3-SE) for the resulting split system in this paper. The proposed schemes allow for a large time step to be used for the three-dimensional baroclinic (slow) mode and a small time step for the two-dimensional barotropic (fast) mode, in which each of the two mode solves just need to satisfy their respective CFL conditions for numerical stability. Specifically, at each time step, the baroclinic velocity is first computed by advancing the baroclinic mode and fluid thickness of the system with the large time-step and the assistance of some intermediate approximations of the barotropicmode obtained by substepping with the small time step; then the barotropic velocity is corrected by using the small time step to re-advance the barotropic mode un-der an improved barotropic forcing produced by interpolation of the forcing terms from the preceding baroclinic mode solves; lastly, the fluid thickness is updated by coupling the baroclinic and barotropic velocities. Additionally, numerical inconsistencies on the discretized sea surface height caused by the mode splitting are relieved via a reconciliation process with carefully calculated flux deficits. Here, two benchmark tests from the “MPAS-Ocean” platform are carried out to numerically demonstrate the performance and parallel scalability of the proposed SSPRK-SE schemes.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2022

HYSPLIT/MACCS Atmospheric Dispersion Model Technical Documentation and Benchmark Analysis

The nuclear accident consequence analysis code MACCS has traditionally modeled dispersion during downwind transport using a Gaussian plume segment model. MACCS is designed to estimate consequence measures such as air concentrations and ground depositions, radiological doses, and health and economic impacts on a statistical basis over the course of a year to produce annualaveraged output measures. The objective of this work is to supplement the Gaussian atmospheric transport and diffusion (ATD) model currently in MACCS with a new option using the HYSPLIT model. HYSPLIT/MACCS coupling has been implemented, with HYSPLIT as an alternative ATD option. The subsequent calculations in MACCS use the HYSPLIT-generated air concentration, and ground deposition values to calculate the same range of output quantities (dose, health effects, risks, etc.) that can be generated when using the MACCS Gaussian ATD model. Based on the results from the verification test cases, the implementation of the HYSPLIT/MACCS coupling is confirmed. This report contains technical details of the HYSPLIT/MACCS coupling and presents a benchmark analysis using the HYSPLIT/MACCS coupling system. The benchmark analysis, which involves running specific scenarios and sensitivity studies designed to examine how the results generated by the traditional MACCS Gaussian plume segment model compare to the new, higher fidelity HYSPLIT/MACCS modeling option, demonstrates the modeling results that can be obtained by using this new option. The comparisons provided herein can also help decision-makers evaluate the potential benefit of using results based on higher fidelity modeling with the additional computational burden needed to perform the calculations. Three sensitivity studies to investigate the potential impact of alternative modeling options, regarding 1) input meteorological data set, 2) method to estimate stability class, and 3) plume dispersion model for larger distances, on consequence results were also performed. The results of these analyses are provided and discussed in this report.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2024

ReactionMechanismSimulator.jl: A modern approach to chemical kinetic mechanism simulation and analysis

Abstract We present ReactionMechanismSimulator.jl (RMS), a modern differentiable software for the simulation and analysis of chemical kinetic mechanisms, including multiphase systems. RMS has already been applied to problems in combustion, pyrolysis, polymers, pharmaceuticals, catalysis, and electrocatalysis. RMS is written in Julia, making it easy to develop and allowing it to take advantage of Julia's extensive numerical computing ecosystem. In addition to its extensive library of optimized analytic Jacobians, RMS can generate and use Jacobians computed using automatic differentiation and symbolically generated analytic Jacobians. RMS is demonstrated to be faster than Cantera and Chemkin in several benchmarks. RMS also implements an extensive set of features for analyzing chemical mechanisms, including a library of easy‐to‐call plotting functions, molecular structure resolved flux diagram generation, crash analysis, traditional sensitivity analysis, transitory sensitivity analysis, and an automatic mechanism analysis toolkit. RMS implements efficient adjoint and parallel forward sensitivity analyses. We also demonstrate the ease of adding new features to RMS.

Johnson, Matthew S.↗
DOE OSTI2021

Validation of SAS4A/SASSYS-1 for predicting steady-state single-phase natural circulation

The validation of system analysis codes for nuclear reactor systems is required for the development and application of these computational tools. Designed as a comprehensive system analysis code for advanced nuclear reactors, SAS4A/SASSYS-1 requires validation of its physics model for capturing single-phase natural circulation behavior. To support the validation of SAS4A/SASSYS-1, high-precision experiments are performed capturing steady-state single-phase natural circulation on a scaled facility with comprehensive instrumentation. In this work, dedicated tests are performed quantifying the critical modeling parameters, and a single-phase natural circulation benchmark dataset is obtained with well-documented uncertainty and comprehensive facility description. The validation is then performed against the dataset examining the capability of SAS4A/SASSYS-1 in simulating steady-state single-phase natural circulation. The experimental facility is modeled in the candidate code. Solution verification is performed using Richardson-extrapolation-based estimators which quantify and restrict numerical errors from discretization. Input uncertainty provided by the benchmark dataset is forward propagated through the candidate code, quantifying the output uncertainty in a Monte Carlo approach. The composition of the output uncertainty is also quantified through a variance-based sensitivity analysis. With the uncertainty quantified for each individual condition, a detailed comparison between the simulation results and experimental data is performed covering the whole dataset. The results show consistent agreement for all primary parameters. The current validation activity provides a valuable benchmark dataset for the validation of system analysis codes in capturing single-phase natural circulation and demonstrates satisfactory prediction capability of SAS4A/SASSYS1 for steady-state single-phase natural circulation.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗
DOE OSTI2025

A quantum eigenvalue solver based on tensor networks

Electronic ground states are of central importance in chemical simulations, but have remained beyond the reach of efficient classical algorithms except in cases of weak electron correlation or one-dimensional spatial geometry. We introduce a hybrid quantum-classical eigenvalue solver that constructs a wavefunction ansatz from a linear combination of matrix product states in rotated orbital bases, enabling the characterization of strongly correlated ground states with arbitrary spatial geometry. The energy is converged via a gradient-free generalized sweep algorithm based on quantum subspace diagonalization, with a potentially exponential speedup in the off-diagonal matrix element contractions upon translation into compact quantum circuits of linear depth in the number of qubits. Chemical accuracy is attained in numerical experiments for both a stretched water molecule and an octahedral arrangement of hydrogen atoms, achieving substantially better correlation energies compared to a unitary coupled-cluster benchmark, with orders of magnitude reductions in quantum resource estimates and a surprisingly high tolerance to shot noise. This proof-of-concept study suggests a promising new avenue for scaling up simulations of strongly correlated chemical systems on near-term quantum hardware.

chemistry↗
DOE OSTI2024

I/O performance studies of analysis workloads on production and dedicated resources at CERN

The recent evolutions of the analysis frameworks and physics data formats of the LHC experiments provide the opportunity of using central analysis facilities with a strong focus on interactivity and short turnaround times, to complement the more common distributed analysis on the Grid. In order to plan for such facilities, it is essential to know in detail the performance of the combination of a given analysis framework, of a specific analysis and of the installed computing and storage resources. This contribution describes performance studies performed at CERN, using the EOS disk-based storage, either directly or through an XCache instance, from both batch resources and highperformance compute nodes which could be used to build an analysis facility. A variety of benchmarks, both synthetic and based on real-world physics analyses and their corresponding input datasets, are utilized. In particular, the RNTuple format from the ROOT project is put to the test and compared to the latest version of the TTree format, and the impact of caches is assessed. In addition, we assessed the difference in performance between the use of storage system specific protocols, like XRootd, and FUSE. The results of this study are intended to be a valuable input in the design of analysis facilities, at CERN and elsewhere.

Sciabà, Andrea↗