Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “computer system benchmarking”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 91 records · Page 5

Benchmarking numerical methods for impact and cratering applications

Large scale computational models are important for studying impact cratering events that are prevalent both on Earth and, more broadly, in this solar system. To address these problems, models must reliably account for both large length scales (e.g., kilometers) and relatively long time scales (hundreds of seconds). This work benchmarks two such approaches, a more traditional hydrodynamics approach and a finite-discrete element method (FDEM), for impact cratering applications. Both 2D and 3D results are discussed for two different impact velocities, 5 km/s and 20 km/s, striking normal to the target and, for 3D simulations, 45° from vertical. In addition, comparisons to previously published data are presented. Finally, differences in how these methods model damage are discussed. Ultimately, both approaches show successful modeling of several different impact scenarios.

58 GEOSCIENCES↗

SBbadger: biochemical reaction networks with definable degree distributions

Abstract Motivation An essential step in developing computational tools for the inference, optimization and simulation of biochemical reaction networks is gauging tool performance against earlier efforts using an appropriate set of benchmarks. General strategies for the assembly of benchmark models include collection from the literature, creation via subnetwork extraction and de novo generation. However, with respect to biochemical reaction networks, these approaches and their associated tools are either poorly suited to generate models that reflect the wide range of properties found in natural biochemical networks or to do so in numbers that enable rigorous statistical analysis. Results In this work, we present SBbadger, a python-based software tool for the generation of synthetic biochemical reaction or metabolic networks with user-defined degree distributions, multiple available kinetic formalisms and a host of other definable properties. SBbadger thus enables the creation of benchmark model sets that reflect properties of biological systems and generate the kinetics and model structures typically targeted by computational analysis and inference software. Here, we detail the computational and algorithmic workflow of SBbadger, demonstrate its performance under various settings, provide sample outputs and compare it to currently available biochemical reaction network generation software. Availability and implementation SBbadger is implemented in Python and is freely available at https://github.com/sys-bio/SBbadger and via PyPI at https://pypi.org/project/SBbadger/. Documentation can be found at https://SBbadger.readthedocs.io. Supplementary information Supplementary data are available at Bioinformatics online.

59 BASIC BIOLOGICAL SCIENCES↗

Fermionic mean-field dynamics for spin systems beyond free fermions

We introduce the fermionized time-dependent Hartree–Fock (fTDHF), a real-time quantum dynamics method for spin-1/2 Hamiltonians following their mapping to fermions via the Jordan-Wigner transformation. fTDHF is formally equivalent to exact dynamics in the case of free fermions, and can efficiently handle non-local string operators arising from long-range interactions via transition matrix elements between non-orthogonal Slater determinants. We show that the fTDHF method can be implemented on a classical computer with a cost that scales polynomially with system size, and linearly with the time steps. We benchmark fTDHF against exact dynamics on three separate spin-1/2 models, representing adiabatic preparation of states with long-range correlations, disorder-driven observation of many-body localization, and particle production in the Schwinger model. For each of these systems, fTDHF is shown to reproduce the qualitative dynamics generated by the exact evolutions, while maintaining a simple physical picture due to its mean-field nature.

Dutta, Rishab↗

Best Practices Guide: Benchmarking Energy Efficiency in Laboratories

Purpose and Audience A wide spectrum of laboratory owners, ranging from universities to federal agencies, have explicit goals for energy efficiency and greenhouse gas reductions in their facilities. For example, new federal buildings and major renovations of existing buildings are to reduce fossil fuel-generated energy consumption by 90% in 2025, and 100% in 2030, compared with a 2003 baseline (FEMP n.d.). Minnesota SB2030 standard requires achieving an 80% reduction from the average building baseline for commercial, institutional, and industrial buildings (SB2030 n.d). A laboratory—new or existing—is much more likely to meet energy efficiency goals if quantitative metrics and targets are explicitly specified and tracked over the life cycle of the building, from design through construction, commissioning, operations, and renovations. If efficiency targets are not explicitly and properly defined, any additional capital costs or design time associated with attaining higher efficiencies can be difficult to justify relative to other priorities. The purpose of this guide is to provide guidance on how to specify and compute energy efficiency metrics and benchmarks for laboratories, at the whole-building as well as the system level. The information in this guide can be used to incorporate quantitative metrics and targets into new construction or retrofit of existing facilities. For information on strategies and technologies to achieve energy efficiency, the reader is referred to I2SL resources, including technology best practice guides and case studies.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗

Impact of Thermal Scattering Law on Similarity Assessment in Light-Water or Polyethylene-Moderated Systems

A collaborative effort between Pacific Northwest National Laboratory (PNNL) and Oak Ridge National Laboratory (ORNL) is underway to provide a technical basis and methodology for the criticality safety community to use the sum of fractions (SoF) method for generating limits for mixtures of “selected actinide nuclides” included in the ANSI/ANS 8.15 standard. The PNNL scope in this project is to define a range of mixtures of 233 U, 235 U, and 239 Pu moderated with either light water or polyethylene and to examine the critical masses for these mixtures. The ORNL scope is primarily to provide validation support for these studies. More complete discussion of the project and its validation aspects will be presented at the upcoming International Conference on Nuclear Criticality Safety (ICNC) this Fall in Sendai, Japan. Clear differences in benchmark similarity to application systems as assessed by the integral parameter ck are noted in the validation studies performed as part of this project as a function of moderator. The c k value is a correlation coefficient that represents that amount of shared uncertainty in k eff due to cross sections between two systems. Individual nuclide-reaction contributions between the two systems can be simply summed to arrive at the total c k value. Specifically, the c k values for light-water–moderated solution experiments are higher for a water-moderated application than for a polyethylene-moderated application. This result is neither totally unexpected nor surprising, but the magnitude of the difference was difficult to anticipate. The TSUNAMI sequence, in the SCALE 6.2.4 code package developed by ORNL, was used to generate eigenvalues and reactivity effects with perturbation-theory based approach through sensitivity coefficients for all nuclides in the system with all reactions and energy groups. The TSUNAMI-Indices and Parameters (IP) sequence then uses the sensitivity data generated through TSUNAMI to generate relational parameters (i.e., c k ) to determine the degree of similarity between systems. One detail of the SCALE material and data implementation must be discussed at this point. Several thermal scattering laws (TSLs) are available for 1 H. SCALE uses a different nuclide ID number for each TSL; essentially, each version of 1 H is treated as a unique nuclide. For example, 1 H bound in water ( 1 H-H2O) is assigned the nuclide ID 1001, whereas 1 H bound in polyethylene (h-poly) is assigned the nuclide ID 9001001. The same cross section data are used for all reactions in 1 H, regardless of TSL, except for scattering below the TSL cutoff energy. TSUNAMI-IP treats different nuclide IDs as different nuclides; thus, no uncertainty is shared between 1 H-H2O and h-poly, despite much of the same data, including covariance data, being used for both nuclides. This presents a question: how much of the difference in assessed similarity between water- and polyethylene-moderated systems is due to the moderators, and how much is caused by the treatment of 1 H-H2O and h poly with cross section and covariance data. The extended edits generated by TSUANMI-IP allow for an investigation of this issue specifically, as well as a demonstration of the general techniques available within TSUNAMI to understand the results of the similarity assessment. This paper presents and analyzes the similarity assessment of both water- and polyethylene-moderated systems for a single benchmark: PST-002-001.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗

LC-Opt: Benchmarking Reinforcement Learning and Agentic AI for End-to-End Liquid Cooling Optimization in Data Centers

Liquid cooling is critical for thermal management in high-density data centers with the rising AI workloads. However, machine learning-based controllers are essential to unlock greater energy efficiency and reliability, promoting sustainability. We present LC-Opt, a Sustainable Liquid Cooling (LC) benchmark environment, for reinforcement learning (RL) control strategies in energy-efficient liquid cooling of high-performance computing (HPC) systems. Built on the baseline of a high-fidelity digital twin of Oak Ridge National Lab's Frontier Supercomputer cooling system, LC-Opt provides detailed Modelica-based end-to-end models spanning site-level cooling towers to data center cabinets and server blade groups. RL agents optimize critical thermal controls like liquid supply temperature, flow rate, and granular valve actuation at the IT cabinet level, as well as cooling tower (CT) setpoints through a Gymnasium interface, with dynamic changes in workloads. This environment creates a multi-objective real-time optimization challenge balancing local thermal regulation and global energy efficiency, and also supports additional components like a heat recovery unit (HRU). We benchmark centralized and decentralized multi-agent RL approaches, demonstrate policy distillation into decision and regression trees for interpretable control, and explore LLM-based methods that explain control actions in natural language through an agentic mesh architecture designed to foster user trust and simplify system management. LC-Opt democratizes access to detailed, customizable liquid cooling models, enabling the ML community, operators, and vendors to develop sustainable data center liquid cooling control solutions.

Naug, Avisek [Hewlett Packard Enterprise]↗

OptiBench: An Optimization Benchmark Tool for Renewable Energy Problems

We propose a benchmark framework and visualization tool, OptiBench, for analyzing the performance of state-of-the-art optimization solvers across a variety of optimization problems in renewable energy research. Our framework is designed from the ground up in the Julia programming language and enables analysis at scale on high performance computing (HPC) systems. Our visualization tool allows effortless evaluation of optimization solver performance, robustness, and accuracy through intuitive plots, e.g., performance profiles, heat maps, and distribution plots. We have tested three benchmark suites relevant to the modeling of renewable energy systems, viz., CUTEst, PGLib-OPF, and WaterTAP water treatment optimization problems. We illustrate benchmarking of CUTEst using OptiBench on the National Laboratory of the Rockies's (NLR) HPC Kestrel. Our findings indicate that MA57 HSL linear solver demonstrated the best overall performance for an experimental IPOPT implementation. Our work is ongoing and we intend to add support for more optimization solvers and benchmark test suites in the future.

97 MATHEMATICS AND COMPUTING↗

CI/CD Efforts for Validation, Verification and Benchmarking OpenMP Implementations

Software developers must adapt to keep up with the changing capabilities of platforms so that they can utilize the power of High-Performance Computers (HPC), including exascale systems. OpenMP, a directive-based parallel programming model, allows developers to include directives to existing C, C++, or Fortran code to allow node level parallelism without compromising performance. This paper describes our CI/CD efforts to provide easy evaluation of the support of OpenMP across different compilers using existing testsuites and benchmark suites on HPC platforms. Our main contributions include (1) the set of a Continuous Integration (CI) and Continuous Development (CD) workflow that captures bugs and provides faster feedback to compiler developers, (2) an evaluation of OpenMP (offloading) implementations supported by AMD, HPE, GNU, LLVM, and Intel, and (3) evaluation of the quality of compilers across different heterogeneous HPC platforms. With the comprehensive testing through the CI/CD workflow, we aim to provide a comprehensive understanding of the current state of OpenMP (offloading) support in different compilers and heterogeneous platforms consisting of CPUs and GPUs from NVIDIA, AMD, and Intel.

Jarmusch, Aaron↗

OptiBench: An Optimization Benchmark Tool for Renewable Energy Problems

We propose a benchmark framework and visualization tool, OptiBench, for analyzing the performance of state-of-the-art optimization solvers across a variety of optimization problems in renewable energy research. Our framework is designed from the ground up in the Julia programming language and enables analysis at scale on high performance computing (HPC) systems. Our visualization tool allows effortless evaluation of optimization solver performance, robustness, and accuracy through intuitive plots, e.g., performance profiles, heat maps, and distribution plots. We have tested three benchmark suites relevant to the modeling of renewable energy systems, viz., CUTEst, PGLib-OPF, and WaterTAP water treatment optimization problems. We illustrate benchmarking of CUTEst using OptiBench on the National Renewable Energy Laboratory's (NREL) HPC Kestrel. Our findings indicate that MA57 HSL linear solver demonstrated the best overall performance for an experimental IPOPT implementation. Our work is ongoing and we intend to add support for more optimization solvers and benchmark test suites in the future.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗

Adaptive variational quantum minimally entangled typical thermal states for finite temperature simulations

Scalable quantum algorithms for the simulation of quantum many-body systems in thermal equilibrium are important for predicting properties of quantum matter at finite temperatures. Here we describe and benchmark a quantum computing version of the minimally entangled typical thermal states (METTS) algorithm for which we adopt an adaptive variational approach to perform the required quantum imaginary time evolution. The algorithm, which we name AVQMETTS, dynamically generates compact and problem-specific quantum circuits, which are suitable for noisy intermediate-scale quantum (NISQ) hardware. We benchmark AVQMETTS on statevector simulators and perform thermal energy calculations of integrable and nonintegrable quantum spin models in one and two dimensions and demonstrate an approximately linear system-size scaling of the circuit complexity. We further map out the finite-temperature phase transition line of the two-dimensional transverse field Ising model. Finally, we study the impact of noise on AVQMETTS calculations using a phenomenological noise model.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

A step towards the final frontier: Lessons learned from acceptance testing of the first HPE/Cray EX 3000 system at ORNL

Summary In this article, we summarize the deployment of the Air Force Weather (AFW) HPC11 system at Oak Ridge National Laboratory (ORNL) including the process followed to successfully complete acceptance testing of the system. HPC11 is the first HPE/Cray EX 3000 system that has been successfully released to its user community in a federal facility. HPC11 consists of two identical 800‐node supercomputers, Fawbush and Miller, with access to two independent and identical lustre parallel file systems. HPC11 is equipped with Slingshot 10 interconnect technology and relies on the HPE Performance Cluster Manager software for system configuration. ORNL has a clearly defined acceptance testing process used to ensure that every new system deployed can provide the necessary capabilities to support user workloads. We worked closely with HPE and AFW to develop a set of tests that used the United Kingdom's Meteorological Office's Unified Model and 4‐dimensional variational data assimilation. We also included benchmarks and applications from the Oak Ridge Leadership Computing Facility portfolio to fully exercise the HPE/Cray programming environment and evaluate the functionality and performance of the system. Acceptance testing of HPC11 required parallel execution of each element on Fawbush and Miller. In addition, careful coordination was needed to ensure successful acceptance of the newly deployed lustre file systems alongside the compute resources. In this work, we present test results from specific system components and provide an overview of the issues identified, challenges encountered, and the lessons learned along the way.

Melesse Vergara, Verónica G.↗

A Benchmark Suite for Evaluating Scientific AI Workloads on GPUs

AI applications have been steadily increasing in the allocation portfolio among leadership computing facilities. These applications depend on deep learning frameworks with hardware acceleration and underlying software systems. With the rapid development of applications, software stacks, and hardware devices, it is essential to evaluate the performance of core operations in AI workloads for direction of optimizations and procurement of next-generation high-performance computing (HPC) infrastructures. Currently, most benchmarks lack scientific AI workloads. So, we present DeepKernelBench and the experimental results of evaluating the benchmark suite for early observations and performance comparisons on datacenter GPUs using representative workloads for scientific AI, including Attentions, General matrix multiplications, Geometrics and Fourier neural operations.

Jin, Zheming [Advanced Micro Devices (AMD)]↗

A Step Towards the Final Frontier: Lessons Learned from Acceptance Testing of the First HPE/Cray EX 3000 System at ORNL

In this paper, we summarize the deployment of the Air Force Weather (AFW) HPC11 system at Oak Ridge National Laboratory (ORNL) including the process followed to successfully complete acceptance testing of the system. HPC11 is the first HPE/Cray EX 3000 system that has been successfully released to its user community in a federal facility. HPC11 consists of two identical 800-node supercomputers, Fawbush and Miller, with access to two independent and identical Lustre parallel file systems. HPC11 is equipped with Slingshot 10 interconnect technology and relies on the HPE Performance Cluster Manager (HPCM) software for system configuration. ORNL has a clearly defined acceptance testing process used to ensure that every new system deployed can provide the necessary capabilities to support user workloads. We worked closely with HPE and AFW to develop a set of tests that used the United Kingdom’s Meteorological Office’s Unified Model (UM) and 4DVAR. We also included benchmarks and applications from the Oak Ridge Leadership Computing Facility (OLCF) portfolio to fully exercise the HPE/Cray programming environment and evaluate the functionality and performance of the system. Acceptance testing of HPC11 required parallel execution of each element on Fawbush and Miller. In addition, careful coordination was needed to ensure successful acceptance of the newly deployed Lustre file systems alongside the compute resources. In this work, we present test results from specific system components and provide an overview of the issues identified, challenges encountered, and the lessons learned along the way.

Melesse Vergara, Veronica↗

Status of the CERBERUS Evaluation for the International Criticality Safety Benchmark Evaluation Project (ICSBEP) Handbook

Modeling & Simulation (M&S) tools are used to analyze advanced reactor designs and the safety of current nuclear operations. As computers continue to improve, we are able to enhance resolution in our calculations. Therefore, the limitations of simulation capability are in the quality of data that is being used, including our ability to quantify the uncertainty and sensitivity of that data. In order to model systems of interest with increasing accuracy, the industry must improve key nuclear data measurements. The International Criticality Safety Benchmark Evaluation Project (ICSBEP) compiles and evaluates experiment data in a handbook that can be used by criticality safety engineers and others to validate computer codes and cross section libraries at nuclear facilities. Both critical and subcritical experiments are included in the handbook. These experiments, along with differential measurements, can help improve the quality of nuclear data. Concerns regarding the accuracy of Cu nuclear data have been published. The large values and trend of C-E for the Zeus intermediate energy benchmark, being one of the primary examples. Furthermore, very few experiments have been designed to be sensitive to Cu (as shown in Figure 1), so an integral, critical experiment is needed to help resolve these differences.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗

Benchmark Specification for FFTF LOFWOS Test #13

The Fast Flux Test Facility (FFTF) at the Hanford site in Washington was designed by the Westinghouse Electric Corporation for the U.S. Department of Energy. FFTF was a 400 MW thermal, oxide-fueled, liquid sodium cooled test reactor, built to assist development and testing of advanced fuels and materials for fast breeder reactors. After reaching criticality in 1980, FFTF operated until 1992, providing the U.S. Department of Energy (DOE) with the means to test fuels, materials, and other components in a fast neutron flux environment. In July 1986, a series of unprotected transients (with the plant protection system intentionally disabled) were performed in FFTF as part of the passive safety demonstration program. Among these were thirteen loss of flow without scram (LOFWOS) tests. The goals of this program included confirming the liquid metal reactor safety margins, providing data for computer code validation, and demonstrating the inherent and passive safety benefits of specific design features. The test defined in this benchmark is LOFWOS Test #13, which was initiated at 50% power and 100% flow with the pump pony motors turned off. This benchmark specification is intended to support collaborative efforts within international partnerships on the validation of simulation tools and models in the area of Sodium-cooled Fast Reactor (SFR) safety. Validated tools and models are needed to evaluate SFR inherent safety characteristics and assess the impact of passive design features in response to accident initiators. Comparisons with experimental data and the results of safety analyses from other groups create unique opportunities to improve predictive capabilities of computational codes and methods for SFR modeling and simulation. The conditions of the LOFWOS test along with the feedback from FFTF’s limited free bow core restraint system and the novel passive reactivity control Gas-Expansion Modules (GEMs) pose a very challenging and uniquely valuable benchmark exercise.

21 SPECIFIC NUCLEAR REACTORS AND ASSOCIATED PLANTS↗

Practical Introduction to Benchmarking and Characterization of Quantum Computers

Rapid progress in quantum technology has transformed quantum computing and quantum information science from theoretical possibilities into tangible engineering challenges. Breakthroughs in quantum algorithms, quantum simulations, and quantum error correction are bringing useful quantum computation closer to fruition. These remarkable achievements have been facilitated by advances in quantum characterization, verification, and validation (QCVV). QCVV methods and protocols enable scientists and engineers to scrutinize, understand, and enhance the performance of quantum information-processing devices. In this tutorial, we review the fundamental principles underpinning QCVV, and introduce a diverse array of QCVV tools used by quantum researchers. We define and explain QCVV’s core models and concepts—quantum states, measurements, and processes—and illustrate how these building blocks are leveraged to examine a target system or operation. We survey and introduce protocols ranging from simple qubit characterization to advanced benchmarking methods. Along the way, we provide illustrated examples and detailed descriptions of the protocols, highlight the advantages and disadvantages of each, and discuss their potential scalability to future large-scale quantum computers. This tutorial serves as a guidebook for researchers unfamiliar with the benchmarking and characterization of quantum computers, and also as a detailed reference for experienced practitioners.

open quantum systems & decoherence↗

IRIS-MASH: Efficient Multi-device Asynchronous Multi-Stream Heterogeneous Computing

In the rapidly evolving field of high-performance computing (HPC), effectively leveraging heterogeneous devices through asynchronous task programming is paramount. This paper presents a robust asynchronous task programming model tailored for a multi-device, multi-stream execution environment that incorporates a diverse array of heterogeneous computing units, including GPUs from various vendors and other accelerators. Current state-of-the-art task programming models provide methodologies to support asynchronous task executions, but they typically handle homogeneous devices using native programming languages, while support for heterogeneous devices is limited to frameworks like OpenCL. This gap presents significant challenges in abstracting heterogeneous devices to harness their true asynchronous capabilities effectively using their native programming languages. By implementing asynchronous task execution, our model significantly boosts the performance of tiled algorithm task graphs through overlapping data transfers with computation and enabling the simultaneous execution of multiple kernels. We integrate this approach into a heterogeneous Intelligent Runtime System (IRIS) and assess its performance using a suite of tiled algorithm benchmarks from the heterogeneous math kernels library (MatRIS) based on IRIS. Experimental results demonstrate a performance improvement ranging from 1.6 × to 2 × over IRIS without asynchronous support, and a notable 22% performance enhancement compared to established runtime systems such as StarPU and PaRSEC. This approach significantly improves computation efficiency of HPC workflows and provides a solid base for future exploration and development in the area of asynchronous task programming in heterogeneous systems.

Miniskar, Narasinga Rao [ORNL] (ORCID:000000018259↗

Unbiased particle conformation extraction from scattering spectra using orthonormal basis expansions

A strategy is outlined for quantitatively evaluating the particle density profiles from small-angle scattering spectra of dilute solutions. The approach employs an orthonormal basis function expansion method, enabling the determination of characteristic mass distributions in self-assembled structures without the need for a specific structural model. Through computational benchmarking, the efficacy of this approach is validated by effectively reconstructing the density profile of soft-ball systems with varying fuzziness from their scattering signatures. Further, the feasibility of the method is demonstrated by fitting small-angle neutron scattering data obtained from Pluronic L64 micelles at different temperatures. This proposed approach is both simple and analytical, eliminating the requirement for a presumptive structural model in scattering analysis. The new method could therefore facilitate quantitative descriptions of complex nanoscopic structures inherent to numerous soft-matter systems using small-angle scattering techniques.

36 MATERIALS SCIENCE↗