SEARCH · Engineering Papers

Results for “computer system benchmarking”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 145 records · Page 8

All sources NASA NTRS DOE OSTI

DOE OSTI2026

A Benchmark Suite for Evaluating Scientific AI Workloads on GPUs

AI applications have been steadily increasing in the allocation portfolio among leadership computing facilities. These applications depend on deep learning frameworks with hardware acceleration and underlying software systems. With the rapid development of applications, software stacks, and hardware devices, it is essential to evaluate the performance of core operations in AI workloads for direction of optimizations and procurement of next-generation high-performance computing (HPC) infrastructures. Currently, most benchmarks lack scientific AI workloads. So, we present DeepKernelBench and the experimental results of evaluating the benchmark suite for early observations and performance comparisons on datacenter GPUs using representative workloads for scientific AI, including Attentions, General matrix multiplications, Geometrics and Fourier neural operations.

Jin, Zheming [Advanced Micro Devices (AMD)]↗

DOE OSTI2021

A Step Towards the Final Frontier: Lessons Learned from Acceptance Testing of the First HPE/Cray EX 3000 System at ORNL

In this paper, we summarize the deployment of the Air Force Weather (AFW) HPC11 system at Oak Ridge National Laboratory (ORNL) including the process followed to successfully complete acceptance testing of the system. HPC11 is the first HPE/Cray EX 3000 system that has been successfully released to its user community in a federal facility. HPC11 consists of two identical 800-node supercomputers, Fawbush and Miller, with access to two independent and identical Lustre parallel file systems. HPC11 is equipped with Slingshot 10 interconnect technology and relies on the HPE Performance Cluster Manager (HPCM) software for system configuration. ORNL has a clearly defined acceptance testing process used to ensure that every new system deployed can provide the necessary capabilities to support user workloads. We worked closely with HPE and AFW to develop a set of tests that used the United Kingdom’s Meteorological Office’s Unified Model (UM) and 4DVAR. We also included benchmarks and applications from the Oak Ridge Leadership Computing Facility (OLCF) portfolio to fully exercise the HPE/Cray programming environment and evaluate the functionality and performance of the system. Acceptance testing of HPC11 required parallel execution of each element on Fawbush and Miller. In addition, careful coordination was needed to ensure successful acceptance of the newly deployed Lustre file systems alongside the compute resources. In this work, we present test results from specific system components and provide an overview of the issues identified, challenges encountered, and the lessons learned along the way.

Melesse Vergara, Veronica↗

NASA NTRS2011

Blade Displacement Measurements of the Full-Scale UH-60A Airloads Rotor

Blade displacement measurements were acquired during a wind tunnel test of the full-scale UH-60A Airloads rotor. The test was conducted in the 40- by 80-Foot Wind Tunnel of the National Full-Scale Aerodynamics Complex at NASA Ames Research Center. Multi-camera photogrammetry was used to measure the blade displacements of the four-bladed rotor. These measurements encompass a range of test conditions that include advance ratios from 0.15 to unique slowed-rotor simulations as high as 1.0, thrust coefficient to rotor solidity ratios from 0.01 to 0.13, and rotor shaft angles from -10.0 to 8.0 degrees. The objective of these measurements is to provide a benchmark blade displacement database to be utilized in the development and validation of rotorcraft computational tools. The methodology, system development, measurement techniques, and preliminary sample blade displacement measurements are presented.

Barrows, Danny A.↗

NASA NTRS2004

Parallel Performance Characterization of Columbia

Using a collection of benchmark problems of increasing levels of realism and computational effort, we will characterize the strengths and limitations of the 10,240 processor Columbia system to deliver supercomputing value to application scientists. Scientists need to be able to determine if and how they can utilize Columbia to carry extreme workloads, either in terms of ultra-large applications that cannot be run otherwise (capability), or in terms of very large ensembles of medium-scale applications to populate response matrices (capacity). We select existing application benchmarks that scale from a small number of processors to the entire machine, and that highlight different issues in running supercomputing-calss applicaions, such as the various types of memory access, file I/O, inter- and intra-node communications and parallelization paradigms. http://www.nas.nasa.gov/Software/NPB/

Biswas, Rupak↗

DOE OSTI2024

Status of the CERBERUS Evaluation for the International Criticality Safety Benchmark Evaluation Project (ICSBEP) Handbook

Modeling & Simulation (M&S) tools are used to analyze advanced reactor designs and the safety of current nuclear operations. As computers continue to improve, we are able to enhance resolution in our calculations. Therefore, the limitations of simulation capability are in the quality of data that is being used, including our ability to quantify the uncertainty and sensitivity of that data. In order to model systems of interest with increasing accuracy, the industry must improve key nuclear data measurements. The International Criticality Safety Benchmark Evaluation Project (ICSBEP) compiles and evaluates experiment data in a handbook that can be used by criticality safety engineers and others to validate computer codes and cross section libraries at nuclear facilities. Both critical and subcritical experiments are included in the handbook. These experiments, along with differential measurements, can help improve the quality of nuclear data. Concerns regarding the accuracy of Cu nuclear data have been published. The large values and trend of C-E for the Zeus intermediate energy benchmark, being one of the primary examples. Furthermore, very few experiments have been designed to be sensitive to Cu (as shown in Figure 1), so an integral, critical experiment is needed to help resolve these differences.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗

DOE OSTI2022

Benchmark Specification for FFTF LOFWOS Test #13

The Fast Flux Test Facility (FFTF) at the Hanford site in Washington was designed by the Westinghouse Electric Corporation for the U.S. Department of Energy. FFTF was a 400 MW thermal, oxide-fueled, liquid sodium cooled test reactor, built to assist development and testing of advanced fuels and materials for fast breeder reactors. After reaching criticality in 1980, FFTF operated until 1992, providing the U.S. Department of Energy (DOE) with the means to test fuels, materials, and other components in a fast neutron flux environment. In July 1986, a series of unprotected transients (with the plant protection system intentionally disabled) were performed in FFTF as part of the passive safety demonstration program. Among these were thirteen loss of flow without scram (LOFWOS) tests. The goals of this program included confirming the liquid metal reactor safety margins, providing data for computer code validation, and demonstrating the inherent and passive safety benefits of specific design features. The test defined in this benchmark is LOFWOS Test #13, which was initiated at 50% power and 100% flow with the pump pony motors turned off. This benchmark specification is intended to support collaborative efforts within international partnerships on the validation of simulation tools and models in the area of Sodium-cooled Fast Reactor (SFR) safety. Validated tools and models are needed to evaluate SFR inherent safety characteristics and assess the impact of passive design features in response to accident initiators. Comparisons with experimental data and the results of safety analyses from other groups create unique opportunities to improve predictive capabilities of computational codes and methods for SFR modeling and simulation. The conditions of the LOFWOS test along with the feedback from FFTF’s limited free bow core restraint system and the novel passive reactivity control Gas-Expansion Modules (GEMs) pose a very challenging and uniquely valuable benchmark exercise.

21 SPECIFIC NUCLEAR REACTORS AND ASSOCIATED PLANTS↗

NASA NTRSDate not supplied

Optimization Algorithms as Quantum Performance Benchmarks

Combinatorial optimization is anticipated to be one of the primary use cases for quantum computation in the coming years. The Quantum Approximate Optimization Algorithm (QAOA) and Quantum Annealing (QA) have the potential to demonstrate significant run-time performance benefits over current state-of-the-art solutions. Using existing methods for characterizing classical optimization algorithms, we analyze solution quality obtained by solving Max-Cut problems using a quantum annealing device and gate-model quantum simulators and devices. This is used to guide the development of an advanced benchmarking framework for quantum computers designed to evaluate the trade-off between run-time execution performance and the solution quality for iterative hybrid quantum-classical applications. The framework generates performance profiles through effective visualizations that show performance progression as a function of time for various problem sizes and illustrates algorithm limitations uncovered by the benchmarking approach. The framework is an enhancement to the existing open-source QED-C Application-Oriented Benchmark suite and can connect to the open-source analysis libraries. The suite can be executed on various quantum simulators and quantum hardware systems.

benchmarking↗

DOE OSTI2025

Practical Introduction to Benchmarking and Characterization of Quantum Computers

Rapid progress in quantum technology has transformed quantum computing and quantum information science from theoretical possibilities into tangible engineering challenges. Breakthroughs in quantum algorithms, quantum simulations, and quantum error correction are bringing useful quantum computation closer to fruition. These remarkable achievements have been facilitated by advances in quantum characterization, verification, and validation (QCVV). QCVV methods and protocols enable scientists and engineers to scrutinize, understand, and enhance the performance of quantum information-processing devices. In this tutorial, we review the fundamental principles underpinning QCVV, and introduce a diverse array of QCVV tools used by quantum researchers. We define and explain QCVV’s core models and concepts—quantum states, measurements, and processes—and illustrate how these building blocks are leveraged to examine a target system or operation. We survey and introduce protocols ranging from simple qubit characterization to advanced benchmarking methods. Along the way, we provide illustrated examples and detailed descriptions of the protocols, highlight the advantages and disadvantages of each, and discuss their potential scalability to future large-scale quantum computers. This tutorial serves as a guidebook for researchers unfamiliar with the benchmarking and characterization of quantum computers, and also as a detailed reference for experienced practitioners.

open quantum systems & decoherence↗

DOE OSTI2025

IRIS-MASH: Efficient Multi-device Asynchronous Multi-Stream Heterogeneous Computing

In the rapidly evolving field of high-performance computing (HPC), effectively leveraging heterogeneous devices through asynchronous task programming is paramount. This paper presents a robust asynchronous task programming model tailored for a multi-device, multi-stream execution environment that incorporates a diverse array of heterogeneous computing units, including GPUs from various vendors and other accelerators. Current state-of-the-art task programming models provide methodologies to support asynchronous task executions, but they typically handle homogeneous devices using native programming languages, while support for heterogeneous devices is limited to frameworks like OpenCL. This gap presents significant challenges in abstracting heterogeneous devices to harness their true asynchronous capabilities effectively using their native programming languages. By implementing asynchronous task execution, our model significantly boosts the performance of tiled algorithm task graphs through overlapping data transfers with computation and enabling the simultaneous execution of multiple kernels. We integrate this approach into a heterogeneous Intelligent Runtime System (IRIS) and assess its performance using a suite of tiled algorithm benchmarks from the heterogeneous math kernels library (MatRIS) based on IRIS. Experimental results demonstrate a performance improvement ranging from 1.6 × to 2 × over IRIS without asynchronous support, and a notable 22% performance enhancement compared to established runtime systems such as StarPU and PaRSEC. This approach significantly improves computation efficiency of HPC workflows and provides a solid base for future exploration and development in the area of asynchronous task programming in heterogeneous systems.

Miniskar, Narasinga Rao [ORNL] (ORCID:000000018259↗

DOE OSTI2024

Unbiased particle conformation extraction from scattering spectra using orthonormal basis expansions

A strategy is outlined for quantitatively evaluating the particle density profiles from small-angle scattering spectra of dilute solutions. The approach employs an orthonormal basis function expansion method, enabling the determination of characteristic mass distributions in self-assembled structures without the need for a specific structural model. Through computational benchmarking, the efficacy of this approach is validated by effectively reconstructing the density profile of soft-ball systems with varying fuzziness from their scattering signatures. Further, the feasibility of the method is demonstrated by fitting small-angle neutron scattering data obtained from Pluronic L64 micelles at different temperatures. This proposed approach is both simple and analytical, eliminating the requirement for a presumptive structural model in scattering analysis. The new method could therefore facilitate quantitative descriptions of complex nanoscopic structures inherent to numerous soft-matter systems using small-angle scattering techniques.

36 MATERIALS SCIENCE↗

NASA NTRS2003

Using Grid Benchmarks for Dynamic Scheduling of Grid Applications

Navigation or dynamic scheduling of applications on computational grids can be improved through the use of an application-specific characterization of grid resources. Current grid information systems provide a description of the resources, but do not contain any application-specific information. We define a GridScape as dynamic state of the grid resources. We measure the dynamic performance of these resources using the grid benchmarks. Then we use the GridScape for automatic assignment of the tasks of a grid application to grid resources. The scalability of the system is achieved by limiting the navigation overhead to a few percent of the application resource requirements. Our task submission and assignment protocol guarantees that the navigation system does not cause grid congestion. On a synthetic data mining application we demonstrate that Gridscape-based task assignment reduces the application tunaround time.

Frumkin, Michael↗

NASA NTRS2022

Evaluating Network Performance of Containerized Test Framework for Distributed Space Systems

Distributed space systems are a mission architecture consisting of multiple spacecraft as a cohesive system which provide multipoint sampling, increased mission coverage, or improved sample resolution, while reducing mission risk through redundancy. To fully realize the potential of these systems, eventually scaling to hundreds or thousands of spacecraft, distributed space systems need to be operated as a single entity, which will enable a variety of novel scientific space missions. The Distributed Spacecraft Autonomy (DSA) project is a software project which aims to mature the technology needed for those systems, namely autonomous decision-making and swarm networking. The DSA project leverages a containerized swarm test framework to simulate spacecraft software, which can identify emergent behavior early in development. Container virtualization allows distributed spacecraft systems to be simulated entirely in software on a single computer, avoiding the overhead associated with conventional approaches like hardware facsimiles and virtual machines. For this approach to be effective, the simulated system behavior must not be artificially influenced by the swarm test framework itself. To address this, we present a series of benchmarks to quantify virtual network bandwidth available on a single-host computer and contextualize this against the network and application behavior of the DSA swarm test framework.

Distributed Spacecraft Autonomy↗

DOE OSTI2025

Packaging HEP Heterogeneous Mini-apps for Portable Benchmarking and Facility Evaluation on Modern HPCs

High Energy Physics (HEP) experiments are making increasing use of GPUs and GPU dominated High Performance Computer facilities. Both the software and hardware of these systems are rapidly evolving, creating challenges for experiments to make informed decisions as to where they wish to devote resources. In its first phase, the High Energy Physics Center for Computational Excellence (HEP-CCE) produced portable versions of a number of heterogeneous HEP mini-apps, such as p2r, FastCaloSim, Patatrack and the WireCell Toolkit, that exercise a broad range of GPU characteristics, enabling cross platform and facility benchmarking and evaluation. However, these miniapps still require a significant amount of manual intervention to deploy on a new facility. We present our work in developing turn-key deployments of these mini-apps, where by means of containerization and automated configuration and build techniques such as Spack, we are able to quickly test new hardware, software, environments and entire facilities with minimal user intervention, and then track performance metrics over time.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗

DOE OSTI2025

Packaging HEP Heterogeneous Mini-apps for Portable Benchmarking and Facility Evaluation on Modern HPCs

High Energy Physics (HEP) experiments are making increasing use of GPUs and GPU dominated High Performance Computer facilities. Both the software and hardware of these systems are rapidly evolving, creating challenges for experiments to make informed decisions as to where they wish to devote resources. In its first phase, the High Energy Physics Center for Computational Excellence (HEP-CCE) produced portable versions of a number of heterogeneous HEP mini-apps, such as \ptor, FastCaloSim, Patatrack and the WireCell Toolkit, that exercise a broad range of GPU characteristics, enabling cross platform and facility benchmarking and evaluation. However, these mini-apps still require a significant amount of manual intervention to deploy on a new facility. We present our work in developing turn-key deployments of these mini-apps, where by means of containerization and automated configuration and build techniques such as Spack, we are able to quickly test new hardware, software, environments and entire facilities with minimal user intervention, and then track performance metrics over time.

Atif, Mohammad [Brookhaven] (ORCID:000000026889770↗

DOE OSTI2025

A Review of Quantum Computing Technologies in Power System Optimization

As modern power grids increasingly integrate variable renewable generation, distributed energy resources, and energy storage systems, classical optimization techniques are facing unprecedented challenges. This review examines the emerging application of quantum computing to overcome these challenges in power system optimization, including optimal power flow (OPF), unit commitment (UC), economic dispatch (ED), and intelligent switching and topology optimization (IS-TO). Recent research has introduced various quantum methodologies—such as gate-based, annealing-based, variational algorithms, and quantum-inspired algorithms—to address the combinatorial complexity inherent in grid reconfiguration and energy management. The review summaries the quantum algorithms, quantum devices and the power system test cases, highlighting hybrid quantum–classical strategies that leverage the complementary strengths of both paradigms. Some quantum advantages have been observed, including theoretical speedup, accurate simulation results, scalable qubit usage, efficient QUBO mapping. In particular, the review emphasizes the importance of integrating quantum optimization techniques with classical control frameworks, these hybrid approaches demonstrate the potential to improve real-time grid management and operational reliability. A significant portion of the analysis is devoted to the practical limitations of current quantum devices. Present-day quantum hardware, operating in the noisy intermediate-scale quantum (NISQ) era, remains highly sensitive to noise and limited in qubit connectivity, which constrains the scale and accuracy of implemented algorithms. The review delves into specific challenges such as the need for qubit-efficient encoding techniques and error mitigation strategies that are critical for handling real-world grid optimization problems. In addition, the work draws attention to the performance discrepancies between theoretical quantum speedups and experimental validations, underscoring the importance of rigorous benchmark studies using representative power grid test cases. In summary, this review highlights both the promise and limitations of quantum computing for power system optimization. It provides a comprehensive overview of the state-of-the-art technologies, categorizes recent advancements in algorithm design, and discusses practical considerations for implementation, and serves as an informative resource on current research. Future research directions include developing robust hybrid frameworks, advancing qubit-efficient formulations, and scaling up experimental demonstrations to confirm the theoretical advantages of quantum methods in large-scale power system operations.

24 POWER TRANSMISSION AND DISTRIBUTION↗

DOE OSTI2026

Benchmarking quantum trial wavefunctions for phaseless auxiliary-field quantum Monte Carlo

The phaseless auxiliary-field quantum Monte Carlo (ph-AFQMC) method is a stochastic imaginary-time projection technique for computing ground-state properties of strongly correlated quantum systems, with accuracy that depends critically on the choice of trial wavefunction. Here, we investigate ph-AFQMC with trial states prepared using parameterized quantum circuits. In this work, we present a comprehensive benchmarking study of quantum trial wavefunctions spanning unitary coupled-cluster, Hamiltonian-informed, Jastrow-inspired, and adaptively constructed ansatze. The benchmarking evaluates accuracy, expressibility, and scalability of these ansatze within the QC-AFQMC framework. We test these ansatze on linear hydrogen chains under bond stretching and find that several ansatz families produce chemically accurate ph-AFQMC energies across the dissociation curve. We have performed simulations using the CUDA-Q quantum development platform on the GPU partition of the Perlmutter supercomputer. When comparing ansatze at similar numbers of variational parameters, we find that different ansatz families yield comparable ph-AFQMC results despite exhibiting substantially different variational energies, optimization costs, and circuit depths. Our results indicate that the variational energy of an ansatz is not always a reliable indicator of its quality for ph-AFQMC and reveal instances of over-parameterization. In the strongly correlated regime, trial wavefunctions obtained from adaptive ansatze, exemplified here by ADAPT-VQE with the UCCSD operator pool, can outperform their fixed-ansatz counterparts (UCCSD) in terms of projected energies while using substantially more compact circuits, providing a flexible route to optimize quantum resources within the ph-AFQMC framework.

Rofougaran, Rod [LBNL, Berkeley; Columbia U.; PNL,↗

DOE OSTI2022

Frontier (HPE Cray EX) Exascale Supercomputer at the Oak Ridge Leadership Computing Facility

Frontier is the HPE Cray EX exascale supercomputer deployed and operated by the Oak Ridge Leadership Computing Facility (OLCF) at Oak Ridge National Laboratory (ORNL). Frontier is designed for large-scale modeling, simulation, and AI workloads and is built from HPE Cray EX system architecture with AMD CPUs and AMD Instinct GPU accelerators connected by the HPE Slingshot interconnect. System composition (representative production configuration): Frontier is composed of approximately 74 cabinets with 128 compute nodes per cabinet (~9,400 compute nodes total). Each compute node contains one 64-core AMD EPYC CPU and four AMD Instinct MI250X GPUs. Nodes are connected using HPE Slingshot (Slingshot-200 class) networking with multiple NIC ports per node providing high injection bandwidth. Frontier is connected to the Orion parallel file system (multi-tier Lustre) providing a large, center-wide high-performance storage namespace. Operational context: Frontier entered public prominence as the first system to reach No. 1 on the TOP500 list in May 2022 (HPL benchmark), establishing the first widely recognized exascale-era performance milestone. The system supports DOE Office of Science mission workloads and enables leadership-class computational science and AI for open science users.

AMD EPYC↗

NASA NTRS1997

BACT Simulation User Guide (Version 7.0)

This report documents the structure and operation of a simulation model of the Benchmark Active Control Technology (BACT) Wind-Tunnel Model. The BACT system was designed, built, and tested at NASA Langley Research Center as part of the Benchmark Models Program and was developed to perform wind-tunnel experiments to obtain benchmark quality data to validate computational fluid dynamics and computational aeroelasticity codes, to verify the accuracy of current aeroservoelasticity design and analysis tools, and to provide an active controls testbed for evaluating new and innovative control algorithms for flutter suppression and gust load alleviation. The BACT system has been especially valuable as a control system testbed.

Waszak, Martin R.↗