Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “computer system benchmarking”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 127 records · Page 7
All sourcesNASA NTRSDOE OSTI
NASA NTRS2008

Benchmark Comparison of Dual- and Quad-Core Processor Linux Clusters with Two Global Climate Modeling Workloads

This viewgraph presentation details the science and systems environments that NASA High End computing program serves. Included is a discussion of the workload that is involved in the processing for the Global Climate Modeling. The Goddard Earth Observing System Model, Version 5 (GEOS-5) is a system of models integrated using the Earth System Modeling Framework (ESMF). The GEOS-5 system was used for the Benchmark tests, and the results of the tests are shown and discussed. Tests were also run for the Cubed Sphere system, results for these test are also shown.

McGalliard, James↗
DOE OSTI2020

Fast-Spectrum Critical Assemblies with a Pb-HEU Core Surrounded by a Copper Reflector

The Department of Energy invests tens of millions of dollars each year to develop the next generation of nuclear engineering modeling & simulation (M&S) tools. These M&S tools are used to analyze advanced reactor designs and the safety of current nuclear operations. As computers become more powerful, we are able to enhance resolution in our calculations. This improved resolution is taking us to a point where the limitations of simulation capability are in the quality of data, including our ability to quantify the uncertainty and sensitivity of the data. In order to accurately model systems of interest, the industry must improve key nuclear data measurements and our confidence of how well we understand the data. Thus, M&S tools need evaluated and quality-assured experimental data for validation purposes. The International Criticality Safety Benchmark Evaluation Project (ICSBEP) compiles benchmark experiment data in a handbook that can be used by criticality safety engineers to validate computer codes and cross-section libraries at nuclear facilities. Both critical and subcritical experiments are included in the handbook. Figure 1 organizes all the benchmark evaluations that have been performed by the isotope of interest, in this case Pb, and the neutron energy within the system. Compared to other isotopes of interest for nuclear applications, there are few benchmark evaluations for Pb systems. This has caused the latest nuclear cross-section libraries to over/underestimate changes in the neutron population compared to experimental results. Therefore, this evaluation fills an important knowledge gap in benchmark evaluations.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗
DOE OSTI2022

WfBench: Automated Generation of Scientific Workflow Benchmarks

The prevalence of scientific workflows with high computational demands calls for their execution on various distributed computing platforms, including large-scale leadership-class high-performance computing (HPC) clusters. To handle the deployment, monitoring, and optimization of workflow executions, many workflow systems have been developed over the past decade. There is a need for workflow benchmarks that can be used to evaluate the performance of workflow systems on current and future software stacks and hardware platforms.We present a generator of realistic workflow benchmark specifications that can be translated into benchmark code to be executed with current workflow systems. Our approach generates workflow tasks with arbitrary performance characteristics (CPU, memory, and I/O usage) and with realistic task dependency structures based on those seen in production workflows. We present experimental results that show that our approach generates benchmarks that are representative of production workflows, and conduct a case study to demonstrate the use and usefulness of our generated benchmarks to evaluate the performance of workflow systems under different configuration scenarios.

Coleman, Taina↗
DOE OSTI2021

Benchmarking numerical methods for impact and cratering applications

Large scale computational models are important for studying impact cratering events that are prevalent both on Earth and, more broadly, in this solar system. To address these problems, models must reliably account for both large length scales (e.g., kilometers) and relatively long time scales (hundreds of seconds). This work benchmarks two such approaches, a more traditional hydrodynamics approach and a finite-discrete element method (FDEM), for impact cratering applications. Both 2D and 3D results are discussed for two different impact velocities, 5 km/s and 20 km/s, striking normal to the target and, for 3D simulations, 45° from vertical. In addition, comparisons to previously published data are presented. Finally, differences in how these methods model damage are discussed. Ultimately, both approaches show successful modeling of several different impact scenarios.

58 GEOSCIENCES↗
DOE OSTI2022

SBbadger: biochemical reaction networks with definable degree distributions

Abstract Motivation An essential step in developing computational tools for the inference, optimization and simulation of biochemical reaction networks is gauging tool performance against earlier efforts using an appropriate set of benchmarks. General strategies for the assembly of benchmark models include collection from the literature, creation via subnetwork extraction and de novo generation. However, with respect to biochemical reaction networks, these approaches and their associated tools are either poorly suited to generate models that reflect the wide range of properties found in natural biochemical networks or to do so in numbers that enable rigorous statistical analysis. Results In this work, we present SBbadger, a python-based software tool for the generation of synthetic biochemical reaction or metabolic networks with user-defined degree distributions, multiple available kinetic formalisms and a host of other definable properties. SBbadger thus enables the creation of benchmark model sets that reflect properties of biological systems and generate the kinetics and model structures typically targeted by computational analysis and inference software. Here, we detail the computational and algorithmic workflow of SBbadger, demonstrate its performance under various settings, provide sample outputs and compare it to currently available biochemical reaction network generation software. Availability and implementation SBbadger is implemented in Python and is freely available at https://github.com/sys-bio/SBbadger and via PyPI at https://pypi.org/project/SBbadger/. Documentation can be found at https://SBbadger.readthedocs.io. Supplementary information Supplementary data are available at Bioinformatics online.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2026

Fermionic mean-field dynamics for spin systems beyond free fermions

We introduce the fermionized time-dependent Hartree–Fock (fTDHF), a real-time quantum dynamics method for spin-1/2 Hamiltonians following their mapping to fermions via the Jordan-Wigner transformation. fTDHF is formally equivalent to exact dynamics in the case of free fermions, and can efficiently handle non-local string operators arising from long-range interactions via transition matrix elements between non-orthogonal Slater determinants. We show that the fTDHF method can be implemented on a classical computer with a cost that scales polynomially with system size, and linearly with the time steps. We benchmark fTDHF against exact dynamics on three separate spin-1/2 models, representing adiabatic preparation of states with long-range correlations, disorder-driven observation of many-body localization, and particle production in the Schwinger model. For each of these systems, fTDHF is shown to reproduce the qualitative dynamics generated by the exact evolutions, while maintaining a simple physical picture due to its mean-field nature.

Dutta, Rishab↗
DOE OSTI2021

Best Practices Guide: Benchmarking Energy Efficiency in Laboratories

Purpose and Audience A wide spectrum of laboratory owners, ranging from universities to federal agencies, have explicit goals for energy efficiency and greenhouse gas reductions in their facilities. For example, new federal buildings and major renovations of existing buildings are to reduce fossil fuel-generated energy consumption by 90% in 2025, and 100% in 2030, compared with a 2003 baseline (FEMP n.d.). Minnesota SB2030 standard requires achieving an 80% reduction from the average building baseline for commercial, institutional, and industrial buildings (SB2030 n.d). A laboratory—new or existing—is much more likely to meet energy efficiency goals if quantitative metrics and targets are explicitly specified and tracked over the life cycle of the building, from design through construction, commissioning, operations, and renovations. If efficiency targets are not explicitly and properly defined, any additional capital costs or design time associated with attaining higher efficiencies can be difficult to justify relative to other priorities. The purpose of this guide is to provide guidance on how to specify and compute energy efficiency metrics and benchmarks for laboratories, at the whole-building as well as the system level. The information in this guide can be used to incorporate quantitative metrics and targets into new construction or retrofit of existing facilities. For information on strategies and technologies to achieve energy efficiency, the reader is referred to I2SL resources, including technology best practice guides and case studies.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗
NASA NTRS2003

Computational Hemodynamic Simulation of Human Circulatory System under Altered Gravity

A computational hemodynamics approach is presented to simulate the blood flow through the human circulatory system under altered gravity conditions. Numerical techniques relevant to hemodynamics issues are introduced to non-Newtonian modeling for flow characteristics governed by red blood cells, distensible wall motion due to the heart pulse, and capillary bed modeling for outflow boundary conditions. Gravitational body force terms are added to the Navier-Stokes equations to study the effects of gravity on internal flows. Six-type gravity benchmark problems are originally presented to provide the fundamental understanding of gravitational effects on the human circulatory system. For code validation, computed results are compared with steady and unsteady experimental data for non-Newtonian flows in a carotid bifurcation model and a curved circular tube, respectively. This computational approach is then applied to the blood circulation in the human brain as a target problem. A three-dimensional, idealized Circle of Willis configuration is developed with minor arteries truncated based on anatomical data. Demonstrated is not only the mechanism of the collateral circulation but also the effects of gravity on the distensible wall motion and resultant flow patterns.

Kim. Chang Sung↗
DOE OSTI2023

Impact of Thermal Scattering Law on Similarity Assessment in Light-Water or Polyethylene-Moderated Systems

A collaborative effort between Pacific Northwest National Laboratory (PNNL) and Oak Ridge National Laboratory (ORNL) is underway to provide a technical basis and methodology for the criticality safety community to use the sum of fractions (SoF) method for generating limits for mixtures of “selected actinide nuclides” included in the ANSI/ANS 8.15 standard. The PNNL scope in this project is to define a range of mixtures of 233 U, 235 U, and 239 Pu moderated with either light water or polyethylene and to examine the critical masses for these mixtures. The ORNL scope is primarily to provide validation support for these studies. More complete discussion of the project and its validation aspects will be presented at the upcoming International Conference on Nuclear Criticality Safety (ICNC) this Fall in Sendai, Japan. Clear differences in benchmark similarity to application systems as assessed by the integral parameter ck are noted in the validation studies performed as part of this project as a function of moderator. The c k value is a correlation coefficient that represents that amount of shared uncertainty in k eff due to cross sections between two systems. Individual nuclide-reaction contributions between the two systems can be simply summed to arrive at the total c k value. Specifically, the c k values for light-water–moderated solution experiments are higher for a water-moderated application than for a polyethylene-moderated application. This result is neither totally unexpected nor surprising, but the magnitude of the difference was difficult to anticipate. The TSUNAMI sequence, in the SCALE 6.2.4 code package developed by ORNL, was used to generate eigenvalues and reactivity effects with perturbation-theory based approach through sensitivity coefficients for all nuclides in the system with all reactions and energy groups. The TSUNAMI-Indices and Parameters (IP) sequence then uses the sensitivity data generated through TSUNAMI to generate relational parameters (i.e., c k ) to determine the degree of similarity between systems. One detail of the SCALE material and data implementation must be discussed at this point. Several thermal scattering laws (TSLs) are available for 1 H. SCALE uses a different nuclide ID number for each TSL; essentially, each version of 1 H is treated as a unique nuclide. For example, 1 H bound in water ( 1 H-H2O) is assigned the nuclide ID 1001, whereas 1 H bound in polyethylene (h-poly) is assigned the nuclide ID 9001001. The same cross section data are used for all reactions in 1 H, regardless of TSL, except for scattering below the TSL cutoff energy. TSUNAMI-IP treats different nuclide IDs as different nuclides; thus, no uncertainty is shared between 1 H-H2O and h-poly, despite much of the same data, including covariance data, being used for both nuclides. This presents a question: how much of the difference in assessed similarity between water- and polyethylene-moderated systems is due to the moderators, and how much is caused by the treatment of 1 H-H2O and h poly with cross section and covariance data. The extended edits generated by TSUANMI-IP allow for an investigation of this issue specifically, as well as a demonstration of the general techniques available within TSUNAMI to understand the results of the similarity assessment. This paper presents and analyzes the similarity assessment of both water- and polyethylene-moderated systems for a single benchmark: PST-002-001.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗
DOE OSTI2025

LC-Opt: Benchmarking Reinforcement Learning and Agentic AI for End-to-End Liquid Cooling Optimization in Data Centers

Liquid cooling is critical for thermal management in high-density data centers with the rising AI workloads. However, machine learning-based controllers are essential to unlock greater energy efficiency and reliability, promoting sustainability. We present LC-Opt, a Sustainable Liquid Cooling (LC) benchmark environment, for reinforcement learning (RL) control strategies in energy-efficient liquid cooling of high-performance computing (HPC) systems. Built on the baseline of a high-fidelity digital twin of Oak Ridge National Lab's Frontier Supercomputer cooling system, LC-Opt provides detailed Modelica-based end-to-end models spanning site-level cooling towers to data center cabinets and server blade groups. RL agents optimize critical thermal controls like liquid supply temperature, flow rate, and granular valve actuation at the IT cabinet level, as well as cooling tower (CT) setpoints through a Gymnasium interface, with dynamic changes in workloads. This environment creates a multi-objective real-time optimization challenge balancing local thermal regulation and global energy efficiency, and also supports additional components like a heat recovery unit (HRU). We benchmark centralized and decentralized multi-agent RL approaches, demonstrate policy distillation into decision and regression trees for interpretable control, and explore LLM-based methods that explain control actions in natural language through an agentic mesh architecture designed to foster user trust and simplify system management. LC-Opt democratizes access to detailed, customizable liquid cooling models, enabling the ML community, operators, and vendors to develop sustainable data center liquid cooling control solutions.

Naug, Avisek [Hewlett Packard Enterprise]↗
DOE OSTI2026

OptiBench: An Optimization Benchmark Tool for Renewable Energy Problems

We propose a benchmark framework and visualization tool, OptiBench, for analyzing the performance of state-of-the-art optimization solvers across a variety of optimization problems in renewable energy research. Our framework is designed from the ground up in the Julia programming language and enables analysis at scale on high performance computing (HPC) systems. Our visualization tool allows effortless evaluation of optimization solver performance, robustness, and accuracy through intuitive plots, e.g., performance profiles, heat maps, and distribution plots. We have tested three benchmark suites relevant to the modeling of renewable energy systems, viz., CUTEst, PGLib-OPF, and WaterTAP water treatment optimization problems. We illustrate benchmarking of CUTEst using OptiBench on the National Laboratory of the Rockies's (NLR) HPC Kestrel. Our findings indicate that MA57 HSL linear solver demonstrated the best overall performance for an experimental IPOPT implementation. Our work is ongoing and we intend to add support for more optimization solvers and benchmark test suites in the future.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2024

CI/CD Efforts for Validation, Verification and Benchmarking OpenMP Implementations

Software developers must adapt to keep up with the changing capabilities of platforms so that they can utilize the power of High-Performance Computers (HPC), including exascale systems. OpenMP, a directive-based parallel programming model, allows developers to include directives to existing C, C++, or Fortran code to allow node level parallelism without compromising performance. This paper describes our CI/CD efforts to provide easy evaluation of the support of OpenMP across different compilers using existing testsuites and benchmark suites on HPC platforms. Our main contributions include (1) the set of a Continuous Integration (CI) and Continuous Development (CD) workflow that captures bugs and provides faster feedback to compiler developers, (2) an evaluation of OpenMP (offloading) implementations supported by AMD, HPE, GNU, LLVM, and Intel, and (3) evaluation of the quality of compilers across different heterogeneous HPC platforms. With the comprehensive testing through the CI/CD workflow, we aim to provide a comprehensive understanding of the current state of OpenMP (offloading) support in different compilers and heterogeneous platforms consisting of CPUs and GPUs from NVIDIA, AMD, and Intel.

Jarmusch, Aaron↗
DOE OSTI2025

OptiBench: An Optimization Benchmark Tool for Renewable Energy Problems

We propose a benchmark framework and visualization tool, OptiBench, for analyzing the performance of state-of-the-art optimization solvers across a variety of optimization problems in renewable energy research. Our framework is designed from the ground up in the Julia programming language and enables analysis at scale on high performance computing (HPC) systems. Our visualization tool allows effortless evaluation of optimization solver performance, robustness, and accuracy through intuitive plots, e.g., performance profiles, heat maps, and distribution plots. We have tested three benchmark suites relevant to the modeling of renewable energy systems, viz., CUTEst, PGLib-OPF, and WaterTAP water treatment optimization problems. We illustrate benchmarking of CUTEst using OptiBench on the National Renewable Energy Laboratory's (NREL) HPC Kestrel. Our findings indicate that MA57 HSL linear solver demonstrated the best overall performance for an experimental IPOPT implementation. Our work is ongoing and we intend to add support for more optimization solvers and benchmark test suites in the future.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗
DOE OSTI2023

Adaptive variational quantum minimally entangled typical thermal states for finite temperature simulations

Scalable quantum algorithms for the simulation of quantum many-body systems in thermal equilibrium are important for predicting properties of quantum matter at finite temperatures. Here we describe and benchmark a quantum computing version of the minimally entangled typical thermal states (METTS) algorithm for which we adopt an adaptive variational approach to perform the required quantum imaginary time evolution. The algorithm, which we name AVQMETTS, dynamically generates compact and problem-specific quantum circuits, which are suitable for noisy intermediate-scale quantum (NISQ) hardware. We benchmark AVQMETTS on statevector simulators and perform thermal energy calculations of integrable and nonintegrable quantum spin models in one and two dimensions and demonstrate an approximately linear system-size scaling of the circuit complexity. We further map out the finite-temperature phase transition line of the two-dimensional transverse field Ising model. Finally, we study the impact of noise on AVQMETTS calculations using a phenomenological noise model.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
NASA NTRS1998

Parallelization of NAS Benchmarks for Shared Memory Multiprocessors

This paper presents our experiences of parallelizing the sequential implementation of NAS benchmarks using compiler directives on SGI Origin2000 distributed shared memory (DSM) system. Porting existing applications to new high performance parallel and distributed computing platforms is a challenging task. Ideally, a user develops a sequential version of the application, leaving the task of porting to new generations of high performance computing systems to parallelization tools and compilers. Due to the simplicity of programming shared-memory multiprocessors, compiler developers have provided various facilities to allow the users to exploit parallelism. Native compilers on SGI Origin2000 support multiprocessing directives to allow users to exploit loop-level parallelism in their programs. Additionally, supporting tools can accomplish this process automatically and present the results of parallelization to the users. We experimented with these compiler directives and supporting tools by parallelizing sequential implementation of NAS benchmarks. Results reported in this paper indicate that with minimal effort, the performance gain is comparable with the hand-parallelized, carefully optimized, message-passing implementations of the same benchmarks.

Waheed, Abdul↗
NASA NTRSDate not supplied

Ab initio ground states of strongly-correlated materials on quantum computers

The accurate first-principles description of strongly-correlated materials is an important and challenging problem in condensed matter physics. Ab initio downfolding has emerged as a way of deriving accurate many-body Hamiltonians including strong correlations, representing a subspace of interest of a material, using density functional theory calculations as a starting point. However, the solution of these material-specific models can scale exponentially on classical computers, constituting a challenge. Here we propose that utilizing quantum computers for obtaining the properties of downfolded Hamiltonians yields an accurate description of the ground state properties of strongly-correlated systems, while circumventing the exponential scaling problem. We benchmark the solution of Hubbard-like models obtained through downfolding by utilizing a classical tensor network implementation of variational quantum eigensolvers (VQE), and we reveal a strategy for driving the optimization through a hybrid minimization of the energy and maximization of the overlap with an approximate solution obtained through low-cost computational methods. This results in a reduction of the energy error by orders of magnitude compared to conventional VQE approaches, and allows us to reproduce long-range correlations for the first time. We demonstrate our first-principles approach for diverse strongly-correlated materials, correctly predicting the antiferromagnetic state of one-dimensional cuprate Ca 2 CuO 3 , the excitonic ground state of monolayer WTe2, and the charge-ordered state of correlated metal SrVO 3 . Our efficient computational implementation allows us to simulate large systems with up to 54 qubits and encompassing up to four correlated bands, which is indicative of the complexity that our framework can address.

Antonios M Alvertis↗
DOE OSTI2023

A step towards the final frontier: Lessons learned from acceptance testing of the first HPE/Cray EX 3000 system at ORNL

Summary In this article, we summarize the deployment of the Air Force Weather (AFW) HPC11 system at Oak Ridge National Laboratory (ORNL) including the process followed to successfully complete acceptance testing of the system. HPC11 is the first HPE/Cray EX 3000 system that has been successfully released to its user community in a federal facility. HPC11 consists of two identical 800‐node supercomputers, Fawbush and Miller, with access to two independent and identical lustre parallel file systems. HPC11 is equipped with Slingshot 10 interconnect technology and relies on the HPE Performance Cluster Manager software for system configuration. ORNL has a clearly defined acceptance testing process used to ensure that every new system deployed can provide the necessary capabilities to support user workloads. We worked closely with HPE and AFW to develop a set of tests that used the United Kingdom's Meteorological Office's Unified Model and 4‐dimensional variational data assimilation. We also included benchmarks and applications from the Oak Ridge Leadership Computing Facility portfolio to fully exercise the HPE/Cray programming environment and evaluate the functionality and performance of the system. Acceptance testing of HPC11 required parallel execution of each element on Fawbush and Miller. In addition, careful coordination was needed to ensure successful acceptance of the newly deployed lustre file systems alongside the compute resources. In this work, we present test results from specific system components and provide an overview of the issues identified, challenges encountered, and the lessons learned along the way.

Melesse Vergara, Verónica G.↗
NASA NTRS2013

Overview of the Aeroelastic Prediction Workshop

The Aeroelastic Prediction Workshop brought together an international community of computational fluid dynamicists as a step in defining the state of the art in computational aeroelasticity. This workshop's technical focus was prediction of unsteady pressure distributions resulting from forced motion, benchmarking the results first using unforced system data. The most challenging aspects of the physics were identified as capturing oscillatory shock behavior, dynamic shock-induced separated flow and tunnel wall boundary layer influences. The majority of the participants used unsteady Reynolds-averaged Navier Stokes codes. These codes were exercised at transonic Mach numbers for three configurations and comparisons were made with existing experimental data. Substantial variations were observed among the computational solutions as well as differences relative to the experimental data. Contributing issues to these differences include wall effects and wall modeling, non-standardized convergence criteria, inclusion of static aeroelastic deflection, methodology for oscillatory solutions, post-processing methods. Contributing issues pertaining principally to the experimental data sets include the position of the model relative to the tunnel wall, splitter plate size, wind tunnel expansion slot configuration, spacing and location of pressure instrumentation, and data processing methods.

Heeg, Jennifer↗