SEARCH · Engineering Papers

Results for “Performance benchmark”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 991 records · Page 55

All sources NASA NTRS DOE OSTI

DOE OSTI2021

Milestone M6 Report: Reducing Excess Data Movement Part 1

This is the second in a sequence of three Hardware Evaluation milestones that provide insight into the following questions: What are the sources of excess data movement across all levels of the memory hierarchy, going out to the network fabric? What can be done at various levels of the hardware/software hierarchy to reduce excess data movement? How does reduced data movement track application performance? The results of this study can be used to suggest where the DOE supercomputing facilities, working with their hardware vendors, can optimize aspects of the system to reduce excess data movement. Quantitative analysis will also benefit systems software and applications to optimize caching and data layout strategies. Another potential avenue is to answer cost-benefit questions, such as those involving memory capacity versus latency and bandwidth. This milestone focuses on techniques to reduce data movement, quantitatively evaluates the efficacy of the techniques in accomplishing that goal, and measures how performance tracks data movement reduction. We study a small collection of benchmarks and proxy mini-apps that run on pre-exascale GPUs and on the Accelsim GPU simulator. Our approach has two thrusts: to measure advanced data movement reduction directives and techniques on the newest available GPUs, and to evaluate our benchmark set on simulated GPUs configured with architectural refinements to reduce data movement.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2021

Milestone M7 Report: Reducing Excess Data Movement Part 2

This milestone evaluates techniques to measure and, if possible, reduce data movement across all levels of the memory hierarchy, focusing on CPU/GPU page level data movement and on intra-GPU memory hierarchy. We quantitatively evaluate the efficacy of the techniques in reducing data movement and measure how performance tracks data movement reduction. We study a small collection of benchmarks and proxy mini-apps that run on advanced pre-exascale GPUs and on the Accelsim GPU simulator. Our approach has two thrusts: to measure advanced data movement reduction directives and techniques on the newest available GPUs, and to evaluate our benchmark set on simulated GPUs configured with architectural refinements to reduce data movement. We primarily evaluated NVidia-based architectures due to the unavailability of AMD GPU hardware and tools until very recently.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2023

Quarknet Project: Simulating Double-Source Plane Lenses and Einstein Rings for Domain Adaptation

Double-Source Plane Lenses (DSPLs) are a form of strong gravitational lensing that can be used to uniquely constrain cosmological parameters like the dark energy equation of state. These objects are exceedingly rare: to date, only four have been discovered in astronomical surveys, like the Sloan Digital Sky Survey and the HyperSuprimeCam Survey. Additionally, the main features of these objects are typically co-located sets of arcs, which look like Einstein Rings of single-source galaxy-galaxy lenses. Larger astronomical surveys typically have low image resolution, which significantly exacerbates issues when trying to find DSPLs. Future surveys are likely to include many more of these objects and have higher image resolution. Traditional methods for finding strong lenses struggle to identify simple galaxy-galaxy lenses. Recently, neural networks have become the standard for lens identification, but these tools have not yet been used to search for DSPLs. In this work, we use the software deeplenstronomy to simulate varied images of DSPLs and Einstein Rings – with and without noise and artifacts of observational surveys, forming a comprehensive benchmark dataset of 40,000 images. We utilise this dataset to perform experiments with convolutional neural networks to discern between the two phenomena. Sofia Grimm and Jerry Zhou are co-first authors on this work.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗

DOE OSTI2023

Hardware-in-the-loop Testing of Network Protectors for Low-Voltage Networks with Distributed Energy Resources

In this report, we developed and validated a network protector relay digital twin model and interfaced a commonly used network protector relay hardware with our real-time simulation system. Hardware-in-the-loop protection studies are performed to assess the impact of distributed energy resources (DER) and benchmark a rate-of-change-based mitigation strategy. Simulation results suggest that the network protector reverse trip and auto-reclose functions are negatively impacted by the high distributed energy resource penetration. To accommodate DER backfeed while remaining secure and reliable for faults on primary feeders, we recommend options for a rate-of-change-based blocking scheme and a protection setting change. Finally, future mitigation ideas and standard revisions are discussed.

24 POWER TRANSMISSION AND DISTRIBUTION↗

DOE OSTI2024

An overview of the E3SM version 2 large ensemble and comparison to other E3SM and CESM large ensembles

Abstract. This work assesses a recently produced 21-member climate model large ensemble (LE) based on the U.S. Department of Energy's Energy Exascale Earth System Model (E3SM) version 2 (E3SM2). The ensemble spans the historical era (1850 to 2014) and 21st century (2015 to 2100), using the SSP370 pathway, allowing for an evaluation of the model's forced response. A companion 500-year preindustrial control simulation is used to initialize the ensemble and estimate drift. Characteristics of the LE are documented and compared against other recently produced ensembles using the E3SM version 1 (E3SM1) and Community Earth System Model (CESM) versions 1 and 2. Simulation drift is found to be smaller, and model agreement with observations is higher in versions 2 of E3SM and CESM versus their version 1 counterparts. Shortcomings in E3SM2 include a lack of warming from the mid to late 20th century, likely due to excessive cooling influence of anthropogenic sulfate aerosols, an issue also evident in E3SM1. Associated impacts on the water cycle and energy budgets are also identified. Considerable model dependence in the response to both aerosols and greenhouse gases is documented and E3SM2's sensitivity to variable prescribed biomass burning emissions is demonstrated. Various E3SM2 and CESM2 model benchmarks are found to be on par with the highest-performing recent generation of climate models, establishing the E3SM2 LE as an important resource for estimating climate variability and responses, though with various caveats as discussed herein. As an illustration of the usefulness of LEs in estimating the potential influence of internal variability, the observed CERES-era trend in net top-of-atmosphere flux is compared to simulated trends and found to be much larger than the forced response in all LEs, with only a few members exhibiting trends as large as observed, thus motivating further study.

58 GEOSCIENCES↗

DOE OSTI2023

Neutron Noise Analysis of the Neptunium Subcritical Observation (NESO) Experiment

The Neptunium Subcritical Observation (NeSO) experiment was a subcritical integral benchmark experiment. This experiment was designed in 2017-2018 and was performed at the National Criticality Experiments Research Center (NCERC) in 2019. Eight different subcritical configurations were measured with varying Nickel reflector thickness. For all configurations, a 3 He detection system was used to record neutron events. The focus of this work is to share neutron noise results of the measured data.

3-Helium↗

DOE OSTI2023

Neutron Noise Analysis of the Neptunium Subcritical Observation (NESO) Experiment

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗

DOE OSTI2024

An Efficient Numerical Algorithm for Solving Coupled Time-Dependent Ginzburg-Landau Equation for Superconductivity and Elasticity

A decoupled finite element algorithm is developed for simulating the vortex dynamics on an elastic superconductor which couples the time-dependent Ginzburg- Landau equation with the complex-valued superconducting order parameter and the vector-valued magnetic potential, and the elasticity equation. We present an iterative algorithm for the decoupled system arising from the time and spatial discretization using a combination of preconditioner, algebraic multigrid method (AMG) and preconditioned conjugate gradient method (PCG). The iterative algorithm allows us to perform large-scale three-dimensional simulations of mesoscale pattern formation during superconducting phase transitions with arbitrary elastic boundary conditions. Here, the performance and efficiency of the algorithm are numerically verified by several benchmark problems, exhibiting up to two orders of magnitude improvement depending on the scale of discrete system compared to the exact solver.

Efficiency↗

DOE OSTI2024

“Understanding Robustness Lottery”: A Geometric Visual Comparative Analysis of Neural Network Pruning Approaches

Deep learning approaches have provided state-of-the-art performance in many applications by relying on large and overparameterized neural networks. However, such networks are very brittle and are difficult to deploy on resource-limited platforms. Model pruning, i.e., reducing the size of the network, is a widely adopted strategy that can lead to a more robust and compact model. Many heuristics exist for model pruning, but our understanding of the pruning process remains limited due to the black-box nature of a neural network model. Empirical studies show that some heuristics improve performance whereas others can make models more brittle. Here, this work aims to shed light on how different pruning methods alter the network’s internal feature representation and the corresponding impact on model performance. To facilitate a comprehensive comparison and characterization of the high-dimensional model feature space, we introduce a visual geometric analysis of feature representations. We evaluated a set of critical geometric concepts decomposed from the commonly adopted classification loss and used them to design a visualization system to compare and highlight the impact of pruning on model performance and feature representation. The proposed tool provides an environment for an in-depth comparison of pruning methods and a comprehensive understanding of how the model responds to common data corruption. By leveraging the proposed visualization, machine learning researchers can reveal the similarities between pruning methods and redundancy in robustness evaluation benchmarks, obtain geometric insights about the differences between pruned models that achieve superior robustness performance, and identify samples that are robust or fragile to model pruning and common data corruption.

Li, Zhimin [Univ. of Utah, Salt Lake City, UT (Uni↗

DOE OSTI2026

A Benchmarking Framework for Evaluating Large Language Model Capabilities in Nuclear Reactor Safety Applications

Large language models (LLMs) are increasingly capable of answering technical questions, synthesizing domain knowledge, and supporting engineering workflows. For nuclear science and engineering, these capabilities require careful, domain-specific evaluation before they can be credibly incorporated into safety-related activities, regulatory review, or technical decision support. This paper presents preliminary results from benchmarking framework for evaluating LLM capabilities in nuclear contexts. The framework is organized into three evaluation categories: nuclear fundamentals, general dual-use knowledge, and plant specific knowledge. These categories are intended to distinguish general nuclear engineering competence from broader technical reasoning and more context-dependent nuclear knowledge. Initial evaluations focus on nuclear fundamentals using questions representative of the knowledge expected of a nuclear professional engineer. Results indicate that contemporary frontier models perform at a high level and substantially exceed the performance of older model generations, with some models approaching saturation of the current benchmark. These findings suggest both the rapid improvement of LLM capabilities in specialized technical domains and the need for more discriminating evaluation methods. The paper presents the benchmark structure, preliminary model-comparison results, and ongoing work. This work supports development of verifiable, responsible, and safety-conscious methods for assessing AI systems in nuclear engineering applications.

21 - SPECIFIC NUCLEAR REACTORS AND ASSOCIATED PLAN↗

DOE OSTI2025

Summary Report Of The FY25 Computational Fluid Dynamics Verification And Validation Exercises In The Advanced Reactor Technologies - Gas-cooled Reactor Program

Verification and Validation (V&V) of numerical tools is critical for ensuring reasonable predictions during design, safety analysis, and licensing. Recent work in the Advanced Reactor Technologies - Gas-cooled Reactor (ART-GCR) program has focused on V&V of common Computational Fluid Dynamics (CFD) tools that are used within the Untied States. This report presents an update on these CFD V&V activities. These Generation IV Forum (GIF) Very High Temperature Reactor (VHTR) Computational Methods, Validation, and Benchmarking (CMVB) is an international organization dedicated to the verification and validation of High Temperature Gas-Cooled Reactor (HTGR) simulation tools. Participation in the CMVB provides additional value to the V&V activities, as it allows for access to a wider range of data, and provides valuable benchmarking exercises. Three HTGR phenomena are targeted: Reactor Cavity Cooling System (RCCS) performance, core bypass flow, and lower plenum mixing. Simulations of the University of Wisconsin-Madison (UW-Madison) RCCS facilities are performed with Reynolds Averaged Navier-Stokes (RANS) in StarCCM+. Results are compared for both forced and natural convection conditions, with both exhibiting good agreement with experimental measurements. The Idaho National Laboratory (INL) matched index of refraction (MIR) and Korean Atomic Energy Research Institute (KAERI) bypass flow expeirments are used to validation CFD predictions of bypass flow. Simulations are performed with RANS in StarCCM+ and with Large Eddy Simulation (LES) in NekRS. Finally, preliminary simulations of the Institute of Nuclear and New Energy Technology (INET) lower plenum mixing facilities are presented. Initial work has developed models with LES, RANS, and porous media models. These preliminary models are presented and compared to each other to gauge differences in predictions with each of the three methods.

and Benchmarking (CMVB)↗

DOE OSTI2022

Assessing the WRF-Solar Model Performance Using Satellite-Derived Irradiance from the National Solar Radiation Database

Abstract WRF-Solar is a numerical weather prediction model specifically designed to meet the increasing demand for accurate solar irradiance forecasting. The model provides flexibility in the representation of the aerosol–cloud–radiation processes. This flexibility can be argued to make it more difficult to improve the model’s performance because of the necessity of inspecting different configurations. To alleviate this situation, WRF-Solar has a reference configuration to use as a benchmark in sensitivity experiments. However, the scarcity of high-quality ground observations is a handicap to accurately quantify the model performance. An alternative to ground observations are satellite irradiance retrievals. Herein we analyze the adequacy of the National Solar Radiation Database (NSRDB) to validate the WRF-Solar performance using high-quality global horizontal irradiance (GHI) observations across the contiguous United States (CONUS). Based on the sufficient performance of NSRDB, we further analyze the WRF-Solar forecast errors across the CONUS, the growth of the forecasting errors as a function of the lead time, and sensitivities to the grid spacing and the representation of the radiative effects of unresolved clouds. Our results based on WRF-Solar forecasts spanning 2018 reveal a 7% median degradation of the mean absolute error (MAE) from the first to the second daytime period. Reducing the grid spacing from 9 to 3 km leads to a 4% improvement in the MAE, whereas activating the radiative effects of unresolved clouds is desirable over most of the CONUS even at 3 km of grid spacing. A systematic overestimation of the GHI is found. These results illustrate the potential of GHI retrievals to contribute to increasing the WRF-Solar performance.

14 SOLAR ENERGY↗

DOE OSTI2025

RAVEN Template for Dynamic Representativity Analysis of the High Temperature Test Facility

These slides present a walkthrough of the template that has been developed for using RAVEN to perform representativity analysis using models of the High Temperature Test Facility and the General Atomics Modular High Temperature Gas-cooled Reactor. The presentation provides participants in the HTTF benchmark with a walkthrough on how to use RAVEN for their sensitivity analysis and how to read results from the MHTGR-350 to perform representativity

22 - GENERAL STUDIES OF NUCLEAR REACTORS↗

DOE OSTI2020

Recent developments in catalyst-related PEM fuel cell durability

Cost and durability remain the two major barriers to the widespread commercialization of polymer electrolyte membrane fuel cell (PEMFC)-based power systems, especially for the most impactful but challenging fuel cell electric vehicle (FCEV) application. Commercial FCEVs are now on the road; however, their PEMFC systems do not meet the cost targets established by the U.S. Department of Energy, primarily due to the high platinum loading needed on the cathode to achieve the requisite performance and lifetime. Furthermore, while the activities of a number of commercial Pt-based alloy cathode catalysts exceed the beginning-of-life (BOL) targets, these activities, and the overall cathode performance, degrade via a variety of mechanisms described herein. Degradation is mitigated in current FCEVs by utilizing a cathode catalyst with a lower BOL activity (e.g., much lower transition metal alloy content and larger BOL nanoparticle size), necessitating higher catalyst loadings, and through the utilization of system controls that avoid conditions known to exacerbate degradation processes, such as limiting the fuel cell stack voltage range. The design and development of active and robust materials and eliminating the need for vehicle mitigation strategies would greatly simplify the operating system, allowing for greater transient operation, avoiding large hybridization, and curtailing of fuel cell power. Although system mitigation strategies have provided the near-term pathway for FCEV commercialization, material-specific solutions are required to further reduce costs and improve operability and efficiency. Future material developments should focus on stabilization of the electrode structure and minimization of the catalyst particle susceptibility to dissolution caused by oxide formation and reduction over PEMFC cathode-relevant operating potentials plus minimization of support corrosion. Ex situ accelerated stress tests have provided insight into the processes responsible for material and performance degradation and will continue to provide useful information on the relative stability of materials and benchmarks for robust and stable materials-based solutions not requiring system mitigation strategies to achieve adequate lifetime.

25 ENERGY STORAGE↗

DOE OSTI2020

A broad-host-range event detector: expanding and quantifying performance between Escherichia coli and Pseudomonas species

Modern microbial biodesign relies on the principle that well-characterized genetic parts can be reused and reconfigured for different functions. However, this paradigm has only been successful in a limited set of hosts, mostly comprised from common lab strains of Escherichia coli . It is clear that new applications such as chemical sensing and event logging in complex environments will benefit from new host chassis. This study quantitatively compared how the same chemical event logger performed across four strains and three different microbial species. An integrase-based sensor and memory device was operated by two representative soil Pseudomonads— Pseudomonas fluorescens SBW25 and Pseudomonas putida DSM 291. Quantitative comparisons were made between these two non-traditional hosts and two benchmark E. coli chassis including the probiotic Nissle 1917 and common cloning strain DH5α. The performance of sensor and memory components changed according to each host, such that a clear chassis effect was observed and quantified. These results were obtained via fluorescence from reporter proteins that were transcriptionally fused to the integrase and downstream recombinant region and via data-driven kinetic models. The Pseudomonads proved to be acceptable chassis for the operation of this event logger, which outperformed the common E. coli DH5α in many ways. This study advances an emerging frontier in synthetic biology that aims to build broad-host-range devices and understand the context by which different species can execute programmable genetic operations.

59 BASIC BIOLOGICAL SCIENCES↗

DOE OSTI2025

PETSc/TAO developments for GPU-based early exascale systems

The Portable Extensible Toolkit for Scientific Computation (PETSc) library provides scalable solvers for nonlinear time-dependent differential and algebraic equations and for numerical optimization via the Toolkit for Advanced Optimization (TAO). PETSc is used in dozens of scientific fields and is an important building block for many simulation codes. During the U.S. Department of Energy’s Exascale Computing Project, the PETSc team has made substantial efforts to enable efficient utilization of the massive fine-grain parallelism present within exascale compute nodes and to enable performance portability across exascale architectures. We recap some of the challenges that designers of numerical libraries face in such an endeavor, and then discuss the many developments we have made, which include the addition of new GPU backends, features supporting efficient on-device matrix assembly, better support for asynchronicity and GPU kernel concurrency, and new communication infrastructure. In conclusion, we evaluate the performance of these developments on some pre-exascale systems as well as the early exascale systems Frontier and Aurora, using compute kernel, communication layer, solver, and mini-application benchmark studies, and then close with a few observations drawn from our experiences on the tension between portable performance and other goals of numerical libraries.

Exascale Computing Project (ECP)↗

DOE OSTI2025

CRADA Number NFE-22-09375 with Vitriform3D, Inc. (CRADA Final Report)

Each year, roughly 8 million tons of glass are landfilled in the United States. Vitriform3D, in collaboration with Oak Ridge National Laboratory through the Innovation Crossroads program, is developing an additive manufacturing approach to transform post-consumer glass into engineered stone building materials. A custom UV-curable binder jetting system was commissioned to enable recycled glass printing without energy-intensive oven curing. While this printer is still under development, the team pursued a parallel research track using ORNL equipment and a polyethyleneimine (PEI) binder. Printed parts were thermally post-cured and infiltrated with epoxy, with the resulting material exceeding fiber cement board performance. Silane treatment of the glass particles further improved glass–epoxy adhesion, yielding an additional 24% strength gain over the epoxy-glass composite, achieving nearly 3x the fiber cement benchmark. Ongoing research is focused on reducing epoxy content to improve cost and recycled content while retaining mechanical performance. This work demonstrates a scalable, low-carbon pathway to convert glass waste into high-value, locally produced construction materials.

36 MATERIALS SCIENCE↗

DOE OSTI2025

ORBIT-2 Dataset for Scaling Exascale Vision Foundation Models for Weather and Climate Downscaling

This dataset release corresponds to the work conducted in ORBIT-2: Scaling Exascale Vision Foundation Models for Weather and Climate Downscaling, where large-scale AI methods were applied to improve climate and weather resolution. The collection integrates four widely used, publicly available datasets: ERA5, PRISM, DAYMET, and IMERG. To prepare the data for ORBIT-2 model training and evaluation, we applied a preprocessing pipeline that generates paired low-resolution and high-resolution samples, enabling supervised downscaling experiments. The transformation from coarse to fine scales was performed using bilinear regridding, consistent with the procedures described in WeatherBench2, a community benchmark for weather and climate AI models. This dataset supports the development and evaluation of foundation models designed for weather and climate downscaling at exascale. Additional details on methodology and applications can be found in Wang et al., ORBIT-2 (arXiv:2505.04802, 2025).

54 ENVIRONMENTAL SCIENCES↗