Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Exascale”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 55 records · Page 3
All sourcesNASA NTRSDOE OSTI
DOE OSTI2024

A Digital Twin Framework for Liquid-cooled Supercomputers as Demonstrated at Exascale

We present ExaDigiT, an open-source framework for developing comprehensive digital twins of liquid-cooled supercomputers. It integrates three main modules: (1) a resource allocator and power simulator, (2) a transient thermo-fluidic cooling model, and (3) an augmented reality model of the supercomputer and central energy plant. The framework enables the study of "what-if" scenarios, system optimizations, and virtual prototyping of future systems. Using Frontier as a case study, we demonstrate the framework's capabilities by replaying six months of system telemetry for systematic verification and validation. Such a comprehensive analysis of a liquid-cooled exascale supercomputer is the first of its kind. ExaDigiT elucidates complex transient cooling system dynamics, runs synthetic or real workloads, and predicts energy losses due to rectification and voltage conversion. Throughout our paper, we present lessons learned to benefit HPC practitioners developing similar digital twins. We envision the digital twin will be a key enabler for sustainable, energy-efficient supercomputing.

Brewer, Wes↗
DOE OSTI2025

Scaling the memory wall using mixed-precision - HPG-MxP on an exascale-class machine

Mixed-precision algorithms have been proposed as a way for scientific computing to benefit from some of the gains seen for AI on recent high performance computing (HPC) platforms. A few applications dominated by dense matrix operations have seen substantial speedups by utilizing low precision formats such as FP16. However, a majority of scientific simulation applications are memory bandwidth limited. Beyond preliminary studies, the practical gain from using mixed-precision algorithms on a given high-performance computing (HPC) system is largely unclear. The High Performance GMRES Mixed Precision (HPG-MxP) benchmark has been proposed to measure the useful performance of a HPC system on sparse matrix-based mixed-precision applications. In this work, we present an implementation of the HPG-MxP benchmark for an exascale system and describe our algorithm enhancements. We show for the first time a speedup of 1.6x using a combination of double- and single-precision keeping the same residual level on modern GPU-based supercomputers.

Kashi, Aditya [ORNL] (ORCID:0000000325893792)↗
DOE OSTI2025

Cosmological Hydrodynamics at Exascale: A Trillion-Particle Leap in Capability

Resolving the most fundamental questions in cosmology requires simulations that match the scale, fidelity, and physical complexity demanded by next-generation sky surveys. To achieve the realism needed for this critical scientific partnership, detailed gas dynamics must be treated self-consistently with gravity for end-to-end modeling of structure formation. Exascale computing enables simulations that span survey-scale volumes while incorporating key astrophysical processes that shape complex cosmic structures. We present results from CRK-HACC, a cosmological hydrodynamics code built for extreme scalability. Using separation-of-scale techniques, GPU-resident tree solvers, in situ analysis pipelines, and multi-tiered I/O, CRK-HACCexecuted Frontier-E: a four trillion particle full-sky simulation, over an order of magnitude larger than previous efforts. The run achieved 513.1 PFLOPs peak performance, processing 46.6 billion particles per second and writing more than 100 PB of data in just over one week of runtime. Frontier-E marks a significant advance in predictive modeling for next-generation cosmological science.

Frontiere, Nicholas [Argonne National Laboratory (↗
DOE OSTI2025

Characterizing the Impact of GPU Power Management on an Exascale System

As GPU-accelerated high-performance computing (HPC) systems approach exascale performance, controlling energy consumption without compromising throughput is essential. Architectures such as the AMD MI250X-based Frontier supercomputer provide runtime mechanisms like frequency and power capping, enabling energy tuning without modifying application code. Although both target energy reduction, they operate via distinct hardware control paths and influence workloads differently. We present a comprehensive evaluation of these strategies on a leadership-class system using diverse HPC proxy applications representative of production workloads. Our study analyzes performance–energy trade-offs across multiple capping levels, node counts (1 and 32), and application profiles. Results show that frequency capping generally achieves higher energy efficiency and scalability, with gains of up to 13.2% without performance loss, while power capping is more effective for single-node runs or bursty GPU utilization. We also provide practical guidelines to help system administrators and users balance energy efficiency and performance in large-scale scientific workloads.

Costa, Mariana [Universidade Federal do Rio Grande↗
DOE OSTI2025

Fine-Grained Application Energy and Power Measurements on the Frontier Exascale System

The increasing complexity and power/energy demands of heterogeneous exascale systems, such as the Frontier supercomputer, present significant challenges for measuring and optimizing power consumption in applications. Current tools either lack the resolution to capture fine-grained power and energy measurements, fail to validate in-band measurements against out-of-band power sensors, or cannot integrate this information with application performance events in a scalable manner. This paper introduces a novel open-source performance toolkit that integrates extended PAPI components with Score-P plugins to enable in-band, fine-grained power and energy measurements, while also supporting validation using power meter measurements for both CPUs and GPUs. One key contribution is the ability to perform millisecond-level power and energy measurements for AMD MI250X GPUs, mapping them to application performance events within a single trace and measurement system that scales. Our toolkit combines coarse-grained measurements from cray_pm counters with high-resolution metrics from rocm_smi and RAPL, converting GPU instantaneous accumulated energy into power to capture both transient and steady-state power behavior, a capability often missed by out-of-band and monitoring tools. By mapping these metrics to specific application regions, developers can identify energy hotspots, address inefficiencies in GPU kernel execution, and validate in-band measurements against external measurements. We demonstrate the effectiveness of this approach through case studies using benchmarks such as GPU rocblas_sgemm, BLIS c_blas_dgemm, and rocHPL, highlighting the variability of the measurements and the impact of transient power spikes on kernel-level efficiency.

Hernandez Mendoza, Oscar [ORNL] (ORCID:00000002538↗
DOE OSTI2024

The ECP SICM project: Managing complex memory hierarchies for exascale applications

The Exascale Computing Project (ECP)’s Simplified Interface to Complex Memories (SICM) effort focuses on developing universal interfaces for discovering, managing, and sharing data across complex memory hierarchies. These facilitate the exploitation of emerging memory technologies and support precise control over their various trade-offs such as high-bandwidth versus low-latency, persistent versus ephemeral, high-capacity versus low-capacity, and near-CPU versus near-GPU. SICM comprises three interrelated components: a low-level interface, a high-level interface, and a persistent-heap interface. The low-level SICM interface is intended for system and run-time developers as well as expert application developers who prefer full control of the memory objects used within their application. The high-level SICM interface builds upon the low-level interface, employing application-level profiling and analysis to optimize data management for complex memory hierarchies. The persistent-heap interface provides applications with a persistent memory allocator that can allocate custom C++ data structures in both block-storage and byte-addressable persistent memories.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2025

Performance analysis and data reduction for exascale scientific workflows

Chimbuko is the first in situ, scalable, workflow-level performance analysis tool for trace-level analysis and visualization of application performance. This tool was developed by the Co-design Center for Online Data Analysis and Reduction and funded by the U.S. Department of Energy’s Exascale Computing Project. We provide a detailed description of Chimbuko’s architecture and illustrate our online and offline visualization with multiple use cases. We also present results for the deployment and scalability of the tool as applied to a high-energy physics workflow running at large scale on the Frontier supercomputer.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2025

Establishing model credibility for process-microstructure-property relationships in additive manufacturing using exascale computing

Additive Manufacturing (AM) of alloys holds significant promise as a disruptive technology in various industries, yet its adoption is often hindered by challenges in achieving consistent part quality. These issues are primarily due to the complex process-microstructure-property (PSP) relationships inherent to AM. Computational models can greatly aid in understanding these relationships, but their widespread impact and adoption has been limited by a lack of validated, open-source, and computationally efficient PSP modeling frameworks and hardware limitations. Here, this study leverages the ExaAM software suite and data from the AMBench-2018 series of laser powder bed fusion (LPBF) benchmark experiments to perform a comprehensive model assessment, including verification, validation, sensitivity analysis, and uncertainty quantification. The RADICAL-EnTK workflow manager was used to perform an ensemble of heat transport, solidification, and mechanical response simulations on the exascale computer Frontier, considering uncertainties in critical model inputs such as laser spot size and nucleation parameters, and consisting of 125 explicit grain structure simulations and 7875 crystal plasticity simulations. For a selected location within the Inconel 625 AMBench-2018 test artifact, sensitivity analysis and uncertainty quantification were performed using the predicted distributions of grain structure and mechanical properties. Qualitative agreement was found between the predicted grain size and texture and the observed AMBench-2018 microstructure, the mean predicted yield stress was within 5% of the experimental measurement mean, and the mean predicted engineering stress at 5% strain was within 10% of the experimental measurement mean. The insights gained from development and validation of the ExaAM PSP modeling framework will help guide future directions for enhancing the credibility and reliability of PSP models in AM, thereby accelerating the adoption of AM technologies in various industries.

Additive manufacturing↗
DOE OSTI2025

Implementation of a turbine farm model into the Energy Exascale Earth System Model for investigation and quantification of global climate impacts

Although there has been widespread deployment of wind farms in the United States, and plans to continue deployment into the future, the complete effects of wind farms on Earth systems are not well understood. The work performed here has incorporated wind farm models into the Energy Exascale Earth System Model (E3SM) capable of simulating the effects of extracting momentum from the atmospheric flow field using power generating wind farms. This new capability will allow scientists to quantify the impacts of wind farm induced changes on Earth systems by exploiting E3SM’s ability to couple atmospheric, oceanic, and biogeochemical (BGC) models on a global scale and monitor precipitation levels, extreme weather events, soil moisture content and jet stream location over decades-long time periods. This tool will be used to inform decision making on wind farm citing and will contribute to the Lab’s ability to assess energy technology impacts on the environment and evaluate the trade-offs between energy infrastructure investments and their impacts on natural systems.

17 WIND ENERGY↗
DOE OSTI2025

Exascale Advancements In Kokkos Core

Summary of Kokokkos (open source) release features relevant to application developers working on exascale systems.

Morales, Nicolas Manuel [Sandia National Laborator↗
DOE OSTI2025

Regime-based aerosol–cloud interactions from CALIPSO-MODIS and the Energy Exascale Earth System Model version 2 (E3SMv2) over the Eastern North Atlantic

This study investigates aerosol-cloud interactions in marine boundary layer (MBL) clouds using an advanced deep-learning-driven synoptic-regime-based framework, combining satellite data (CALIPSO vertically resolved aerosol extinction and MODIS cloud properties) with 1° nudged Energy Exascale Earth System Model version 2 (E3SMv2) simulation over the Eastern North Atlantic (ENA; ∼10°×10°, 2006–2014). The E3SMv2 captures observed seasonal variations in cloud droplet number concentrations (N d ) and liquid water path (LWP), though it systematically underestimates N d . We then partition ENA meteorology into four synoptic regimes (Pre-Trough, Post-Trough, Ridge, Trough) via a deep-learning clustering of ERA5 reanalysis fields, enabling regime-dependent aerosol-cloud interactions analyses. Both satellite and E3SMv2 exhibit an inverted-V LWP-N d relationship. In Post-Trough and Ridge regimes, the satellite shows stronger negative LWP-N d sensitivities than in Pre-Trough regime. The Trough regime displays a muted satellite LWP response. In comparison, the model predicts more exaggerated LWP responses across regimes, with LWP increasing too quickly at low N d and decreasing more sharply at high N d , especially in Pre-Trough and Trough regimes. These exaggerated model LWP sensitivities may stem from uncertainties in representing drizzle processes, entrainment, and turbulent mixing. As for N d susceptibility to aerosols, N d increases with MBL aerosol extinction in both datasets, but the simulated aerosol-cloud interactions appear oversensitive to meteorological conditions. Overall, E3SMv2 better captures aerosol effects under regimes that favor stratiform clouds (Post-Trough, Ridge), but performance deteriorates for regimes with deeper, dynamically complex clouds (Trough), highlighting the need for improved representations of those cloud processes in climate models.

Environmental sciences↗
DOE OSTI2025

Subsurface hydrological controls on the short-term effects of hurricanes on nitrate–nitrogen runoff loading: a case study of Hurricane Ida using the Energy Exascale Earth System Model (E3SM) Land Model (v2.1)

When the nutrient level in the soil surpasses vegetation demand, nutrient losses due to surface runoff and subsurface leaching are the major reasons for the deterioration of water quality. The lower Mississippi River basin (LMRB) is one of the sub-basins that deliver the highest nitrogen loads to the Gulf of Mexico. Potential changes in episodic events induced by hurricanes may exacerbate water quality issue in the future. However, uncertainties in modeling the hydrologic response to hurricanes may limit the modeling of nutrient losses during such events. Using a machine learning approach, we calibrated the land component of the Energy Exascale Earth System Model (E3SM), or ELM, version 2.1, based on the water table depth (WTD) of a calibrated 3D subsurface hydrology model. While the overall performance of the calibrated ELM is satisfactory, some discrepancies in WTD remain in slope areas with low precipitation due to the missing lateral flow process in ELM. Simulations including biogeochemistry performed using ELM with and without model calibration showed important influences of soil hydrology, precipitation intensity, and runoff parameterization on the magnitude of nitrogen runoff loss and the leaching pathway. Despite such sensitivities, both ELM simulations produced reduced WTD and increased runoff and accelerated nitrate–nitrogen runoff loading during Hurricane Ida in August 2021, consistent with the observations. With observations suggesting more pronounced effects of Hurricane Ida on nitrogen runoff than the simulations, we identified factors for model improvement to provide a useful tool for studying hurricane-induced nutrient losses in the LMRB region.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2025

Asynchronous-many-task systems: Challenges and opportunities - Scaling an AMR astrophysics code on exascale machines using Kokkos and HPX

Dynamic and adaptive mesh refinement is pivotal in high-resolution, multi-physics, multi-model simulations, necessitating precise physics resolution in localized areas across expansive domains. Today’s supercomputers’ extreme heterogeneity presents a significant challenge for dynamically adaptive codes, highlighting the importance of achieving performance portability at scale. Our research focuses on astrophysical simulations, particularly stellar mergers, to elucidate early universe dynamics. Here, we present Octo-Tiger, leveraging Kokkos, HPX, and SIMD for portable performance at scale in complex, massively parallel adaptive multi-physics simulations. Octo-Tiger supports diverse processors, accelerators, and network backends. Experiments demonstrate exceptional scalability across several heterogeneous supercomputers including Perlmutter, Frontier, and Fugaku, encompassing major GPU architectures and x86, ARM, and RISC-V CPUs. Parallel efficiency of 47.59% (110,080 cores and 6880 hybrid A100 GPUs) on a full-system run on Perlmutter (26% HPCG peak performance) and 51.37% (using 32,768 cores and 2048 MI250X) on Frontier are achieved.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2025

Small-scale properties from exascale computations of turbulence on a $\mathbf{32\,768^3}$ periodic cube

To study the physics of small-scale properties of homogeneous isotropic turbulence at increasingly high Reynolds numbers, direct numerical simulation results have been obtained for forced isotropic turbulence at Taylor-scale Reynolds number R λ = 2500 on a 32 768 3 three-dimensional periodic domain using a GPU pseudo-spectral code on a 1.1 exaflop GPU supercomputer (Frontier). These simulations employ the multi-resolution independent simulation (MRIS) technique (Yeung & Ravikumar 2020, Phys. Rev. Fluids, vol. 5, 110517) where ensemble averaging is performed over multiple short segments initiated from velocity fields at modest resolution, and subsequently taken to higher resolution in both space and time. Reynolds numbers are increased by reducing the viscosity with the large-scale forcing parameters unchanged. Although MRIS segments at the highest resolution for each Reynolds number last for only a few Kolmogorov time scales, small-scale physics in the dissipation range is well captured – for instance, in the probability density functions and higher moments of the dissipation rate and enstrophy density, which appear to show monotonic trends persisting well beyond the Reynolds number range in prior works in the literature. Attainment of range of length and time scales consistent with classical scaling also reinforces the potential utility of the present high-resolution data for studies of short-time-scale turbulence physics at high Reynolds numbers where full-length simulations spanning many large-eddy time scales are still not accessible. A single snapshot of the 32 768 3 data is publicly available for further analyses via the Johns Hopkins Turbulence Database.

intermittency↗
DOE OSTI2024

Towards Sustainable Post-Exascale Leadership Computing

As computing systems approach the limits of traditional silicon technology, the diminishing returns in performance per watt present a significant barrier to sustaining growth in HPC. From a large-scale scientific supercomputing facility point of view, we propose a multifaceted strategy toward specialized hardware and architectures that are optimized for energy efficiency in specific applications. We also emphasize the need for integrating energy-aware practices across all levels of HPC, from system design and software development to operational policies. We discuss strategic opportunities such as the adoption of application-specific accelerators, the development of energy-efficient algorithms, and the implementation of data-driven operational analytics. Our goal is to develop a comprehensive roadmap ensuring that future leadership systems at OLCF can meet scientific demands while operating within stringent energy budgets, thereby supporting sustainable computing growth.

Shin, Woong↗
DOE OSTI2024

Navigating Exascale Operational Data Analytics: From Inundation to Insight

In this paper, we address the challenges in achieving sustainable data-driven efficiency by providing a detailed exploration of the end-to-end operational data analytics (ODA) framework that evolved through two generations of supercomputer systems at the Oak Ridge Leadership Computing Facility (OLCF). This framework addresses large data streams ingested from heavily instrumented HPC environment that accumulates multi-terabytes per day. We outline the multifaceted data life cycle across HPC procurement, operations, and research & development, identifying key obstacles and design decisions that shape effective strategies in building and supporting data pipelines end-to-end. By sharing key insights and lessons learned from our experience, we offer recommendations for the HPC community on enabling sustainable operational data analytics and beyond. Our contributions aim to bridge the gap between potential and real benefits of operational data, guiding future efforts towards integrated and sustainable operational intelligence in high-performance computing environments.

Shin, Woong↗