Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “GPU acceleration”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 451 records · Page 25

Preparing the HARVEY Circulatory Modeling Code for Exascale

Atherosclerotic cardiovascular disease is the leading global cause of death, accounting for more than 17.3 million deaths per year, a number that is expected to grow to more than 23.6 million by 2030. Personalized computational models of the coronary arteries can be used to analyze the influence that a narrowing of the blood vessel can have on the associated hemodynamics. To achieve the necessary scale and precision in circulatory models with explicit fluid coupled to deformable cell models, our simulation code HARVEY must be designed to efficiently use current and next-generation HPC architectures. This Feasibility Study will involve a careful assessment and redesign of the code’s core data structure to prepare for GPU-accelerated systems like Sierra as well as next-generation architectures like the A21 exascale system.

59 BASIC BIOLOGICAL SCIENCES↗

TChem v2.0 - A Software Toolkit for the Analysis of Complex Kinetic Models

TChem is an open source software library for solving complex computational chemistry problems and analyzing detailed chemical kinetic models. The software provides support for: complex kinetic models for gas-phase and surface chemistry; thermodynamic properties based on NASA polynomials; species production/consumption rates; stable time integrator for solving stiff time ordinary differential equations; and, reactor models such as homogenous gas-phase ignition (with analytical Jacobian matrices), continuously stirred tank reactor, plug-flow reactor. This toolkit builds upon earlier versions that were written in C and featured tools for gas-phase chemistry only. The current version of the software was completely refactored in C++, uses an object-oriented programming model, and adopts Kokkos as its portability layer to make it ready for the next generation computing architectures i.e., multi/many core computing platforms with GPU accelerators. We have expanded the range of kinetic models to include surface chemistry and have added examples pertaining to Continuously Stirred Tank Reactors (CSTR) and Plug Flow Reactor (PFR) models to complement the homogenous ignition examples present in the earlier versions. To exploit the massive parallelism available from modern computing platforms, the current software interface is designed to evaluate samples in parallel, which enables large scale parametric studies, e.g. for sensitivity analysis and model calibration.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

A Hybrid Climate Modeling System Using AI-assisted Process Emulators

This white paper addresses Focus Area II. We advocate developing a hybrid modeling system to improve the understanding of decadal- and longer-scale predictability of high impact water cycle components. This hybrid model combines a partial differential equation (PDE)-based dynamic core with AI/ML based emulators to represent many of the computationally expensive processes in Earth’s climate models. The hybrid modeling system has the potential to exploit emerging graphics processing unit (GPU)-accelerated architectures and allows for the generation of large ensemble (~1000’s) simulations to better characterize the model uncertainty and understand predictability.

58 GEOSCIENCES↗

Physics Basis, Optimization, and Control for Integrated 3D Edge Long-pulse Tokamak Scenarios

The aim of this project with international collaborations (PI: J.-K. Park of Princeton Plasma Physics Laboratory) is to develop a unified physics basis and predictive capability for the control of edge-localized modes (ELMs) with optimized non-axisymmetric (3D) fields, by leveraging the unique research capabilities of international KSTAR tokamak in Korea. At the University of California, Irvine, we use a fusion simulation code GTC to study the effects of the 3D magnetic fields (including both magnetic islands and stochastic magnetic field lines) on microturbulence. We focus on the effects of 3D equilibrium on turbulent and neoclassical transport that leads to ELM suppression. During the project period, the UCI team of this collaboration has made significant progress in understanding turbulent transport in 3D geometry including both tokamaks with RMP and stellarators. Four UCI postdoctoral researchers, Drs. Javier Nicolau, Gyungjin Choi, Pengfei Liu, and Xishuo Wei were partially supported by this project and collaborating with the entire project team. We have published 4 journal papers and gave several invited talks. This project advance physics understand of 3D effects on microturbulence, specially effects of magnetic islands. It will contribute to the DOE SciDAC ISEP project in the area of GTC development for general 3D geometry capability, rigorous verification and validation, and readiness for the emerging exascale computing. Through collaborations with computational scientists, the project helps to advance high performance computing in the area of GPU acceleration, linear solver, and data management.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗

Center for Tokamak Transient Simulations (RPI Unstructured Mesh Developments for FES SciDAC4 Partnerships) (Final Report)

The goal of this project was the development of unstructured mesh technologies for fusion simulation codes” for FES SciDAC partnerships and to integrate those technologies into the simulation workflows of those partnerships. Specific developments were executed in support of the following FES SciDAC4 partnerships: Partnership Center for High‐fidelity Boundary Plasma Simulation (HBPS), Center for Integrated Simulation of Fusion Relevant RF (RF‐SciDAC), Center for Plasma Surface Interactions: Predicting the Performance and Impact of Dynamic PFC Surfaces (PSI2), and Center for Tokamak Transient Simulations (CTTS). The key unstructured mesh development areas addressed in this project include (i) methods to most effectively perform PIC calculations on unstructured meshes; (ii) creating meshes for fusion systems accounting for any desired level of geometric complexity and providing physics aware mesh configurations, (iii) adapting unstructured meshes to control the discretization errors, (iv) executing unstructured mesh calculations on GPU accelerated systems, (v) supporting physics/application‐specific PIC operations including surface/wall interactions of particles, (vi) providing infrastructure tools to support the interactions of solvers with unstructured meshes, and (vii) providing advanced methods for coupling plasma physics codes.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗

E3SM: Improved Climate Prediction with Exascale Capability

The Energy Exascale Earth System Model (E3SM) project is an ongoing, state-of-the-science earth system modeling, simulation, and prediction effort that optimizes Department of Energy (DOE) computing resources to meet the science needs of the nation and the agency’s mission objectives. Climate simulation has become a proven tool for identifying and quantifying the impacts of climate change, but even greater accuracy is required at all levels to improve forecast precision. Understanding the impact of climate change on global and regional water cycles is one of the highest priorities and most difficult challenges in climate change prediction. As part of a subproject of DOE’s Exascale Computing Project, a multidisciplinary team including geophysical and computational scientists developed a multiscale modeling framework (MMF) to refine cloud representation in E3SM climate simulation on GPU accelerated supercomputers, making higher resolution, more computationally efficient predictions possible.

54 ENVIRONMENTAL SCIENCES↗

LLNL FESP Theory Highlights: August 2024

The 2024 ABOUND SciDAC and BOUT++ combined workshop was held August 5-9 th 2024 at the University of California Livermore Collaboration Center (UCLC) in Livermore. Bringing together leading scientists and researchers from across the globe, this pivotal event focused on advancing plasma physics and boundary plasma dynamics within the context of fusion energy research. Key discussions throughout the meeting highlighted significant advancements in the BOUT++ framework, including enhanced simulations of small Edge Localized Modes (ELMs) and the initiation of integrating the integration of the 5D GEM gyrokinetic turbulence core code with the 2D SOLPS-ITER boundary transport code. These developments are crucial for managing heat loads in fusion reactors and supporting the longevity of plasma-facing components. The event also featured a session on Inter-SciDAC Collaborations, where principal investigators from multiple U.S. FES SciDAC tokamak projects explored opportunities for cross-collaboration. Additionally, the meeting showcased cutting-edge advancements in GPU acceleration and AI/ML technologies, poised to drive the next generation of fusion research. In his closing remarks, Dr. Xueqiao Xu emphasized the importance of the collaborative efforts and discussions that took place, noting their potential to shape future breakthroughs in fusion energy. The event underscored the global nature of the BOUT++ collaboration, with contributions from over 57 institutions worldwide. The 2024 BOUT++ and ABOUND Joint Hybrid Meeting continues to drive forward the research and innovations needed to achieve fusion energy, setting the stage for future collaboration and discovery.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗

A Block-Structured Adaptive Mesh Framework to Solve Radiation Transfer Equation in Irregular Embedded Geometries

Radiation transport arises in various scientific, industrial, and medical fields, and understanding its effect in applications is needed to make accurate predictions, safety assessments and performance optimizations. Solving the Radiation Transport Equation (RTE) is challenging due to its integro-differential nature, which involves both differential and integral terms. The differential term describes the change in radiation intensity due to absorption and emission, while the integral term accounts for scattering. The accurate modeling of radiation is further complicated in many applications due to the complex, irregular geometries. Various methods exist for solving the RTE, including the zonal, Monte Carlo, spherical harmonics, discrete ordinates, and finite volume methods. Traditional mesh-based approaches, which rely on structured or unstructured meshes, struggle with irregular geometries due to: a) the difficulty of conforming structured grids to irregular domains, b) challenges in enforcing boundary conditions correctly, and c) the additional computational cost of unstructured mesh methods. This work presents a second-order accurate method for solving the RTE in irregular geometries. The radiation intensity is discretized using the finite-volume method in both spatial and angular directions on regular Cartesian grid blocks. Leveraging the block-structured adaptive mesh refinement (AMR) framework provided by AMReX, our method refines the grid locally to reduce spatial discretization error, ensuring a converged numerical solution while minimizing computational costs elsewhere. A two-stage deferred correction approach is employed: First, a first-order discretization on grid blocks is solved using an algebraic multigrid method in HYPRE. Second, a correction term is applied explicitly to achieve second-order accuracy. The correction term is calculated by approximating the radiation flux on cell faces using a Total Variation Diminishing (TVD) scheme. This approach ensures quick convergence of the multigrid method while preserving higher-order accuracy of the numerical solution. Irregular geometries are resolved as embedded boundaries (EB), resulting in both cut cells and regular cells. In cut cells, we modify the fluxes using face fractions and incorporate additional contributions from EB boundary conditions. To ensure higher-order convergence near the EB interface, the correction term is modified by interpolating the radiation intensity to fictitious ghost points. The implementation takes advantage of modern supercomputers by leveraging AMReX’sMPI/X parallelization strategy where X can be MPI or a GPU accelerator including CUDA, HIP and DPC++. We validate our solver using classical test cases, both with and without EB, demonstrating accuracy and efficiency. Additionally, we analyze the impact of adaptive mesh refinement on solution accuracy and computational cost, highlighting the advantages of our approach for high-resolution radiation transport simulations.

computational fluid dynamics (CFD)↗

Challenge Problem 1: Preliminary Results of the Direct Numerical Simulation of Transient Flows

This report presents the first direct numerical simulations (DNS) of transient mixed convection in an idealized downcomer-like channel (Challenge Problem 1, Phase II). Using the GPU-accelerated NekRS solver, we modeled a sudden decay in driving pressure, mimicking loss-of-flow events, and tracked the resulting evolution of Reynolds number, boundary-layer structure, turbulence statistics, and heat-transfer metrics. Key findings include the systematic thickening and eventual asymmetry of velocity and thermal boundary layers under buoyant deceleration; minimal “memory” lag in Reynolds shear stress and TKE profiles when sampled at matching Re, yet clear shifts of peak locations toward the cooled wall; overshoots in transient eddy-viscosity and eddy-diffusivity (and corresponding sub-unity turbulent Prandtl numbers) on the cooled side; and a pronounced transient Nusselt-number enhancement driven by wall-temperature inertia and residual eddy mixing. These effects combined to offer a temporary cooling margin above steady-state predictions during reactor LOF transients. Future work will extend this work to a more complex “Case II” geometry (90° turn + lower plenum) and generate multi-Re/Pr datasets for data-driven turbulence closures.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗

Hybrid learning techniques for scientific data reduction with performance guarantees

The research initiatives supported by the U.S. Department of Energy (DOE) Grant DE-SC0022265 are fundamentally aimed at pioneering advanced machine learning (ML) techniques for scientific data compression within high-performance computing (HPC) environments. This comprehensive body of work addresses the critical challenge posed by the exponential growth of data generated by scientific simulations in domains such as fusion energy, climate modeling, and computational fluid dynamics (CFD). A core objective is to develop compression algorithms that achieve substantial data reduction—often by orders of magnitude—while rigorously ensuring the fidelity of both the primary data (PD) and scientifically crucial derived quantities of interest (QoI). The methodologies deployed under this grant integrate sophisticated deep learning architectures, prominently featuring autoencoders, advanced generative models like conditional diffusion, and hybrid learning techniques. Key innovations include the development of Guaranteed Autoencoders (GAE) and the Guaranteed Conditional Diffusion with Tensor Correction (GCDTC) framework, which provide explicit, instance-level error bounds on reconstructed data. Furthermore, specialized strategies such as nonlinear constraint satisfaction are employed to preserve the integrity of QoI, a vital requirement for the trustworthiness of downstream scientific analyses. This research also focuses on the design and implementation of scalable, GPU-accelerated software pipelines that seamlessly integrate into existing HPC workflows, ensuring both computational efficiency and practical applicability. The CAESAR framework, for example, unifies foundation and generative models to create an adaptive and efficient compression solution for spatio-temporal scientific data. Collectively, these efforts represent a significant advancement in mitigating the scientific data deluge, enabling more effective data management, accelerated scientific discovery, and optimized utilization of HPC resources.

97 MATHEMATICS AND COMPUTING↗

Final report- UFL - RAPIDS2: A SciDAC Institute for Computer Science, Data, and Artificial Intelligence

The research initiatives supported by the U.S. Department of Energy (DOE) Grant DE-SC0022265 are fundamentally aimed at pioneering advanced machine learning (ML) techniques for scientific data compression within high-performance computing (HPC) environments. This comprehensive body of work addresses the critical challenge posed by the exponential growth of data generated by scientific simulations in domains such as fusion energy, climate modeling, and computational fluid dynamics (CFD). A core objective is to develop compression algorithms that achieve substantial data reduction—often by orders of magnitude—while rigorously ensuring the fidelity of both the primary data (PD) and scientifically crucial derived quantities of interest (QoI). The methodologies deployed under this grant integrate sophisticated deep learning architectures, prominently featuring autoencoders, advanced generative models like conditional diffusion, and hybrid learning techniques. Key innovations include the development of Guaranteed Autoencoders (GAE) and the Guaranteed Conditional Diffusion with Tensor Correction (GCDTC) framework, which provide explicit, instance-level error bounds on reconstructed data. Furthermore, specialized strategies such as nonlinear constraint satisfaction are employed to preserve the integrity of QoI, a vital requirement for the trustworthiness of downstream scientific analyses. This research also focuses on the design and implementation of scalable, GPU-accelerated software pipelines that seamlessly integrate into existing HPC workflows, ensuring both computational efficiency and practical applicability. The CAESAR framework, for example, unifies foundation and generative models to create an adaptive and efficient compression solution for spatio-temporal scientific data. Collectively, these efforts represent a significant advancement in mitigating the scientific data deluge, enabling more effective data management, accelerated scientific discovery, and optimized utilization of HPC resources.

97 MATHEMATICS AND COMPUTING↗

GASNet-EX Memory Kinds: Support for Device Memory in PGAS Programming Models

There is an emerging need for adaptive, lightweight communication in irregular HPC applications at exascale, where GPU accelerators provide the majority of available compute cycles. To address this need, Lawrence Berkeley National Lab is developing a programming system to support distributed-memory HPC application development using the Partitioned Global Address Space (PGAS) model. This work includes two major components: UPC++ and GASNet-EX. UPC++ is a C++ template library providing Remote Memory Access (RMA) and Remote Procedure Call (RPC) communication interfaces. GASNet-EX is a portable, high-performance communication middleware library, used by the implementations of UPC++ and many other PGAS programming models. We describe recent advances in GASNet-EX to efficiently implement zero-copy Remote Memory Access (RMA) communication to and from memory on accelerator devices such as GPUs. We demonstrate performance improvements via benchmark results from UPC++ (on Summit) and the Legion programming system (on DGX-1), both using GASNet-EX for communication.

Hargrove, Paul H↗

Sensitivity of Simulations of Double-detonation Type Ia Supernovae to Integration Methodology

Abstract We study the coupling of hydrodynamics and reactions in simulations of the double-detonation model for Type Ia supernovae. When assessing the convergence of simulations, the focus is usually on spatial resolution; however, the method of coupling the physics together as well as the tolerances used in integrating a reaction network also play an important role. In this paper, we explore how the choices made in both coupling and integrating the reaction portion of a simulation (operator/Strang splitting versus the simplified spectral deferred corrections method we introduced previously) influences the accuracy, efficiency, and nucleosynthesis of simulations of double detonations. We find no need to limit reaction rates or reduce the simulation time step to the reaction timescale. The entire simulation methodology used here is GPU-accelerated and made freely available as part of the Castro simulation code.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗

PCMS: Parallel Coupler For Multimodel Simulations

This paper presents the Parallel Coupler for Multimodel Simulations (PCMS), a new GPU accelerated generalized coupling framework for coupling simulation codes on leadership class supercomputers. PCMS includes distributed control and field mapping methods for up to five dimensions. For field mapping PCMS can utilize discretization and field information to accommodate physics constraints. PCMS is demonstrated with a coupling of the gyrokinetic microturbulence code XGC with a Monte Carlo neutral transport code DEGAS2 and with a 5D distribution function coupling of an energetic particle transport code (GNET) to a gyrokinetic microturbulence code (GTC). Weak scaling is also demonstrated on up to 2,080 GPUs of Frontier with a weak scaling efficiency of 85%.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗

Towards multiscale modeling of ocean surface turbulent mixing using coupled MPAS-Ocean v6.3 and PALM v5.0

A multiscale modeling approach for studying the ocean surface turbulent mixing is explored by coupling an ocean general circulation model (GCM) MPAS-Ocean with the Parallelized Large Eddy Simulation Model (PALM). The coupling approach is similar to the superparameterization approach that has been used to represent the effects of deep convection in atmospheric GCMs. However, the focus of this multiscale modeling approach is on the small-scale turbulent mixing and their interactions with the larger-scale processes in the ocean, so that a more flexible coupling strategy is used. To reduce the computational cost, a customized version of PALM is ported on the general-purpose graphics processing unit (GPU) with OpenACC, achieving 10–16 times overall speedup as compared to running on a single CPU. Even with the GPU-acceleration technique, a superparameterization-like approach to represent the ocean surface turbulent mixing in GCMs using embedded high fidelity and three-dimensional large eddy simulations (LESs) over the global ocean is still computationally intensive and infeasible for long simulations. However, running PALM regionally on selected MPAS-Ocean grid cells is shown to be a promising approach moving forward. The flexible coupling between MPAS-Ocean and PALM allows further exploration of the interactions between the ocean surface turbulent mixing and larger-scale processes, as well as future development and improvement of ocean surface turbulent mixing parameterizations for GCMs.

54 ENVIRONMENTAL SCIENCES↗

Importance of ice nucleation and precipitation on climate with the Parameterization of Unified Microphysics Across Scales version 1 (PUMASv1)

Cloud microphysics is critical for weather and climate prediction. In this work, we document updates and corrections to the cloud microphysical scheme used in the Community Earth System Model (CESM) and other models. These updates include a new nomenclature for the scheme, now called Parameterization of Unified Microphysics Across Scales (PUMAS), and the ability to run the scheme on graphics processing units (GPUs). The main science changes include refactoring an ice number limiter and associated changes to ice nucleation, adding vapor deposition onto snow, and introducing an implicit numerical treatment for sedimentation. We also detail the improvements in computational performance that can be achieved with GPU acceleration. We then show the impact of these scheme changes on the (a) mean state climate, (b) cloud feedback response to warming, and (c) aerosol forcing. We find that corrections are needed to the immersion freezing parameterization and that ice nucleation has important impacts on climate. We also find that the revised scheme produces less cloud liquid and ice but that this can be adjusted by changing the loss process for cloud liquid (autoconversion). Furthermore, there are few discernible effects of the PUMAS changes on cloud feedbacks but some reductions in the magnitude of aerosol–cloud interactions (ACIs). Small cloud feedback changes appear to be related to the implicit sedimentation scheme, with a number of factors affecting ACIs.

54 ENVIRONMENTAL SCIENCES↗

Comparing the Performance of Julia on CPUs versus GPUs and Julia-MPI versus Fortran-MPI: a case study with MPAS-Ocean (Version 7.1)

Abstract. Some programming languages are easy to develop at the cost of slow execution, while others are fast at runtime but much more difficult to write. Julia is a programming language that aims to be the best of both worlds – a development and production language at the same time. To test Julia's utility in scientific high-performance computing (HPC), we built an unstructured-mesh shallow water model in Julia and compared it against an established Fortran-MPI ocean model, the Model for Prediction Across Scales–Ocean (MPAS-Ocean), as well as a Python shallow water code. Three versions of the Julia shallow water code were created: for single-core CPU, graphics processing unit (GPU), and Message Passing Interface (MPI) CPU clusters. Comparing identical simulations revealed that our first version of the Julia model was 13 times faster than Python using NumPy, where both used an unthreaded single-core CPU. Further Julia optimizations, including static typing and removing implicit memory allocations, provided an additional 10–20× speed-up of the single-core CPU Julia model. The GPU-accelerated Julia code was almost identical in terms of performance to the MPI parallelized code on 64 processes, an unexpected result for such different architectures. Parallelized Julia-MPI performance was identical to Fortran-MPI MPAS-Ocean for low processor counts and ranges from 2× faster to 2× slower for higher processor counts. Our experience is that Julia development is fast and convenient for prototyping but that Julia requires further investment and expertise to be competitive with compiled codes. We provide advice on Julia code optimization for HPC systems.

54 ENVIRONMENTAL SCIENCES↗

Zooming in: SCREAM at 100 m using regional refinement over the San Francisco Bay Area

Pushing global climate models to large-eddy simulation (LES) scales over complex terrain has remained a major challenge. This study presents the first known implementation of a global model – SCREAM (Simple Cloud-Resolving E3SM Atmosphere Model) – at 100 m horizontal resolution using a regionally refined mesh (RRM) over the San Francisco Bay Area. Two hindcast simulations were conducted to test performance under both strong synoptic forcing and weak, boundary-layer-driven conditions. We demonstrate that SCREAM can stably run at LES scales while realistically capturing topography, surface heterogeneity, and coastal processes. The 100 m SCREAM-RRM substantially improves near-surface wind speed, temperature, humidity, and pressure biases compared to the baseline 3.25 km simulation, and better reproduces fine-scale wind oscillations and boundary-layer structures. These advances leverage SCREAM's scale-aware SHOC turbulence parameterization, which transitions smoothly across scales without tuning. Performance tests show that while CPU-only simulations remain costly, GPU acceleration with SCREAMv1 on NERSC's Perlmutter system enables two-day hindcasts to complete in under two wall-clock days. Our results open the door to LES-scale studies of orographic flows, boundary-layer turbulence, and coastal clouds within a fully comprehensive global modeling framework.

Geosciences↗