Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “supercomputing”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 163 records · Page 9
All sourcesNASA NTRSDOE OSTI
DOE OSTI2022

Analysis of Vector Particle-In-Cell (VPIC) memory usage optimizations on cutting-edge computer architectures

Vector Particle-In-Cell (VPIC) is one of the fastest plasma simulation codes in the world, with particle numbers ranging from one trillion on the first petascale system, Roadrunner, to ten trillion particles on the more recent Blue Waters supercomputer. As supercomputers continue to grow rapidly in size, so too does the gap between computing capability and memory capability. Current memory systems limit VPIC simulations greatly as the maximum number of particles that can be simulated directly depends on the available memory. In this study, we present a suite of VPIC memory optimizations (i.e., particle weight, half-precision, and fixed-point optimizations) that enable a significant increase in the number of particles in VPIC simulations. Here, we assess the optimizations’ impact on memory and runtime performance for a suite of cutting-edge computer architectures such has the NVIDIA V100 GPU, the IBM Power9, and the Fujitsu A64FX architectures. Our optimizations enable a 31.25% reduction in memory usage and up to 40% increase in the number of particles. This paper extends our work on developing particle storage format optimizations Tan et al.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2021

OpenMP application experiences: Porting to accelerated nodes

As recent enhancements to the OpenMP specification become available in its implementations, there is a need to share the results of experimentation in order to better understand the OpenMP implementation’s behavior in practice, to identify pitfalls, and to learn how the implementations can be effectively deployed in scientific codes. We report on experiences gained and practices adopted when using OpenMP to port a variety of ECP applications, mini-apps and libraries based on different computational motifs to accelerator-based leadership-class high-performance supercomputer systems at the United States Department of Energy. Additionally, we identify important challenges and open problems related to the deployment of OpenMP. Through our report of experiences, we find that OpenMP implementations are successful on current supercomputing platforms and that OpenMP is a promising programming model to use for applications to be run on emerging and future platforms with accelerated nodes.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2023

Characterizing the performance of node-aware strategies for irregular point-to-point communication on heterogeneous architectures

Supercomputer architectures are trending toward higher computational throughput due to the inclusion of heterogeneous compute nodes. These multi-GPU nodes increase on-node computational efficiency, while also increasing the amount of data to be communicated and the number of potential data flow paths. In this work, we characterize the performance of irregular point-to-point communication with MPI on heterogeneous compute environments through performance modeling, demonstrating the limitations of standard communication strategies for both device-aware and staging-through-host communication techniques. Presented models suggest staging communicated data through host processes then using node-aware communication strategies for high inter-node message counts. Notably, the models also predict that node-aware communication utilizing all available CPU cores to communicate inter-node data leads to the most performant strategy when communicating with a high number of nodes. Furthermore, model validation is provided via a case study of irregular point-to-point communication patterns in distributed sparse matrix–vector products. Importantly, we include a discussion on the implications model predictions have on communication strategy design for emerging supercomputer architectures.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2026

Uncertain quantum computing futures and potential energy and physical resource impacts at scale

Considerable attention has recently focused on the vast energy and water demands of supercomputing, namely large-scale data centers that underpin artificial intelligence (AI), one of the great disruptors of contemporary society. Looking ahead some years from now, quantum computing is poised to disrupt established computing paradigms once again. Scientists and engineers are now working intensely to bring this century-old dream of physicists to fruition. Yet, as quantum computers begin to be integrated with classical supercomputing architectures, the implications for energy and physical resource use also need to be understood, especially how they compare to today’s AI data centers. These impacts have not yet been quantified by the research community – a notable gap in the literature, even if commercial-scale deployment of Quantum-Accelerated Computing Infrastructure (QuACI) is not expected for a few more years. This study is the first to conduct such an assessment. Using publicly available information from academic sources and private industry, we characterize multiple configurations of superconducting qubit-based, fault-tolerant quantum computers (FTQC) that could plausibly be deployed at scale in the 2030s and into the 2040s. By parameterizing these FTQC systems at a process level, we conduct a prospective scenario analysis to quantify their energy and physical resource needs. While these estimates are uncertain, given the current state of quantum technologies and their unknown future trajectories, important insights can already be drawn. One key finding is that while the electricity needs for a fleet of FTQCs are within the bounds of previous modeling studies that have explored high electricity demand futures, the needs for certain physical resources, namely water and helium-3, could pose bottlenecks to QuACI scale-up.

Computing↗
DOE OSTI2025

Acceleration of the particle-in-cell code Osiris with graphics processing units

Fully relativistic particle-in-cell (PIC) simulations are crucial for advancing our knowledge of plasma physics. Modern supercomputers based on graphics processing units (GPUs) offer the potential to perform PIC simulations of unprecedented scale, but require robust and feature-rich codes that can fully leverage their computational resources. In this work, this demand is addressed by adding GPU acceleration to the PIC code Osiris. An overview of the algorithm, which features a CUDA extension to the underlying Fortran architecture, is given. Detailed performance benchmarks for thermal plasmas are presented, which demonstrate excellent weak scaling on NERSC's Perlmutter supercomputer and high levels of absolute performance. The robustness of the code to model a variety of physical systems is demonstrated via simulations of Weibel filamentation and laser-wakefield acceleration run with dynamic load balancing. Finally, measurements and analysis of energy consumption are provided that indicate that the GPU algorithm is up to ~14 times faster and ~7 times more energy efficient than the optimized CPU algorithm on a node-to-node basis. The described development addresses the PIC simulation community's computational demands both by contributing a robust and performant GPU-accelerated PIC code and by providing insight into efficient use of GPU hardware.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗
DOE OSTI2022

GronOR: Scalable and Accelerated Nonorthogonal Configuration Interaction for Molecular Fragment Wave Functions

GronOR is a program package for nonorthogonal configuration interaction calculations. Electronic wave functions are constructed in terms of antisymmetrized products of multiconfiguration molecular fragment wave functions. The computational complexity of the nonorthogonal methodologies implemented in GronOR applied to large molecular assemblies requires a design that takes full advantage of massively parallel supercomputer architectures and accelerator technologies. This work describes the implementation strategy and resulting performance characteristics. In addition to parallelization and acceleration, the software development strategy includes aspects of fault resiliency and heterogeneous computing. The program was designed for large-scale supercomputers but also runs effectively on small clusters and workstations for small molecular systems. GronOR is available as open source to the scientific community.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2024

Static Subspace Approximation for Random Phase Approximation Correlation Energies: Implementation and Performance

Developing theoretical understanding of complex reactions and processes at interfaces requires using methods that go beyond semilocal density functional theory to accurately describe the interactions between solvent, reactants and substrates. Methods based on many-body perturbation theory, such as the random phase approximation (RPA), have previously been limited due to their computational complexity. However, this is now a surmountable barrier due to the advances in computational power available, in particular through modern GPU-based supercomputers. In this work, we describe the implementation of RPA calculations within BerkeleyGW and show its favorable computational performance on large complex systems relevant for catalysis and electrochemistry applications. Our implementation builds off of the static subspace approximation which, by employing a compressed representation of the frequency dependent polarizability, enables the evaluation of the RPA correlation energy with significant acceleration and systematically controllable accuracy. We find that the computational cost of calculating the RPA correlation energy scales only linearly with system size for systems containing up to 50 thousand bands, and is expected to scale quadratically thereafter. We also show excellent strong scaling results across several supercomputers, demonstrating the performance and portability of this implementation.

algorithmic development↗
DOE OSTI2021

Guest-Host Interactions in Clathrate Hydrates: Benchmark MP2 and CCSD(T)/CBS Binding Energies of CH4, CO2 and H2S in (H2O)20 Cages

We present benchmark binding energies of naturally occurring gas molecules CH4, CO2, and H2S in the small cage, namely the pentagonal dodecahedron (512) (H2O)20, which is one of the constituent cages of the 3 major lattices (structures I, II and H) of clathrate hydrates. These weak interactions require higher levels of electron correlation and converge slowly with increasing basis set to the Complete Basis Set (CBS) limit, necessitating the use of large basis sets up to the augcc- pV5Z and subsequent correction for Basis Set Superposition Error (BSSE). For the host hollow (H2O)20 cages we have identified a most stable isomer with binding energy of -200.8 ± 2.1 kcal/mol at the CCSD(T)/CBS limit (-199.2 ± 0.5 kcal/mol at the MP2/CBS limit). Additionally, we report converged second order Moller-Plesset (MP2) CBS binding energies for the encapsulation of guests in the (H2O)20 cage of -4.3 ± 0.1 for CH4@(H2O)20, -6.6 ± 0.1 for CO2@(H2O)20 and -8.5 ± 0.1 kcal/mol for H2S@(H2O)20, respectively. For CH4@(H2O)20, exhibiting the weakest encapsulation affinity among the three, we report CCSD(T)/aug-cc-pVTZ binding energies and, based on them, a CCSD(T)/CBS estimate of -4.75 ± 0.1 kcal/mol. To the best of our knowledge, the CCSD(T)/aug-cc-pVTZ calculation for CH4@(H2O)20 is the largest one reported to date (168 valence electrons, 1978 basis functions and the correlation of 84 doubly occupied and 1873 virtual orbitals) and required a scalable implementation of the (T) module on 6144 nodes (350208 cores) of the “Cori” supercomputer at the National Energy Research Supercomputing Center (NERSC) for a total execution time of 195 minutes (for the (T) part). These efficient scalable implementations of highly correlated methods offer the capability to obtain long-lasting benchmarks of intermolecular interactions in complex systems. They also provide a path towards parametrizing classical potentials needed to study the dynamical and transport properties in these complex systems as well as assess the accuracy of lower scaling electronic structure methods such as Density Functional Theory (DFT) and MP2 including its spin-biased variants.

Heindel, Joseph↗
DOE OSTI2023

Advances in machine-learning-based sampling motivated by lattice quantum chromodynamics

Sampling from known probability distributions is a ubiquitous task in computational science, underlying calculations in domains from linguistics to biology and physics. Generative machine-learning (ML) models have emerged as a promising tool in this space, building on the success of this approach in applications such as image, text, and audio generation. Often, however, generative tasks in scientific domains have unique structures and features—such as complex symmetries and the requirement of exactness guarantees—that present both challenges and opportunities for ML. This Perspective outlines the advances in ML-based sampling motivated by lattice quantum field theory, in particular for the theory of quantum chromodynamics. Enabling calculations of the structure and interactions of matter from our most fundamental understanding of particle physics, lattice quantum chromodynamics is one of the main consumers of open-science supercomputing worldwide. Here, the design of ML algorithms for this application faces profound challenges, including the necessity of scaling custom ML architectures to the largest supercomputers, but also promises immense benefits, and is spurring a wave of development in ML-based sampling more broadly. In lattice field theory, if this approach can realize its early promise it will be a transformative step towards first-principles physics calculations in particle, nuclear and condensed matter physics that are intractable with traditional approaches.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
DOE OSTI2024

Celeritas: Accelerating Geant4 with GPUs

Celeritas [1] is a new Monte Carlo (MC) detector simulation code designed for computationally intensive applications (specifically, High Lumi- nosity Large Hadron Collider (HL-LHC) simulation) on high-performance heterogeneous architectures. In the past two years Celeritas has advanced from prototyping a GPU-based single physics model in infinite medium to implementing a full set of electromagnetic (EM) physics processes in complex geometries. The current release of Celeritas, version 0.3, has incorporated full device-based navigation, an event loop in the presence of magnetic fields, and detector hit scoring. New functionality incorporates a scheduler to offload electromagnetic physics to the GPU within a Geant4-driven simulation, enabling integration of Celeritas into high energy physics (HEP) experimental frameworks such as CMSSW. On the Summit supercomputer, Celeritas performs EM physics between 6 and 32 faster using the machine’s Nvidia GPUs compared to using only CPUs. When running a multithreaded Geant4 ATLAS test beam application with full hadronic physics, using Celeritas to accelerate the EM physics results in an overall simulation speedup of 1.8–2.3× on GPU and 1.2× on CPU.

Johnson, Seth R.↗
DOE OSTI2023

Distributed memory, GPU accelerated Fock construction for hybrid, Gaussian basis density functional theory

With the growing reliance of modern supercomputers on accelerator-based architecture such a graphics processing units (GPUs), the development and optimization of electronic structure methods to exploit these massively parallel resources has become a recent priority. While significant strides have been made in the development GPU accelerated, distributed memory algorithms for many modern electronic structure methods, the primary focus of GPU development for Gaussian basis atomic orbital methods has been for shared memory systems with only a handful of examples pursing massive parallelism. Here in this work, we present a set of distributed memory algorithms for the evaluation of the Coulomb and exact exchange matrices for hybrid Kohn–Sham DFT with Gaussian basis sets via direct density-fitted (DF-J-Engine) and seminumerical (sn-K) methods, respectively. The absolute performance and strong scalability of the developed methods are demonstrated on systems ranging from a few hundred to over one thousand atoms using up to 128 NVIDIA A100 GPUs on the Perlmutter supercomputer.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2020

Establishing the quantum supremacy frontier with a 281 Pflop/s simulation

Noisy intermediate-scale quantum (NISQ) computers are entering an era in which they can perform computational tasks beyond the capabilities of the most powerful classical computers, thereby achieving 'quantum supremacy', a major milestone in quantum computing. NISQ supremacy requires comparison with a state-of-the-art classical simulator. In this paper, we report HPC simulations of hard random quantum circuits (RQC), which have been recently used as a benchmark for the first experimental demonstration of quantum supremacy, sustaining an average performance of 281 Pflop/s (true single precision) on Summit, currently the fastest supercomputer in the world. These simulations were carried out using qFlex, a tensor-network-based classical high-performance simulator of RQCs. Our results show an advantage of many orders of magnitude in energy consumption of NISQ devices over classical supercomputers. In addition, we propose a standard benchmark for NISQ computers based on qFlex.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2021

Pandemic drugs at pandemic speed: infrastructure for accelerating COVID-19 drug discovery with hybrid machine learning- and physics-based simulations on high-performance computers

The race to meet the challenges of the global pandemic has served as a reminder that the existing drug discovery process is expensive, inefficient and slow. There is a major bottleneck screening the vast number of potential small molecules to shortlist lead compounds for antiviral drug development. New opportunities to accelerate drug discovery lie at the interface between machine learning methods, in this case, developed for linear accelerators, and physics-based methods. The two in silico methods, each have their own advantages and limitations which, interestingly, complement each other. Here, we present an innovative infrastructural development that combines both approaches to accelerate drug discovery. The scale of the potential resulting workflow is such that it is dependent on supercomputing to achieve extremely high throughput. We have demonstrated the viability of this workflow for the study of inhibitors for four COVID-19 target proteins and our ability to perform the required large-scale calculations to identify lead antiviral compounds through repurposing on a variety of supercomputers.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2020

MARBLE: A Multi-GPU Aware Job Scheduler for Deep Learning on HPC Systems

Deep learning (DL) has become a key tool for solving complex scientific problems. However, managing the multi-dimensional large-scale data associated with DL, especially atop extant multiple graphics processing units (GPUs) in modern supercomputers poses significant challenges. Moreover, the latest high-performance computing (HPC) architectures bring different performance trends in training throughput compared to the existing studies. Existing DL optimizations such as larger batch size and GPU locality-aware scheduling have little effect on improving DL training throughput performance due to fast CPU-to-GPU connections. Additionally, DL training on multiple GPUs scales sublinearly. Thus, simply adding more GPUs to a system is ineffective. To this end, we design MARBLE, a first-of-its-kind job scheduler, which considers the non-linear scalability of GPUs at the intra-node level to schedule an appropriate number of GPUs per node for a job. By sharing the GPU resources on a node with multiple DL jobs, MARBLE avoids low GPU utilization in current multi-GPU DL training on HPC systems. Our comprehensive evaluation in the Summit supercomputer shows that MARBLE is able to improve DL training performance by up to 48.3% compared to the popular Platform Load Sharing Facility (LSF) scheduler. Compared to the state-of-the-art of DL scheduler, Optimus, MARBLE reduces the job completion time by up to 47%.

Han, Jingoo↗
DOE OSTI2021

Early Evaluation of Fugaku A64FX Architecture Using Climate Workloads

The Energy Exascale Earth System Model (E3SM) Project is an ongoing, state-of-the-science Earth system modeling, simulation, and prediction project that targets efficient utilization of U.S. Department of Energy’s (DOE) supercomputers to meet the science needs of the nation and the mission needs of DOE. This work focuses on our early evaluation of the A64FX architecture on Fugaku supercomputer using E3SM benchmarks. We will present results that track hardware trends, facilitate architecture comparison and the specific impact on our workload using an atmospheric model benchmark. We have two variants of the code written in Fortran and C++/Kokkos respectively which were used to collect data on a variety of CPU and GPU platforms. Furthermore, we have conducted a comparative evaluation of the compilers on the A64FX architecture and found GNU to be the best performer for our workload. Our experience so far indicates that Fugaku/A64FX shows promising energy efficiency (performance/Watt) with further performance gains possible through architecture-aware optimization efforts.

Sreepathi, Sarat↗
DOE OSTI2025

JACC.shared: Leveraging HPC Metaprogramming and Performance Portability for Computations That Use Shared Memory GPUs

In this work, we present JACC.shared, a new feature of Julia for ACCelerators (JACC), which is the performanceportable and metaprogramming model of the just-in-time and LLVM-based Julia language. This new feature allows JACC applications to leverage the high-performance computing (HPC) capabilities of high-bandwidth, on-chip GPU memory. Historically, exploiting high-bandwidth, shared-memory GPUs has not been a priority for high-level programming solutions. JACC.shared covers that gap for the first time, thereby providing a highlevel, portable, and easy-to-use solution for programmers to exploit this memory and supporting all current major accelerator architectures. Well-known HPC and AI workloads, such as multi/hyperspectral imaging and AI convolutions, have been used to evaluate JACC.shared on two exascale GPU architectures hosted by some of the most powerful US Department of Energy supercomputers: Perlmutter (NVIDIA A100) and Frontier (AMD MI250X). The performance evaluation reports speedup of up to 3.5× by adding only one line of code to the base codes, thus providing important accelerators in a simple, portable, and transparent way and elevating the programming productivity and performance-portability capabilities for Julia/JACC HPC, AI, and scientific applications.

Valero Lara, Pedro [ORNL] (ORCID:0000000214794310)↗
DOE OSTI2026

Pixel-Resolved Long-Context Learning for Turbulence at Exascale: Resolving Small-scale Eddies Toward the Viscous Limit

Turbulence plays a crucial role in multiphysics applications, including aerodynamics, fusion, and combustion. Accurately capturing turbulence's multiscale characteristics is essential for reliable predictions of multiphysics interactions, but remains a grand challenge even for exascale supercomputers and advanced deep learning models. The extreme-resolution data required to represent turbulence, ranging from billions to trillions of grid points, pose prohibitive computational costs for models based on architectures like vision transformers. To address this challenge, we introduce a multiscale hierarchical Turbulence Transformer that reduces sequence length from billions to a few millions and a novel RingX sequence parallelism approach that enables scalable long-context learning. We perform scaling and science runs on the Frontier supercomputer. Our approach demonstrates excellent performance up to 1.1 EFLOPS on 32,768 AMD GPUs, with a scaling efficiency of 94\%. To our knowledge, this is the first AI model for turbulence that can capture small-scale eddies down to the dissipative range in three-dimensional turbulence at high Reynolds numbers.

Yin, Junqi [ORNL] (ORCID:0000000338435520)↗
DOE OSTI2022

Modeling pre-Exascale AMR Parallel I/O Workloads via Proxy Applications

The present work investigates the modeling of preexascale input/output (I/O) workloads of Adaptive Mesh Refinement (AMR) simulations through a simple proxy application. We collect data from the AMReX Castro framework running on the Summit supercomputer for a wide range of scales and mesh partitions for the hydrodynamic Sedov case as a baseline to provide sufficient coverage to the formulated proxy model. The non-linear analysis data production rates are quantified as a function of a set of input parameters such as output frequency, grid size, number of levels, and the Courant-Friedrichs-Lewy (CFL) condition number for each rank, mesh level and simulation time step. Linear regression is then applied to formulate a simple analytical model which allows to translate AMReX inputs into MACSio proxy I/O application parameters, resulting in a simple “kernel” approximation for data production at each time step. Results show that MACSio can simulate actual AMReX nonlinear “static” I/O workloads to a certain degree of confidence on the Summit supercomputer using the present methodology. The goal is to provide an initial level of understanding of AMR I/O workloads via lightweight proxy applications models to facilitate autotune data management strategies in anticipation of exascale systems.

Godoy, William↗