Engineering Papers⌕ Search

Engineering topics

Budiardja, Reuben D.

Publications and source records attributed to Budiardja, Reuben D..

Early experiences evaluating the HPE/Cray ecosystem for AMD GPUs

Summary The Oak Ridge Leadership Computing Facility (OLCF) has a long history of supporting and promoting GPU‐accelerated computing starting with the deployment of the Titan supercomputer in 2021 and continuing with the Summit supercomputer which has a theoretical peak performance of approximately 200 petaflops. Because the majority of Summit's computational power comes from its 27,972 GPUs, users must port their applications to one of the supported programming models in order to make efficient use of the system. To prepare the transition to Frontier, the OLCF's exascale supercomputer, users will need to adapt to an entirely new ecosystem which will include new hardware and software technologies. First, users will need to familiarize themselves with the AMD Radeon GPU architecture. Furthermore, users who have been previously relying on CUDA will need to transition to the Heterogeneous‐Computing Interface for Portability (HIP) or one of the other supported programming models (e.g., OpenMP, OpenACC). In this work, we describe our initial experiences and lessons learned in porting three applications or proxy apps currently running on Summit to the HPE/Cray ecosystem to leverage the compute power from AMD GPUs: minisweep, GenASiS, and Sparkler. Each one is representative of current production workloads utilized at the OLCF, different programming languages, and different programming models.

Melesse Vergara, Verónica G.↗

Experiences Readying Applications for Exascale

The advent of Exascale computing invites an assessment of existing best practices for developing application readiness on the world's largest supercomputers. This work details observations from the last four years in preparing scientific applications to run on the Oak Ridge Leadership Computing Facility's (OLCF) Frontier system. This paper addresses a range of topics in software including programmability, tuning, and portability considerations that are key to moving applications from existing systems to future installations. A set of representative workloads provides case studies for general system and software testing. We evaluate the use of early access systems for development across several generations of hardware. Finally, we discuss how best practices were identified and disseminated to the community through a wide range of activities including user-guides and trainings. We conclude with recommendations for ensuring application readiness on future leadership computing systems.

exascale↗

GenASiS Mathematics: Object-oriented manifolds, operations, and solvers for large-scale physics simulations (version 2)

We report GenASiS Mathematics provides modern Fortran classes furnishing extensible object-oriented functionality for the solution of fields governed by selected partial differential equations. The initial release included extensible object-oriented implementations of simple meshes and the evolution of generic conserved currents thereon. This revision - Version 2 of Mathematics - includes significant reorganization and streamlining of these classes, higher-order reconstruction by a different method, a Poisson solver, coarsening to avoid Courant time step limitations near coordinate singularities, and the offloading of computational kernels to GPUs.

97 MATHEMATICS AND COMPUTING↗

GenASiS Basics: Object-oriented utilitarian functionality for large-scale physics simulations (Version 4)

GenASiS Basics provides modern Fortran classes furnishing extensible object-oriented utilitarian functionality for large-scale physics simulations on distributed memory supercomputers. This functionality includes physical units and constants; display to the screen or standard output device; message passing; I/O to disk; and runtime parameter management and usage statistics. Herein, this revision—Version 4 of Basics—includes a name change and additions to functionality, including the facilitation of direct communication between GPUs.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

C HIMERA : A Massively Parallel Code for Core-collapse Supernova Simulations

Herein we provide a detailed description of the C HIMERA code, a code developed to model core collapse supernovae (CCSNe) in multiple spatial dimensions. The CCSN explosion mechanism remains the subject of intense research. Progress to date demonstrates that it involves a complex interplay of neutrino production, transport, and interaction in the stellar core, three-dimensional stellar core fluid dynamics and its associated instabilities, nuclear burning, and the fundamental physics of the neutrino–stellar core weak interactions and the equations of state of all stellar core constituents—particularly, the nuclear equation of state associated with core nucleons, both free and bound in nuclei. C HIMERA , by incorporating detailed neutrino transport, realistic neutrino–matter interactions, three-dimensional hydrodynamics, realistic nuclear, leptonic, and photonic equations of state, and a nuclear reaction network, along with other refinements, can be used to study the role of neutrino radiation, hydrodynamic instabilities, and a variety of input physics in the explosion mechanism itself. It can also be used to compute observables such as neutrino signatures, gravitational radiation, and the products of nucleosynthesis associated with CCSNe. The code contains modules for neutrino transport, multidimensional compressible hydrodynamics, nuclear reactions, a variety of neutrino interactions, equations of state, and modules to provide data for post-processing observables such as the products of nucleosynthesis, and gravitational radiation. C HIMERA is an evolving code, being updated periodically with improved input physics and numerical refinements. We detail here the current version of the code, from which future improvements will stem, which can in turn be described as needed in future publications.

79 ASTRONOMY AND ASTROPHYSICS↗

Experiences in porting mini-applications to OpenACC and OpenMP on heterogeneous systems

This article studies mini-applications—Minisweep, GenASiS , GPP, and FF—that use computational methods commonly encountered in HPC. We have ported these applications to develop OpenACC and OpenMP versions, and evaluated their performance on Titan (Cray XK7 with K20x GPUs), Cori (Cray XC40 with Intel KNL), Summit (IBM AC922 with Volta GPUs), and Cori-GPU (Cray CS-Storm 500NX with Intel Skylake and Volta GPUs). Our goals are for these new ports to be useful to both application and compiler developers, to document and describe the lessons learned and the methodology to create optimized OpenMP and OpenACC versions, and to provide a description of possible migration paths between the two specifications. Cases where specific directives or code patterns result in improved performance for a given architecture are highlighted. Here, we also include discussions of the functionality and maturity of the latest compilers available on the above platforms with respect to OpenACC or OpenMP implementations.

97 MATHEMATICS AND COMPUTING↗

Targeting GPUs with OpenMP directives on Summit: A simple and effective Fortran experience

We use OpenMP to target hardware accelerators (GPUs) on Summit, a newly deployed supercomputer at the Oak Ridge Leadership Computing Facility (OLCF), demonstrating simplified access to GPU devices for users of our astrophysics code GenASiS and useful speedup on a sample fluid dynamics problem. We modify our workhorse class for data storage to include members and methods that significantly streamline the persistent allocation of and association to GPU memory. Users offload computational kernels with OpenMP target directives that are rather similar to constructs already familiar from multi-core parallelization. In this initial example we ask, “With a given number of Summit nodes, how fast can we compute with and without GPUs?”, and find total wall time speedups of ~ 12X. We also find reasonable weak scaling up to 8000 GPUs (1334 Summit nodes). We make available the source code from this work at https://github.com/GenASiS/GenASiS_Basics.

97 MATHEMATICS AND COMPUTING↗

GenASiS Basics: Object-oriented utilitarian functionality for large-scale physics simulations (Version 3)

GenASiS Basics provides Fortran 2003 classes furnishing extensible object-oriented utilitarian functionality for large-scale physics simulations on distributed memory supercomputers. This functionality includes physical units and constants; display to the screen or standard output device; message passing; I/O to disk; and runtime parameter management and usage statistics. This revision – Version 3 of Basics – includes a significant name change, some minor additions to functionality, and a major addition to functionality: infrastructure facilitating the offloading of computational kernels to devices such as GPUs.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

GenASiSBasics: Object-oriented utilitarian functionality for large-scale physics simulations (Version 2)

GenASiSBasics provides Fortran 2003 classes furnishing extensible object-oriented utilitarian functionality for large-scale physics simulations on distributed memory supercomputers. This functionality includes physical units and constants; display to the screen or standard output device; message passing; I/O to disk; and runtime parameter management and usage statistics. Finally, this revision–Version 2 of Basics–makes mostly minor additions to functionality and includes some simplifying name changes.

97 MATHEMATICS AND COMPUTING↗

Convection- and SASI-driven flows in parametrized models of core-collapse supernova explosions

We present initial results from three-dimensional simulations of parametrized core-collapse supernova (CCSN) explosions obtained with our astrophysical simulation code General Astrophysical Simulation System (GenASIS). Here, we are interested in nonlinear flows resulting from neutrino-driven convection and the standing accretion shock instability (SASI) in the CCSN environment prior to and during the explosion. By varying parameters in our model that control neutrino heating and shock dissociation, our simulations result in convection-dominated and SASI-dominated evolution. We describe this initial set of simulation results in some detail. To characterize the turbulent flows in the simulations, we compute and compare velocity power spectra from convection-dominated and SASI-dominated (both non-exploding and exploding) models. When compared to SASI-dominated models, convection-dominated models exhibit significantly more power on small spatial scales.

79 ASTRONOMY AND ASTROPHYSICS↗

STOCHASTICITY AND EFFICIENCY IN SIMPLIFIED MODELS OF CORE-COLLAPSE SUPERNOVA EXPLOSIONS

We present an initial report on 160 simulations of a highly simplified model of the post-bounce core-collapse supernova environment in three spatial dimensions (3D). We set different values of a parameter characterizing the impact of nuclear dissociation at the stalled shock in order to regulate the post-shock fluid velocity, thereby determining the relative importance of convection and the stationary accretion shock instability (SASI). While our convection-dominated runs comport with the paradigmatic notion of a “critical neutrino luminosity” for explosion at a given mass accretion rate (albeit with a nontrivial spread in explosion times just above threshold), the outcomes of our SASI-dominated runs are much more stochastic: a sharp threshold critical luminosity is “smeared out” into a rising probability of explosion over a ∼20% range of luminosity. We also find that the SASI-dominated models are able to explode with 3–4 times less efficient neutrino heating, indicating that progenitor properties, and fluid and neutrino microphysics, conducive to the SASI would make the neutrino-driven explosion mechanism more robust.

79 ASTRONOMY AND ASTROPHYSICS↗

GenASiS Basics: Object-oriented utilitarian functionality for large-scale physics simulations

Aside from numerical algorithms and problem setup, large-scale physics simulations on distributed-memory supercomputers require more basic utilitarian functionality, such as physical units and constants; display to the screen or standard output device; message passing; I/O to disk; and runtime parameter management and usage statistics. Here we describe and make available Fortran 2003 classes furnishing extensible object-oriented implementations of this sort of rudimentary functionality, along with individual `unit test' programs and larger example problems demonstrating their use. Lastly, these classes compose the Basics division of our developing astrophysics simulation code GenASiS (General Astrophysical Simulation System), but their fundamental nature makes them useful for physics simulations in many fields.

97 MATHEMATICS AND COMPUTING↗