Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “EXASCALE”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 163 records · Page 9
All sourcesNASA NTRSDOE OSTI
DOE OSTI2024

GPU-enabled extreme-scale turbulence simulations: Fourier pseudo-spectral algorithms at the exascale using OpenMP offloading

Fourier pseudo-spectral methods for nonlinear partial differential equations are of wide interest in many areas of advanced computational science, including direct numerical simulation of three-dimensional (3-D) turbulence governed by the Navier-Stokes equations in fluid dynamics. This paper presents a new capability for simulating turbulence at a new record resolution up to 35 trillion grid points, on the world's first exascale computer, Frontier, comprising AMD MI250x GPUs with HPE's Slingshot interconnect and operated by the US Department of Energy's Oak Ridge Leadership Computing Facility (OLCF). Key programming strategies designed to take maximum advantage of the machine architecture involve performing almost all computations on the GPU which has the same memory capacity as the CPU, performing all-to-all communication among sets of parallel processes directly on the GPU, and targeting GPUs efficiently using OpenMP offloading for intensive number-crunching including 1-D Fast Fourier Transforms (FFT) performed using AMD ROCm library calls. With 99% of computing power on Frontier being on the GPU, leaving the CPU idle leads to a net performance gain via avoiding the overhead of data movement between host and device except when needed for some I/O purposes. Memory footprint including the size of communication buffers for MPI_ALLTOALL is managed carefully to maximize the largest problem size possible for a given node count. Detailed performance data including separate contributions from different categories of operations to the elapsed wall time per step are reported for five grid resolutions, from 2048 3 on a single node to 32768 3 on 4096 or 8192 nodes out of 9408 on the system. Both 1D and 2D domain decompositions which divide a 3D periodic domain into slabs and pencils respectively are implemented. The present code suite (labeled by the acronym GESTS, GPUs for Extreme Scale Turbulence Simulations) achieves a figure of merit (in grid points per second) exceeding goals set in the Center for Accelerated Application Readiness (CAAR) program for Frontier. The performance attained is highly favorable in both weak scaling and strong scaling, with notable departures only for 2048 3 where communication is entirely intra-node, and for 32768 3 , where a challenge due to small message sizes does arise. Communication performance is addressed further using a lightweight test code that performs all-to-all communication in a manner matching the full turbulence simulation code. Performance at large problem sizes is affected by both small message size due to high node counts as well as dragonfly network topology features on the machine, but is consistent with official expectations of sustained performance on Frontier. Overall, although not perfect, the scalability achieved at the extreme problem size of 32768 3 (and up to 8192 nodes — which corresponds to hardware rated at just under 1 exaflop/sec of theoretical peak computational performance) is arguably better than the scalability observed using prior state-of-the-art algorithms on Frontier's predecessor machine (Summit) at OLCF. New science results for the study of intermittency in turbulence enabled by this code and its extensions are to be reported separately in the near future.

3D fast Fourier transform↗
DOE OSTI2026

Shadow of the Future: Developing Trust and Software within the Exascale Computing Project

Collaboration and team science are emerging areas of interest in software production. Historically, multi-institutional research collaborations are difficult to initiate and maintain, negatively impacting communication, negotiation, and dialogue between industry, government, and academic researchers. The Exascale Computing Project (ECP), a massive, multi-team, high-stakes initiative, facilitated broader research collaboration under a shared funding structure and extended timeline to support scientific discovery. Here, we conducted interviews with ECP teams, representing a variety of domain specialties, research institutions, and programming backgrounds. Using thematic analysis, we assessed how ECP’s structure created an environment of increased trust among projects and how software shared between teams facilitated sustained collaboration. We found that the expectation of future collaboration, i.e., the shadow of the future, greatly enhanced trust among teams and the quality of scientific software produced. Based on our findings within ECP projects, we connect to the existing literature on trust in software engineering and share recommendations for sustainable multi-institutional collaboration and shared best software practices.

Exascale computing project↗
DOE OSTI2021

Toward performance-portable PETSc for GPU-based exascale systems

The Portable Extensible Toolkit for Scientific computation (PETSc) library delivers scalable solvers for nonlinear time-dependent differential and algebraic equations and for numerical optimization. The PETSc design for performance portability addresses fundamental GPU accelerator challenges and stresses flexibility and extensibility by separating the programming model used by the application from that used by the library, and it enables application developers to use their preferred programming model, such as Kokkos, RAJA, SYCL, HIP, CUDA, or OpenCL, on upcoming exascale systems. Furthermore, a blueprint for using GPUs from PETSc-based codes is provided, and case studies emphasize the flexibility and high performance achieved on current GPU-based systems.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2022

Global Sensitivity Analysis Using the Ultra‐Low Resolution Energy Exascale Earth System Model

Abstract For decades, Arctic temperatures have increased twice as fast as average global temperatures. As a first step toward quantifying parametric uncertainty in Arctic climate, we performed a variance‐based global sensitivity analysis (GSA) using a fully coupled, ultra‐low resolution (ULR) configuration of version 1 of the U.S. Department of Energy's Energy Exascale Earth System Model (E3SMv1). Specifically, we quantified the sensitivity of six quantities of interests (QOIs), which characterize changes in Arctic climate over a 75 year period, to uncertainties in nine model parameters spanning the sea ice, atmosphere, and ocean components of E3SMv1. Sensitivity indices for each QOI were computed with a Gaussian process emulator using 139 random realizations of the random parameters and fixed preindustrial forcing. Uncertainties in the atmospheric parameters in the Cloud Layers Unified by Binormals (CLUBB) scheme were found to have the most impact on sea ice status and the larger Arctic climate. Our results demonstrate the importance of conducting sensitivity analyses with fully coupled climate models. The ULR configuration makes such studies computationally feasible today due to its low computational cost. When advances in computational power and modeling algorithms enable the tractable use of higher‐resolution models, our results will provide a baseline that can quantify the impact of model resolution on the accuracy of sensitivity indices. Moreover, the confidence intervals provided by our study, which we used to quantify the impact of the number of model evaluations on the accuracy of sensitivity estimates, have the potential to inform the computational resources needed for future sensitivity studies.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2024

To Exascale and Beyond—The Simple Cloud-Resolving E3SM Atmosphere Model (SCREAM), a Performance Portable Global Atmosphere Model for Cloud-Resolving Scales

The new generation of heterogeneous CPU/GPU computer systems offer much greater computational performance but are not yet widely used for climate modeling. One reason for this is that traditional climate models were written before GPUs were available and would require an extensive overhaul to run on these new machines. In addition, even conventional “high–resolution” simulations don't currently provide enough parallel work to keep GPUs busy, so the benefits of such overhaul would be limited for the types of simulations climate scientists are accustomed to. The vision of the Simple Cloud-Resolving Energy Exascale Earth System (E3SM) Atmosphere Model (SCREAM) project is to create a global atmospheric model with the architecture to efficiently use GPUs and horizontal resolution sufficient to fully take advantage of GPU parallelism. After 5 years of model development, SCREAM is finally ready for use. In this paper, we describe the design of this new code, its performance on both CPU and heterogeneous machines, and its ability to simulate real-world climate via a set of four 40 day simulations covering all 4 seasons of the year.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2020

Toward Exascale: Overview of Large Eddy Simulations and Direct Numerical Simulations of Nuclear Reactor Flows with the Spectral Element Method in Nek5000

At the beginning of the last decade, Petascale supercomputers (i.e., computers capable of more than 1 petaFLOP) emerged. Now, at the dawn of exascale supercomputing, we provide a review of recent landmark simulations of portions of reactor components with turbulence-resolving techniques that this computational power has made possible. In fact, these simulations have provided invaluable insight into flow dynamics, which is difficult or often impossible to obtain with experiments alone. We focus on simulations performed with the spectral element method, as this method has emerged as a powerful tool to deliver massively parallel calculations at high fidelity by using large eddy simulation or direct numerical simulation. We also limit this paper to constant-property incompressible flow of a Newtonian fluid in the absence of other body or external forces, although the method is by no means limited to this class of flows. We briefly review the fundamentals of the method and the reasons it is compelling for the simulation of nuclear engineering flows. We review in detail a series of Petascale simulations, including the simulations of helical coil steam generators, fuel assemblies, and pebble beds. Even with Petascale computing, however, limitations for nuclear modeling and simulation tools remain. In particular, the size and scope of turbulence-resolving simulations are still limited by computing power and resolution requirements, which scale with the Reynolds number. In the final part of this paper, we discuss the future of the field, including recent advancements in emerging architectures such as GPUbased supercomputers, which are expected to power the next generation of high-performance computers.

computational fluid dynamics↗
DOE OSTI2022

PICSAR-QED: a Monte Carlo module to simulate strong-field quantum electrodynamics in particle-in-cell codes for exascale architectures

Abstract Physical scenarios where the electromagnetic fields are so strong that quantum electrodynamics (QED) plays a substantial role are one of the frontiers of contemporary plasma physics research. Investigating those scenarios requires state-of-the-art particle-in-cell (PIC) codes able to run on top high-performance computing (HPC) machines and, at the same time, able to simulate strong-field QED processes. This work presents the PICSAR-QED library, an open-source, portable implementation of a Monte Carlo module designed to provide modern PIC codes with the capability to simulate such processes, and optimized for HPC. Detailed tests and benchmarks are carried out to validate the physical models in PICSAR-QED, to study how numerical parameters affect such models, and to demonstrate its capability to run on different architectures (CPUs and GPUs). Its integration with WarpX, a state-of-the-art PIC code designed to deliver scalable performance on upcoming exascale supercomputers, is also discussed and validated against results from the existing literature.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗
DOE OSTI2020

Training Efforts in the Exascale Computing Project

This article delineates the training activities carried out under the auspices of the U.S. Department of Energy’s (DOE) Exascale Computing Project (ECP). While some of these activities are specific to members of ECP, others can be beneficial to the community at large. Additionally, we report on training opportunities and resources that the broad Computer Science and Engineering (CS&E) community can tap into; we seek to increase awareness about these resources, which we expect to go beyond ECP’s scope and life cycle.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2024

Approaches for the Simulation of Coupled Processes in Evolving Fractured Porous Media Enabled by Exascale Computing

Models have historically represented fractured porous media with continuum descriptions that characterize the media using bulk parameters. The impact of small-scale features is not captured in these models, although they may be controlling the performance of subsurface applications. Pore-scale models can simulate processes in small-scale features by representing the pore space geometry explicitly but are computationally expensive for large domains. The alternative multiscale approach entails the combination of pore-scale and continuum-scale descriptions in a single framework. We use Chombo-Crunch, a computational capability that discretizes complex geometries with an adaptive, embedded boundary method to contrast these two approaches. Chombo-Crunch takes advantage of recent computational performance and memory bandwidth improvements resulting from the emergence of exascale computing resources. These combined improvements enable the efficient simulation of reactive transport in fractured media with a high degree of fidelity and the ability to capture the control small-scale processes exert on the overall medium evolution.

42 ENGINEERING↗
DOE OSTI2023

Experiences Readying Applications for Exascale

The advent of Exascale computing invites an assessment of existing best practices for developing application readiness on the world's largest supercomputers. This work details observations from the last four years in preparing scientific applications to run on the Oak Ridge Leadership Computing Facility's (OLCF) Frontier system. This paper addresses a range of topics in software including programmability, tuning, and portability considerations that are key to moving applications from existing systems to future installations. A set of representative workloads provides case studies for general system and software testing. We evaluate the use of early access systems for development across several generations of hardware. Finally, we discuss how best practices were identified and disseminated to the community through a wide range of activities including user-guides and trainings. We conclude with recommendations for ensuring application readiness on future leadership computing systems.

exascale↗
DOE OSTI2021

MFIX-Exa: A path toward exascale CFD-DEM simulations

MFIX-Exa is a computational fluid dynamics–discrete element model (CFD-DEM) code designed to run efficiently on current and next-generation supercomputing architectures. MFIX-Exa combines the CFD-DEM expertise embodied in the MFIX code—which was developed at NETL and is used widely in academia and industry—with the modern software framework, AMReX, developed at LBNL. The fundamental physics models follow those of the original MFIX, but the combination of new algorithmic approaches and a new software infrastructure will enable MFIX-Exa to leverage future exascale machines to optimize the modeling and design of multiphase chemical reactors.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2024

Towards exascale for wind energy simulations

We examine large-eddy-simulation modeling approaches and computational performance of two open-source computational fluid dynamics codes for the simulation of atmospheric boundary layer flows that are of direct relevance to wind energy production. The first code, NekRS, is a high-order, unstructured-grid, spectral element code. The second code, AMR-Wind, is a second-order, block-structured, finite-volume code with adaptive mesh refinement capabilities. The objective of this study is to co-develop these codes in order to improve model fidelity and performance for each. These features will be critical for running ABL-based applications such as wind farm analysis on advanced computing architectures. To this end, we investigate the performance of NekRS and AMR-Wind on the Oak Ridge Leadership Facility supercomputers Summit, using 4 to 800 nodes (24 to 4,800 NVIDIA V100 GPUs), and Crusher, the testbed for the Frontier exascale system, using 18 to 384 Graphics Compute Dies on AMD MI250X GPUs. We compare strong- and weak-scaling capabilities, linear solver performance, and time to solution. We also identify leading inhibitors to parallel scaling.

17 WIND ENERGY↗
DOE OSTI2015

Final report: Compiled MPI. Cost-Effective Exascale Application Development

This is the final report on Compiled MPI: Cost-Effective Exascale Application Development, and summarizes the results under this project. The project investigated runtime enviroments that improve the performance of MPI (Message-Passing Interface) programs; work at Illinois in the last period of this project looked at optimizing data access optimizations expressed with MPI datatypes.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2016

Developing Materials Processing to Performance Modeling Capabilities and the Need for Exascale Computing Architectures (and Beyond)

Additive Manufacturing techniques are presenting the Department of Energy and the NNSA Laboratories with new opportunities to consider novel component production and repair processes, and to manufacture materials with tailored response and optimized performance characteristics. Additive Manufacturing technologies already are being applied to primary NNSA mission areas, including Nuclear Weapons. These mission areas are adapting to these new manufacturing methods, because of potential advantages, such as smaller manufacturing footprints, reduced needs for specialized tooling, an ability to embed sensing, novel part repair options, an ability to accommodate complex geometries, and lighter weight materials. To realize the full potential of Additive Manufacturing as a game-changing technology for the NNSA’s national security missions; however, significant progress must be made in several key technical areas. In addition to advances in engineering design, process optimization and automation, and accelerated feedstock design and manufacture, significant progress must be made in modeling and simulation. First and foremost, a more mature understanding of the process-structure-property-performance relationships must be developed. Because Additive Manufacturing processes change the nature of a material’s structure below the engineering scale, new models are required to predict materials response across the spectrum of relevant length scales, from the atomistic to the continuum. New diagnostics will be required to characterize materials response across these scales. And not just models, but advanced algorithms, next-generation codes, and advanced computer architectures will be required to complement the associated modeling activities. Based on preliminary work in each of these areas, a strong argument for the need for Exascale computing architectures can be made, if a legitimate predictive capability is to be developed.

36 MATERIALS SCIENCE↗
DOE OSTI2016

Exascale Virtualized and Programmable Distributed Cyber Resource Control: Final Scientific Technical Report

Extreme-science drives the need for distributed exascale processing and communications that are carefully, yet flexibly, managed. Exponential growth of data for scientific simulations, experimental data, collaborative data analyses, remote visualization and GRID computing requirements of scientists in fields as diverse as high energy physics, climate change, genomics, fusion, synchrotron radiation, material science, medicine, and other scientific disciplines cannot be accommodated by simply applying existing transport protocols to faster pipes. Further, scientific challenges today demand diverse research teams, heightening the need for and increasing the complexity of collaboration. To address these issues within the network layer and physical layer, we have performed a number of research activities surrounding effective allocation and management of elastic optical network (EON) resources, particularly focusing on FlexGrid transponders. FlexGrid transponders support the opportunity to build Layer-1 connections at a wide range of bandwidths and to reconfigure them rapidly. The new flexibility supports complex new ways of using the physical layer that must be carefully managed and hidden from the scientist end-users. FlexGrid networks utilize flexible (or elastic) spectral bandwidths for each data link without using fixed wavelength grids. The flexibility in spectrum allocation brings many appealing features to network operations. Current networks are designed for the worst case impairments in transmission performance and the assigned spectrum is over-provisioned. In contrast, the FlexGrid networks can operate with the highest spectral efficiency and minimum bandwidth for the given traffic demand while meeting the minimum quality of transmission (QoT) requirement. Two primary focuses of our research are: (1) resource and spectrum allocation (RSA) for IP traffic over EONs, and (2) RSA for cross-domain optical networks. Previous work concentrates primarily on large file transfers within a single domain. Adding support for IP traffic changes the nature of the RSA problem: instead of choosing to accept or deny each request for network support, IP traffic is inherently elastic and thus lends itself to a bandwidth maximization formulation. We developed a number of algorithms that could be easily deployed within existing and new FlexGrid networks, leading to networks that better support scientific collaboration. Cross-domain RSA research is essential to support large-scale FlexGrid networks, since configuration information is generally not shared or coordinated across domains. The results presented here are in their early stages. They are technically feasible and practical, but still require coordination among organizations and equipment owners and a higher-layer framework for managing network requests.

42 ENGINEERING↗
DOE OSTI2018

Scalable compiler technology for exascale

This is the Final report of the DOE Project, "Scalable Compiler Technology for Exascale." It summarizes the main findings, the students suported and the technical contributions.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2017

XPRESS: eXascale PRogramming Environment and System Software

The XPRESS Project is one of four major projects of the DOE Office of Science Advanced Scientific Computing Research X-stack Program initiated in September, 2012. The purpose of XPRESS is to devise an innovative system software stack to enable practical and useful exascale computing around the end of the decade with near-term contributions to efficient and scalable operation of trans-Petaflops performance systems in the next two to three years; both for DOE mission-critical applications. To this end, XPRESS directly addresses critical challenges in computing of efficiency, scalability, and programmability through introspective methods of dynamic adaptive resource management and task scheduling.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2026

Providing Thermal Stability for an Exascale Supercomputer: A Case Study of Frontier's Cooling System

High performance computing (HPC) systems frequently produce large dynamic power swings, even under typical operating conditions, that can present a significant challenge for their direct-liquid cooling systems. Further, the primary cooling loops that must remove this waste heat have response times measured in minutes while the underlying HPC component thermal stress is measured in seconds. The per-socket power demand for both compute processing units (CPUs) and graphic processing units ( GPUs) continues to increase with each successive generation while case temperatures are declining. New HPC systems are expected to exacerbate the challenge of these dynamic power swings and the impact on effective and timely cooling systems. This paper describes the cooling and controls system for Oak Ridge National Laboratory’s Frontier Supercomputer, the first sustained exascale system, as a case study for this situation. The cooling and control system for Frontier demonstrates specific success, but with a number of trade-offs and decisions that suggest further design and operating optimizations for the community at large to consider.

42 ENGINEERING↗