Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “exascale computing”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 667 records · Page 37
All sourcesNASA NTRSDOE OSTI
DOE OSTI2015

Performance characterization of irregular I/O at the extreme scale

Here, this paper reports our experience with irregular I/O and describes lessons learned when running applications with such I/O on supercomputers at the extreme scale. Specifically, we study how irregularities in I/O patterns (i.e., irregular amount of data written per process at each I/O step) in scientific simulations can cause increasing I/O times and substantial loss in scalability. To this end, we quantify the impact of irregular I/O patterns on the I/O performance of scientific applications at the extreme scale by statistically modeling the irregular I/O behavior of two scientific applications: the Monte Carlo application QMCPack and the adaptive mesh refinement application ENZO. For our testing, we feed our model into I/O kernels of two well-known I/O data models (i.e., ADIOS and HDF) to measure the performance of the two applications’ I/O under different I/O settings. Empirically, we show how the growing data sizes and the irregular I/O patterns in these applications are both relevant factors impacting performance.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2023

FORGE: Pre-Training Open Foundation Models for Science

Large language models (LLMs) are poised to revolutionize the way we conduct scientific research. However, both model complexity and pre-training cost are impeding effective adoption for the wider science community. Identifying suitable scientific use cases, finding the optimal balance between model and data sizes, and scaling up model training are among the most pressing issues that need to be addressed. In this study, we provide practical solutions for building and using LLM-based foundation models targeting scientific research use cases. We present an end-to-end examination of the effectiveness of LLMs in scientific research, including their scaling behavior and computational requirements on Frontier, the first Exascale supercomputer. We have also developed for release to the scientific community a suite of open foundation models called FORGE with up to 26B parameters using 257B tokens from over 200M scientific articles, with performance either on par or superior to other state-of-the-art comparable models. We have demonstrated the use and effectiveness of FORGE on scientific downstream tasks. Our research establishes best practices that can be applied across various fields to take advantage of LLMs for scientific discovery.

Yin, Junqi↗
DOE OSTI2022

ornladios/ADIOS2

ADIOS 2: The Adaptable Input Output (I/O) System version 2 is an open-source framework that addresses scientific data management challenges, e.g. scalable parallel I/O, as we approach the exascale era in high-performance computing (HPC). ADIOS 2 bindings are available in C++, C, Fortran, Python and can be used on supercomputers, personal computers, and cloud systems running on Linux, macOS and Windows. ADIOS 2 has out-of-the-box support for MPI and serial environments.

ECP↗
DOE OSTI2023

Examination of a Methane/Diesel RCCI Engine Using Pele: Preprint

Multi-fuel, advanced injection strategies have become increasingly promising as a strategy to mitigate the emissions generated from internal combustion engines. By carefully controlling the combustion phasing in-cylinder, these new multi-pulse, multi-fuel injection strategies are able to burn in the low-temperature combustion regime where both NOx and soot are not readily produced, reducing the need for extensive exhaust gas recirculation systems. In this study, we examine a reactivity-controlled compression ignition (RCCI) strategy that uses an early pre-filled methane-air mixture with low turbulence background as the low-reactivity fuel and a direct injection of four discrete dodecane jets as a surrogate for the high-reactivity diesel fuel. We use the Pele software suite, a highly optimized, exascale-ready, adaptive mesh refinement codebase to perform high-resolution numerical simulations of a scaled down, single cylinder from the RCCI engine. Here, we resolve the ignition kernels down to micrometer scales and present several statistical quantities evaluating the development of the flow and detailing the onset of ignition and subsequent flame development. Particular attention is paid to the conditions surrounding the onset of the first ignition kernels and discussing what led to the development of those conditions.

CFD↗
DOE OSTI2016

Exascale Virtualized and Programmable Distributed Cyber Resource Control: Final Scientific Technical Report

Extreme-science drives the need for distributed exascale processing and communications that are carefully, yet flexibly, managed. Exponential growth of data for scientific simulations, experimental data, collaborative data analyses, remote visualization and GRID computing requirements of scientists in fields as diverse as high energy physics, climate change, genomics, fusion, synchrotron radiation, material science, medicine, and other scientific disciplines cannot be accommodated by simply applying existing transport protocols to faster pipes. Further, scientific challenges today demand diverse research teams, heightening the need for and increasing the complexity of collaboration. To address these issues within the network layer and physical layer, we have performed a number of research activities surrounding effective allocation and management of elastic optical network (EON) resources, particularly focusing on FlexGrid transponders. FlexGrid transponders support the opportunity to build Layer-1 connections at a wide range of bandwidths and to reconfigure them rapidly. The new flexibility supports complex new ways of using the physical layer that must be carefully managed and hidden from the scientist end-users. FlexGrid networks utilize flexible (or elastic) spectral bandwidths for each data link without using fixed wavelength grids. The flexibility in spectrum allocation brings many appealing features to network operations. Current networks are designed for the worst case impairments in transmission performance and the assigned spectrum is over-provisioned. In contrast, the FlexGrid networks can operate with the highest spectral efficiency and minimum bandwidth for the given traffic demand while meeting the minimum quality of transmission (QoT) requirement. Two primary focuses of our research are: (1) resource and spectrum allocation (RSA) for IP traffic over EONs, and (2) RSA for cross-domain optical networks. Previous work concentrates primarily on large file transfers within a single domain. Adding support for IP traffic changes the nature of the RSA problem: instead of choosing to accept or deny each request for network support, IP traffic is inherently elastic and thus lends itself to a bandwidth maximization formulation. We developed a number of algorithms that could be easily deployed within existing and new FlexGrid networks, leading to networks that better support scientific collaboration. Cross-domain RSA research is essential to support large-scale FlexGrid networks, since configuration information is generally not shared or coordinated across domains. The results presented here are in their early stages. They are technically feasible and practical, but still require coordination among organizations and equipment owners and a higher-layer framework for managing network requests.

42 ENGINEERING↗
DOE OSTI2019

A Comparison of Classical and Aggregation-Based Algebraic Multigrid Preconditioners for High-Fidelity Simulation of Wind Turbine Incompressible Flows

This paper presents a comparison of parallel strong scaling performance of classical and aggregation algebraic multigrid (AMG) preconditioners in the context of wind turbine simulations. Fluid motion is governed by the incompressible Navier--Stokes equations, discretized in space with control-volume finite elements and in time with an inexact projection scheme using an implicit integrator. A discontinuous-Galerkin sliding-mesh algorithm captures rotor motion. The momentum equations are solved with iterative Krylov methods, preconditioned by symmetric Gauss--Seidel (SGS) in Trilinos and $\ell_1$ SGS in hypre. The mass-continuity equation is solved with GMRES preconditioned by AMG and can account for the majority of simulation time. Reducing this continuity solve time is crucial. Wind turbine simulations present two unique challenges for AMG preconditioned solvers: the computational meshes include strongly anisotropic elements, and mesh motion requires matrix reinitialization and computation of preconditioners at each time step. Detailed timing profiles are presented and analyzed, and best practices are discussed for both classical and aggregation-based AMG. Results are presented for simulations of two different wind turbines with up to 6 billion grid points on two different computer architectures. For moving mesh problems that require linear-system reinitialization, the well-established strategy of amortizing preconditioner setup costs over a large number of time steps to reduce the solve time is no longer valid. Instead, results show that faster time to solution is achieved by reducing preconditioner setup costs at the expense of linear-system solve costs. Standard smoothed aggregation with Chebyshev relaxation was found to perform poorly when compared with classical AMG in terms of solve time and robustness. However, plain aggregation was comparable to classical AMG.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2019

Scalable Contour Tree Computation by Data Parallel Peak Pruning

As data sets grow to exascale, automated data analysis and visualization are increasingly important, to intermediate human understanding and to reduce demands on disk storage via in situ analysis. Trends in architecture of high performance computing systems necessitate analysis algorithms to make effective use of combinations of massively multicore and distributed systems. One of the principal analytic tools is the contour tree, which analyses relationships between contours to identify features of more than local importance. Unfortunately, the predominant algorithms for computing the contour tree are explicitly serial, and founded on serial metaphors, which has limited the scalability of this form of analysis. While there is some work on distributed contour tree computation, and separately on hybrid GPU-CPU computation, there is no efficient algorithm with strong formal guarantees on performance allied with fast practical performance. Furthermore, we report the first shared SMP algorithm for fully parallel contour tree computation, with formal guarantees of O(lg V lg t) parallel steps and O(V lg V) work for data with V samples and t contour tree supernodes, and implementations with more than 30× parallel speed up on both CPU using TBB and GPU using Thrust and up 70× speed up compared to the serial sweep and merge algorithm.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2024

Optimizing Distributed Training on Frontier for Large Language Models

Large language models (LLMs) have demonstrated remarkable success as foundational models, benefiting various downstream applications through fine-tuning. Loss scaling studies have demonstrated the superior performance of larger LLMs compared to their smaller counterparts. Nevertheless, training LLMs with billions of parameters poses significant challenges and requires considerable computational resources. For example, training a one trillion parameter GPT-style model on 20 trillion tokens requires a staggering 120 million exaflops. This research explores efficient distributed training strategies to extract this computation from Frontier, the world's first exascale supercomputer. We enable and investigate various model and data parallel training techniques, such as tensor parallelism, pipeline parallelism, and sharded data parallelism, to facilitate training a trillion-parameter model on Frontier. We empirically assess these techniques and their associated parameters to determine their impact on memory footprint, communication latency, and GPU's computational efficiency. We analyze the complex interplay among these techniques and find a strategy to combine them to achieve high throughput through hyperparameter tuning. We have identified efficient strategies for training large LLMs of varying sizes through empirical analysis and hyperparameter tuning. For 22 Billion, 175 Billion, and 1 Trillion parameters, we achieved GPU throughputs of 38.38%, 36.14%, and 31.96%, respectively. For the training of the 175 Billion parameter model and the 1 Trillion parameter model, we achieved 100% weak scaling efficiency on 1024 and 3072 Mi250X GPUs, respectively. We also achieved strong scaling efficiencies of 89% and 87% for these two models. We trained these models only tens of iterations instead of training till completion.

Yin, Junqi↗
DOE OSTI2018

Analytical Cost Metrics : Days of Future Past

As we move towards the exascale era, the new architectures must be capable of running the massive computational problems efficiently. Scientists and researchers are continuously investing in tuning the performance of extreme-scale computational problems. These problems arise in almost all areas of computing, ranging from big data analytics, artificial intelligence, search, machine learning, virtual/augmented reality, computer vision, image/signal processing to computational science and bioinformatics. With Moore’s law driving the evolution of hardware platforms towards exascale, the dominant performance metric (time efficiency) has now expanded to also incorporate power/energy efficiency. Therefore the major challenge that we face in computing systems research is: “how to solve massive-scale computational problems in the most time/power/energy efficient manner?”

97 MATHEMATICS AND COMPUTING↗
NASA NTRSDate not supplied

Implicit Thermochemical Nonequilibrium Compressible Flow Simulations on Unstructured Grids Using GPUs

As next-generation exascale-class systems arrive, existing software must be updated accordingly to effectively utilize these systems. For high concurrency and energy efficiency, many of these systems utilize GPU architectures. In this work, we present a CUDA C++ implementation of FUN3D's thermochemical nonequilibrium capability for turbulent flows. Efficiency is demonstrated at scale using the Summit system at the Oak Ridge Leadership Computing Facility which is representative of future exascale systems. This work enables faster, higher fidelity, and scale-resolving simulations of thermochemical nonequilibrium flows including reentry, hypersonics, and combustion.

CFD, GPU, HPC, Hypersonics, Chemistry↗
DOE OSTI2015

Quantifying Scheduling Challenges for Exascale System Software

The move towards high-performance computing (HPC) ap- plications comprised of coupled codes and the need to dra- matically reduce data movement is leading to a reexami- nation of time-sharing vs. space-sharing in HPC systems. In this paper, we discuss and begin to quantify the perfor- mance impact of a move away from strict space-sharing of nodes for HPC applications. Specifically, we examine the po- tential performance cost of time-sharing nodes between ap- plication components, we determine whether a simple coor- dinated scheduling mechanism can address these problems, and we research how suitable simple constraint-based opti- mization techniques are for solving scheduling challenges in this regime. Our results demonstrate that current general- purpose HPC system software scheduling and resource al- location systems are subject to significant performance de- ciencies which we quantify for six representative applica- tions. Based on these results, we discuss areas in which ad- ditional research is needed to meet the scheduling challenges of next-generation HPC systems.

C.1.2 [Multiprocessors]: Parallel Processors↗
DOE OSTI2015

A survey of CPU-GPU heterogeneous computing techniques

As both CPU and GPU become employed in a wide range of applications, it has been acknowledged that both of these processing units (PUs) have their unique features and strengths and hence, CPU-GPU collaboration is inevitable to achieve high-performance computing. This has motivated significant amount of research on heterogeneous computing techniques, along with the design of CPU-GPU fused chips and petascale heterogeneous supercomputers. In this paper, we survey heterogeneous computing techniques (HCTs) such as workload-partitioning which enable utilizing both CPU and GPU to improve performance and/or energy efficiency. We review heterogeneous computing approaches at runtime, algorithm, programming, compiler and application level. Further, we review both discrete and fused CPU-GPU systems; and discuss benchmark suites designed for evaluating heterogeneous computing systems (HCSs). Furthermore, we believe that this paper will provide insights into working and scope of applications of HCTs to researchers and motivate them to further harness the computational powers of CPUs and GPUs to achieve the goal of exascale performance.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2020

Toward Exascale: Overview of Large Eddy Simulations and Direct Numerical Simulations of Nuclear Reactor Flows with the Spectral Element Method in Nek5000

At the beginning of the last decade, Petascale supercomputers (i.e., computers capable of more than 1 petaFLOP) emerged. Now, at the dawn of exascale supercomputing, we provide a review of recent landmark simulations of portions of reactor components with turbulence-resolving techniques that this computational power has made possible. In fact, these simulations have provided invaluable insight into flow dynamics, which is difficult or often impossible to obtain with experiments alone. We focus on simulations performed with the spectral element method, as this method has emerged as a powerful tool to deliver massively parallel calculations at high fidelity by using large eddy simulation or direct numerical simulation. We also limit this paper to constant-property incompressible flow of a Newtonian fluid in the absence of other body or external forces, although the method is by no means limited to this class of flows. We briefly review the fundamentals of the method and the reasons it is compelling for the simulation of nuclear engineering flows. We review in detail a series of Petascale simulations, including the simulations of helical coil steam generators, fuel assemblies, and pebble beds. Even with Petascale computing, however, limitations for nuclear modeling and simulation tools remain. In particular, the size and scope of turbulence-resolving simulations are still limited by computing power and resolution requirements, which scale with the Reynolds number. In the final part of this paper, we discuss the future of the field, including recent advancements in emerging architectures such as GPUbased supercomputers, which are expected to power the next generation of high-performance computers.

computational fluid dynamics↗
DOE OSTI2014

A Pervasive Parallel Processing Framework for Data Visualization and Analysis at Extreme Scale

The evolution of the computing world from teraflop to petaflop has been relatively effortless, with several of the existing programming models scaling effectively to the petascale. The migration to exascale, however, poses considerable challenges. All industry trends infer that the exascale machine will be built using processors containing hundreds to thousands of cores per chip. It can be inferred that efficient concurrency on exascale machines requires a massive amount of concurrent threads, each performing many operations on a localized piece of data. Currently, visualization libraries and applications are based off what is known as the visualization pipeline. In the pipeline model, algorithms are encapsulated as filters with inputs and outputs. These filters are connected by setting the output of one component to the input of another. Parallelism in the visualization pipeline is achieved by replicating the pipeline for each processing thread. This works well for today’s distributed memory parallel computers but cannot be sustained when operating on processors with thousands of cores. Our project investigates a new visualization framework designed to exhibit the pervasive parallelism necessary for extreme scale machines. Our framework achieves this by defining algorithms in terms of worklets, which are localized stateless operations. Worklets are atomic operations that execute when invoked unlike filters, which execute when a pipeline request occurs. The worklet design allows execution on a massive amount of lightweight threads with minimal overhead. Only with such fine-grained parallelism can we hope to fill the billions of threads we expect will be necessary for efficient computation on an exascale machine.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2024

Making Uintah Performance Portable for Department of Energy Exascale Testbeds

To help ease ports to forthcoming Department of Energy (DOE) exascale systems, testbeds have been made available to select users. These testbeds are helpful for preparing codes to run on the same hardware and similar software as in their respective exascale systems. This paper describes how the Uintah Computational Framework, an open-source asynchronous many-task (AMT) runtime system, has been modified to be performance portable across the DOE Crusher, DOE Polaris, and DOE Sunspot testbeds in preparation for portable simulations across the exascale DOE Frontier and DOE Aurora systems. The Crusher, Polaris, and Sunspot testbeds feature the AMD MI250X, NVIDIA A100, and Intel PVC GPUs, respectively. This performance portability has been made possible by extending Uintah’s intermediate portability layer [18] to additionally support the Kokkos::HIP, Kokkos::OpenMPTarget, and Kokkos::SYCL back-ends. This paper also describes notable updates to Uintah’s support for Kokkos, which were required to make this extension possible. Results are shown for a challenging radiative heat transfer calculation, central to the University of Utah’s predictive boiler simulations. These results demonstrate single-source portability across AMD-, NVIDIA-, and Intel-based GPUs using various Kokkos back-ends.

Holmen, John↗
DOE OSTI2014

Reducing Communication in Algebraic Multigrid Using Additive Variants

Algebraic multigrid (AMG) has proven to be an effective scalable solver on many high performance computers. However, its increasing communication complexity on coarser levels has shown to seriously impact its performance on computers with high communication cost. Moreover, additive AMG variants provide not only increased parallelism as well as decreased numbers of messages per cycle but also generally exhibit slower convergence. Here we present various new additive variants with convergence rates that are significantly improved compared to the classical additive algebraic multigrid method and investigate their potential for decreased communication, and improved communication-computation overlap, features that are essential for good performance on future exascale architectures.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2020

Pre-exascale accelerated application development: The ORNL Summit experience

High-performance computing (HPC) increasingly relies on heterogeneous architectures to achieve higher performance. In the Oak Ridge Leadership Facility (OLCF), Oak Ridge, TN, USA, this trend continues as its latest supercomputer, Summit, entered production in early 2019. The combination of IBM POWER9 CPU and NVIDIA V100 GPU, along with a fast NVLink2 interconnect and other latest technologies, pushes system performance to a new height and breaks the exascale barrier by certain measures. Due to Summit's powerful GPUs and much higher GPU–CPU ratio, offloading to accelerators becomes a requirement for any application, which intends to effectively use the system. To facilitate navigating a complex landscape of competing heterogeneous architectures, a collection of applications from a wide spectrum of scientific domains is selected for early adoption on Summit. In this article, the experience and lessons learned are summarized, in the hope of providing useful guidance to address new programming challenges, such as scalability, performance portability, and software maintainability, for future application development efforts on heterogeneous HPC systems.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2025

Lessons Learned and Scalability Achieved When Porting Uintah to DOE Exascale Systems

A key challenge faced when preparing codes for Department of Energy (DOE) exascale systems was designing scalable applications for systems featuring hardware and software not yet available at leadership-class scale. With such systems now available, it is important to evaluate scalability of the resulting software solutions on these target systems. One such code designed with the exascale DOE Aurora and DOE Frontier systems in mind is the Uintah Computational Framework, an open-source asynchronous many-task (AMT) runtime system. To prepare for exascale, Uintah adopted a portable MPI+X hybrid parallelism approach using the Kokkos performance portability library (i.e., MPI+Kokkos). This paper complements recent work with additional details and an evaluation of the resulting approach on Aurora and Frontier. Results are shown for a challenging benchmark demonstrating interoperability of 3 portable codes essential to Uintah-related combustion research. These results demonstrate single-source portability across Aurora and Frontier with scaling characteristics shown to 3,072 Aurora nodes and 9,216 Frontier nodes. In addition to showing results run to new scales on new systems, this paper also discusses lessons learned through efforts preparing Uintah for exascale systems.

Holmen, John [ORNL] (ORCID:0000000259342641)↗