Engineering PapersSearch

SEARCH · Engineering Papers

Results for “Exascale computing”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 19 records

Exascale Computing and Data Handling: Challenges and Opportunities for Weather and Climate Prediction

The emergence of exascale computing and artificial intelligence offer tremendous potential to significantly advance Earth system prediction capabilities. However, enormous challenges must be overcome to adapt models and prediction systems to use these new technologies effectively. A 2022 WMO report on exascale computing recommends “urgency in dedicating efforts and attention to disruptions associated with evolving computing technologies that will be increasingly difficult to overcome, threatening continued advancements in weather and climate prediction capabilities.” Further, the explosive growth in data from observations, model and ensemble output, and postprocessing threatens to overwhelm the ability to deliver timely, accurate, and precise information needed for decision-making. Artificial intelligence (AI) offers untapped opportunities to alter how models are developed, observations are processed, and predictions are analyzed and extracted for decision-making. Given the extraordinarily high cost of computing, growing complexity of prediction systems, and increasingly unmanageable amount of data being produced and consumed, these challenges are rapidly becoming too large for any single institution or country to handle. This paper describes key technical and budgetary challenges, identifies gaps and ways to address them, and makes a number of recommendations.

Atmosphere

Approaches for the Simulation of Coupled Processes in Evolving Fractured Porous Media Enabled by Exascale Computing

Models have historically represented fractured porous media with continuum descriptions that characterize the media using bulk parameters. The impact of small-scale features is not captured in these models, although they may be controlling the performance of subsurface applications. Pore-scale models can simulate processes in small-scale features by representing the pore space geometry explicitly but are computationally expensive for large domains. The alternative multiscale approach entails the combination of pore-scale and continuum-scale descriptions in a single framework. We use Chombo-Crunch, a computational capability that discretizes complex geometries with an adaptive, embedded boundary method to contrast these two approaches. Chombo-Crunch takes advantage of recent computational performance and memory bandwidth improvements resulting from the emergence of exascale computing resources. These combined improvements enable the efficient simulation of reactive transport in fractured media with a high degree of fidelity and the ability to capture the control small-scale processes exert on the overall medium evolution.

42 ENGINEERING

Ginkgo - A math library designed to accelerate Exascale Computing Project science applications

Large-scale simulations require efficient computation across the entire computing hierarchy. A challenge of the Exascale Computing Project (ECP) was to reconcile highly heterogeneous hardware with the myriad of applications that were required to run on these supercomputers. Mathematical software forms the backbone of almost all scientific applications, providing efficient abstractions and operations that are crucial to harness the performance of computing systems. Ginkgo is one such mathematical software library, nurtured by ECP, providing high-performance, user-friendly, and performance portable interfaces for applications in ECP and beyond. In this paper, we elaborate on Ginkgo’s philosophy of high-performance software that is sustainable, reproducible, and easy to use. We showcase the wide feature set of solvers and preconditioners available in Ginkgo and the central concepts involved in their design. We elaborate on four different ECP software integrations: MFEM, PeleLM + SUNDIALS, XGC, and ExaSGD that use Ginkgo to accelerate their science runs. Performance studies of different problems from these applications highlight the effectiveness of Ginkgo and the benefits incurred by these ECP applications.

Cojean, Terry

Exabiome: Advancing Microbial Science through Exascale Computing

The Exabiome project seeks to improve the understanding of microbiomes through the development of methods for accelerating metagenomic science using exascale computing. This article gives an overview of scientific impact of the three components of the project: metagenome assembly, protein family detection, and comparative analysis of metagenomes. Exabiome developed MetaHipMer, the only metagenome assembler capable of scaling to full exascale systems. MetaHipMer has enabled ground-breaking assemblies on the Frontier supercomputer, with many scientific benefits, such as the discovery of rare species and viral genomes. To investigate protein families, Exabiome developed two exascale tools, PASTIS and HipMCL. Together, these can utilize exascale resources to understand the functional diversity of billions of dark matter proteins and novel protein families. For comparative analysis, Exabiome developed kmerprof, a tool that can be used to compare huge metagenomes for many different scientific purposes, for example, grouping human microbiomes according to body location.

59 BASIC BIOLOGICAL SCIENCES

Shadow of the Future: Developing Trust and Software within the Exascale Computing Project

Collaboration and team science are emerging areas of interest in software production. Historically, multi-institutional research collaborations are difficult to initiate and maintain, negatively impacting communication, negotiation, and dialogue between industry, government, and academic researchers. The Exascale Computing Project (ECP), a massive, multi-team, high-stakes initiative, facilitated broader research collaboration under a shared funding structure and extended timeline to support scientific discovery. Here, we conducted interviews with ECP teams, representing a variety of domain specialties, research institutions, and programming backgrounds. Using thematic analysis, we assessed how ECP’s structure created an environment of increased trust among projects and how software shared between teams facilitated sustained collaboration. We found that the expectation of future collaboration, i.e., the shadow of the future, greatly enhanced trust among teams and the quality of scientific software produced. Based on our findings within ECP projects, we connect to the existing literature on trust in software engineering and share recommendations for sustainable multi-institutional collaboration and shared best software practices.

Exascale computing project

AMReX and pyAMReX: Looking beyond the exascale computing project

AMReX is a software framework for the development of block-structured mesh applications with adaptive mesh refinement (AMR). AMReX was initially developed and supported by the AMReX Co-Design Center as part of the U.S. DOE Exascale Computing Project (ECP), and is continuing to grow post-ECP. In addition to adding new functionality and performance improvements to the core AMReX framework, we have also developed a Python binding, pyAMReX, that provides a bridge between AMReX-based application codes and the data science ecosystem. pyAMReX provides zero-copy application GPU data access for AI/ML, in situ analysis and application coupling, and enables rapid, massively parallel prototyping. In this paper we review the overall functionality of AMReX and pyAMReX, focusing on new developments, new functionality, and optimizations of key operations. We also summarize capabilities of ECP projects that used AMReX and provide an overview of new, non-ECP applications.

Myers, Andrew

Establishing model credibility for process-microstructure-property relationships in additive manufacturing using exascale computing

Additive Manufacturing (AM) of alloys holds significant promise as a disruptive technology in various industries, yet its adoption is often hindered by challenges in achieving consistent part quality. These issues are primarily due to the complex process-microstructure-property (PSP) relationships inherent to AM. Computational models can greatly aid in understanding these relationships, but their widespread impact and adoption has been limited by a lack of validated, open-source, and computationally efficient PSP modeling frameworks and hardware limitations. Here, this study leverages the ExaAM software suite and data from the AMBench-2018 series of laser powder bed fusion (LPBF) benchmark experiments to perform a comprehensive model assessment, including verification, validation, sensitivity analysis, and uncertainty quantification. The RADICAL-EnTK workflow manager was used to perform an ensemble of heat transport, solidification, and mechanical response simulations on the exascale computer Frontier, considering uncertainties in critical model inputs such as laser spot size and nucleation parameters, and consisting of 125 explicit grain structure simulations and 7875 crystal plasticity simulations. For a selected location within the Inconel 625 AMBench-2018 test artifact, sensitivity analysis and uncertainty quantification were performed using the predicted distributions of grain structure and mechanical properties. Qualitative agreement was found between the predicted grain size and texture and the observed AMBench-2018 microstructure, the mean predicted yield stress was within 5% of the experimental measurement mean, and the mean predicted engineering stress at 5% strain was within 10% of the experimental measurement mean. The insights gained from development and validation of the ExaAM PSP modeling framework will help guide future directions for enhancing the credibility and reliability of PSP models in AM, thereby accelerating the adoption of AM technologies in various industries.

Additive manufacturing

Symbolic construction of the chemical Jacobian of quasi-steady state (QSS) chemistries for Exascale computing platforms

The Quasi-Steady State Approximation (QSSA) can be an effective tool for reducing the size and stiffness of chemical mechanisms for implementation in computational reacting flow solvers. However, for many applications, the resulting model still requires implicit methods for efficient time integration. Here, in this paper, we outline an approach to formulating the QSSA reduction that is coupled with a strategy to generate C++ source code to evaluate the net species production rates, and the chemical Jacobian. The code-generation component employs a symbolic approach enabling a simple and effective strategy to analytically compute the chemical Jacobian. For computational tractability, the symbolic approach needs to be paired with common subexpression elimination which can negatively affect memory usage. Several solutions are outlined and successfully tested on a 3D multipulse ignition problem, thus allowing portable application across chemical model sizes and GPU capabilities. The implementation of the proposed method is available at https://github.com/AMReX-Combustion/PelePhysics under an open-source license.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH

Small-scale properties from exascale computations of turbulence on a $\mathbf{32\,768^3}$ periodic cube

To study the physics of small-scale properties of homogeneous isotropic turbulence at increasingly high Reynolds numbers, direct numerical simulation results have been obtained for forced isotropic turbulence at Taylor-scale Reynolds number R λ = 2500 on a 32 768 3 three-dimensional periodic domain using a GPU pseudo-spectral code on a 1.1 exaflop GPU supercomputer (Frontier). These simulations employ the multi-resolution independent simulation (MRIS) technique (Yeung & Ravikumar 2020, Phys. Rev. Fluids, vol. 5, 110517) where ensemble averaging is performed over multiple short segments initiated from velocity fields at modest resolution, and subsequently taken to higher resolution in both space and time. Reynolds numbers are increased by reducing the viscosity with the large-scale forcing parameters unchanged. Although MRIS segments at the highest resolution for each Reynolds number last for only a few Kolmogorov time scales, small-scale physics in the dissipation range is well captured – for instance, in the probability density functions and higher moments of the dissipation rate and enstrophy density, which appear to show monotonic trends persisting well beyond the Reynolds number range in prior works in the literature. Attainment of range of length and time scales consistent with classical scaling also reinforces the potential utility of the present high-resolution data for studies of short-time-scale turbulence physics at high Reynolds numbers where full-length simulations spanning many large-eddy time scales are still not accessible. A single snapshot of the 32 768 3 data is publicly available for further analyses via the Johns Hopkins Turbulence Database.

intermittency

Distributed Multi-GPU Community Detection on Exascale Computing Platforms

Community detection is a fundamental operation in graph mining, and by uncovering hidden structures and patterns within complex systems it helps solve fundamental problems pertaining to social networks, such as information diffusion, epidemics, and recommender systems. Scaling graph algorithms for massive networks becomes challenging on modern distributed-memory multi-GPU (Graphics Processing Unit) systems due to limitations such as irregular memory access patterns, load imbalances, higher communication-computation ratios, and cross-platform support. We present a novel algorithm HiPDPL-GPU (distributed parallel Louvain) to address these challenges. We conduct experiments involving different partitioning techniques to achieve optimized performance of HiPDPL-GPU on the two largest supercomputers: Frontier and Summit. Remarkably, HiPDPL-GPU processes a graph with 4.2 billion edges in less than 3 minutes using 1024 GPUs. Qualitatively performance of HiPDPL-GPU is similar or better compared to other state-of-the-art CPU- and GPU-based implementations. While prior GPU implementations have predominantly employed CUDA, our first-of-its-kind implementation for community detection is cross-platform, accommodating both AMD and NVIDIA GPUs.

graph algorithms, high performance comptuing

Bridging the time scale in exascale computing of chemical systems (Final Technical Report)

This report summarizes the work carried out with support of the United States Department of Energy under Award DE-SC0019441. The theme of this project was to develop and apply methods that allowed for the acceleration of atomistic calculations, particularly in challenging areas such as multiphase systems, electrified interfaces, uncertainty estimation, and applications requiring chemical accuracy, which tend to be applications where simulation time is severely bottlenecked by the computational time requirements. Much of the focus was on the application of emerging machine-learning methodologies, although a wide range of methodologies were employed. This report has two major sections. The first focuses on the methodological advances themselves. Within this part, we report a number of major advances, a few examples of which are described here. We report the first machine-learning scheme for the acceleration of electronically grand-canonical calculations (that is, those applicable to electrochemistry). We report new methods of performing transfer learning, in which physics-based priors can be used to provide predictions, often with uncertainty estimates, of images well outside of training sets; we also offer ways to fine-tune these transfer-learning models. We provide a new systematic means to generate and apply minimal training data sets to very large (10,000’s of atoms) systems, with only small training sets appropriate for electronic structure. We developed new methodologies to integrate surface vibrations into surface adsorption calculations. We made advances to the applicability of diffusion Monte Carlo methods to allow (learned) force prediction, finite-size error correction, and force-free means of searching for transition states. We integrated machine-learned atomistic predictions into mechanism generation codes. Additionally, we released new software including AmpTorch, a modernized version of our original atomistic machine-learning code Amp. The second part of this report focuses on the scientific applications that accompanied, and were often enabled by, the methodological advances described earlier. A few examples follow, but full details are in the individual chapters of the report. For example, we developed a general theory of phonon-induced friction on molecular adsorbates. We showed fundamentally how solvent influences the adsorption and desorption process and how it differs from the processes typically involved at the solid–gas interface, making aqueous-phase and electrocatalysis different from traditional thermocatalysis. We examined how metal–insulator and magnetic transitions can be probed, and accelerated exciton dynamics via Frenkel Hamiltonian parameters. We showed that the nearsighted force-training approach, developed within this project, can predict both the stability and reactivity of large nanoparticles, and can also lead to insights on catalyst coverage on binding energies and entropies. These applied studies, which generally integrated with our method development, allowed us to push forward the theoretical understanding of several reaction classes.

08 HYDROGEN

A multiphysics coupling framework for exascale simulation of fracture evolution in subsurface energy applications

Predicting the evolution of fractured media is challenging due to coupled thermal, hydrological, chemical and mechanical processes that occur over a broad range of spatial scales, from the microscopic pore scale to field scale. We present a software framework and scientific workflow that couples the pore scale flow and reactive transport simulator Chombo-Crunch with the field scale geomechanics solver in GEOS to simulate fracture evolution in subsurface fluid-rock systems. This new multiphysics coupling capability comprises several novel features. An HDF5 data schema for coupling fracture positions between the two codes is employed and leverages the coarse resolution of the GEOS mechanics solver which limits the size of data coupled, and is, thus, not taxed by data resulting from the high resolution pore scale Chombo-Crunch solver. The coupling framework requires tracking of both before and after coarse nodal positions in GEOS as well as the resolved embedded boundary in Chombo-Crunch. We accomplished this by developing an approach to geometry generation that tracks the fracture interface between the two different methodologies. The GEOS quadrilateral mesh is converted to triangles which are organized into bins and an accessible tree structure; the nodes are then mapped to the Chombo representation using a continuous signed distance function that determines locations inside, on and outside of the fracture boundary. The GEOS positions are retained in memory on the Chombo-Crunch side of the coupling. The time stepping cadence for coupled multiphysics processes of flow, transport, reactions and mechanics is stable and demonstrates temporal reach to experimental time scales. The approach is validated by demonstration of 9 days of simulated time of a core flood experiment with fracture aperture evolution due to invasion of carbonated brine in wellbore-cement and sandstone. We also demonstrate usage of exascale computing resources by simulating a high resolution version of the validation problem on OLCF Frontier.

97 MATHEMATICS AND COMPUTING

HPDR: High-Performance Portable Scientific Data Reduction Framework

The rapid growth in scientific data generation is outpacing advancements in computing systems necessary for efficient storage, transfer, and analysis, particularly in the context of exascale computing. With the deployment of first-generation exascale computing systems and next-generation experimental facilities, this gap is widening and necessitates effective data reduction techniques to manage enormous data volumes. Over the past decade, various data reduction methods, including lossless compression, error-controlled lossy compression, and data refactoring, have been developed to accelerate I/O in scientific workflows. Despite significant reductions in data volume, these methods introduce considerable computational overhead, which can become the new bottleneck in data processing. To mitigate this, GPU-accelerated data reduction algorithms have been introduced. However, challenges remain in their integration into exascale workflows, including limited portability across different GPU architectures, substantial memory transfer overhead, and reduced scalability on dense multi-GPU systems. To address these challenges, we propose HPDR, a high-performance and portable data reduction framework. HPDR is designed to enable the execution of state-of-the-art reduction algorithms across diverse processor architectures while reducing memory transfer overhead to 2.3 % of the original, resulting in up to 3.5× faster throughput compared to existing solutions. It also achieves up to 96% of the theoretical speedup in multi-GPU settings. In addition, evaluations on accelerating I/O operations at scale up to 1,024 nodes of the Frontier supercomputer demonstrate that HPDR can achieve up to 103 TB/s reduction throughput, providing up to 4× acceleration in parallel I/O performance compared to existing data reduction routines. This work highlights the potential of HPDR to significantly enhance data reduction efficiency in exascale computing environments.

Chen, Jieyang [University of Oregon]

Transforming Science Through Software: Improving While Delivering 100×

The U.S. Department of Energy (DOE) Exascale Computing Project (ECP) funded the development of new (and the transformation of important existing) applications, libraries, and tools that realized improvement in performance and capabilities of often 100 times or more on emerging exascale computers. This exceptional gain inspired the title of this special issue: Transforming Science through Software: Improving while delivering 100X. The term 100X refers to advancing capabilities in modeling, simulation, and analysis by a factor of 100 or more using some combination of new algorithms, optimization techniques, software libraries, and programming models, coupled with the next generation of hardware for high-performance computing (HPC). The papers in this issue share experiences with the practice and science of scientific software development, with an emphasis on developing a coherent, portable, and sustainable HPC software ecosystem for next-generation computational science. Finally, we hope to foster expanded community efforts related to the fundamental role of sustainable scientific software ecosystems in advancing the computing sciences.

97 MATHEMATICS AND COMPUTING

The ECP ALPINE project: In situ and post hoc visualization infrastructure and analysis capabilities for exascale

A significant challenge on an exascale computer is the speed at which we compute results exceeds by many orders of magnitude the speed at which we save these results. Therefore the Exascale Computing Project (ECP) ALPINE project focuses on providing exascale-ready visualization solutions including in situ processing. In situ visualization and analysis runs as the simulation is run, on simulations results are they are generated avoiding the need to save entire simulations to storage for later analysis. The ALPINE project made post hoc visualization tools, ParaView and VisIt, exascale ready and developed in situ algorithms and infrastructures. The suite of ALPINE algorithms developed under ECP includes novel approaches to enable automated data analysis and visualization to focus on the most important aspects of the simulation. Many of the algorithms also provide data reduction benefits to meet the I/O challenges at exascale. ALPINE developed a new lightweight in situ infrastructure, Ascent.

97 MATHEMATICS AND COMPUTING

Taking the MPI standard and the open MPI library to exascale

The Open MPI for Exascale (OMPI-X) project was one of two in the Exascale Computing Project (ECP) focused on advancing the MPI ecosystem. The OMPI-X team worked with other MPI Forum members to champion several important features for inclusion in the MPI 4.0, 4.1, and upcoming 5.0 MPI standard versions, in support of the needs of exascale applications and systems. The team also worked with the larger Open MPI community to bring implementations of these new features and other enhancements into Open MPI, one of the leading open-source implementations of the MPI interface. Here, this paper describes the motivation for the work of the OMPI-X project in the context of exascale computing needs, the nature of the resulting new capabilities in the MPI standard, and how they were implemented in the Open MPI library. Features include improved support for “MPI + X” programming models through partitioned communications and support for user-level threading, sessions, fault tolerance through the user-level fault mitigation (ULFM) and Reinit models, and other features. We also discuss enhancements to Open MPI providing improved performance and scalability for existing features, such as collective operations, one-sided operations, support for the Slingshot-11 interconnect of the initial exascale systems, and how the OMPI-X team worked to improve quality assurance for the Open MPI library, particularly on platforms of interest to the Department of Energy community.

97 MATHEMATICS AND COMPUTING