Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Exascale computing”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,045 records · Page 58

Reconstructing High Resolution ESM Data Through a Novel Fast Super Resolution Convolutional Neural Network (FSRCNN)

In this work, we present the first application of a fast super resolution convolutional neural network (FSRCNN) based approach for downscaling earth system model (ESM) simulations. Unlike other SR approaches, FSRCNN uses the same input feature dimensions as the low resolution input. This allows it to have smaller convolution layers, avoiding over-smoothing, and reducing computational costs. We adapt the FSRCNN to improve reconstruction on ESM data, we term the FSRCNN-ESM. We use high-resolution (~0.25°) monthly averaged model output of five surface variables over North America from the US Department of Energy's Energy Exascale Earth System Model's control simulation. These high-resolution and corresponding coarsened low-resolution (~1°) pairs of images are used to train the FSRCNN-ESM and evaluate its use as a downscaling approach. We find that FSRCNN-ESM outperforms FSRCNN and other super-resolution methods in reconstructing high resolution images producing finer spatial scale features with better accuracy for surface temperature, surface radiative fluxes, and precipitation.

58 GEOSCIENCES↗

JACC.shared: Leveraging HPC Metaprogramming and Performance Portability for Computations That Use Shared Memory GPUs

In this work, we present JACC.shared, a new feature of Julia for ACCelerators (JACC), which is the performanceportable and metaprogramming model of the just-in-time and LLVM-based Julia language. This new feature allows JACC applications to leverage the high-performance computing (HPC) capabilities of high-bandwidth, on-chip GPU memory. Historically, exploiting high-bandwidth, shared-memory GPUs has not been a priority for high-level programming solutions. JACC.shared covers that gap for the first time, thereby providing a highlevel, portable, and easy-to-use solution for programmers to exploit this memory and supporting all current major accelerator architectures. Well-known HPC and AI workloads, such as multi/hyperspectral imaging and AI convolutions, have been used to evaluate JACC.shared on two exascale GPU architectures hosted by some of the most powerful US Department of Energy supercomputers: Perlmutter (NVIDIA A100) and Frontier (AMD MI250X). The performance evaluation reports speedup of up to 3.5× by adding only one line of code to the base codes, thus providing important accelerators in a simple, portable, and transparent way and elevating the programming productivity and performance-portability capabilities for Julia/JACC HPC, AI, and scientific applications.

Valero Lara, Pedro [ORNL] (ORCID:0000000214794310)↗

Development of a CPU/GPU portable software library for Lagrangian–Eulerian simulations of liquid sprays

The Lagrangian–Eulerian method is widely used for simulations of fuel sprays in turbulent combustion because of the advantage of treating the spray droplets as discrete points. One challenge of the Lagrangian–Eulerian method is the intense computational requirement when tracking the large number of Lagrangian particles needed for high fidelity. We have developed a performance-portable library, Grit, to track the Lagrangian particles in parallel on central processing unit (CPU) and graphics processing unit (GPU) accelerated high performance computing (HPC) architectures. Grit is a C++ library which employs Message Passing Interface (MPI) for distributed memory parallelism and Kokkos programming model for on-node shared memory parallelism with performance portability across different architectures of GPUs and multi-core/manycore CPUs. The parallel algorithms, key parallel kernels, and their performances on the pre-exascale supercomputer, Summit, are presented. Grit is coupled with a direct numerical simulation (DNS) solver, S3D, for multiphase simulations. A conservative formulation has been developed and implemented in Grit for phase coupling with thermodynamic consistency. The formulation separates the conservation of mass, momentum and energy from the physical models to prevent accidental violation of conservation laws due to inconsistent models. The formulation also enforces consistent definitions of enthalpies of the fuel for both phases and the latent heat of evaporation. Finally, simulations of turbulent particle-laden flow and the evaporation of dilute turbulent spray jet are performed to verify the software implementation, and to demonstrate the scalability of Grit for large-scale multiphase simulations.

97 MATHEMATICS AND COMPUTING↗

Assessment of ESM Readiness Level for Exascale HPC

Advancement of Earth System Models (ESMs) is becoming increasingly challenging due to a confluence of factors including increasing model complexity – to more fully represent the earth system, increasing spatial resolution - to achieve higher accuracy by resolving fine-scale dynamical to physical, biological, and chemical processes and their interaction, increasing ensemble size - to more accurately represent predictive uncertainty, and increased computing requirements – to enable more accurate and timely weather predictions and climate projections for societal benefit. The belief by many that computing will take care of itself is no longer valid given the disruptive changes in HPC that are driving up the cost of computing, increasing the difficulty of using emerging HPC effectively, and exposing limits in parallelism, portability and scalability of the ESM applications themselves.

54 ENVIRONMENTAL SCIENCES↗

Simulations of the shock-driven Kelvin–Helmholtz instability in inclined gas curtains

In this paper, we present simulation results for the two-dimensional, shock-driven Kelvin–Helmholtz instability. Simulations are performed with a Mach 2.0 shock propagating through a finite-thickness curtain of gas inclined at an angle α0=30° with respect to the shock plane. After the passage of the shock, the gas curtain is accelerated along its axis. A perturbation develops due to shock reflection near the lower wall, and a Kelvin–Helmholtz instability forms near the vertical center of the curtain. This is the first known numerical reproduction of these phenomena that have previously been observed in experiments with an inclined cylindrical gas column. The effects of varying Mach number and column width were explored in detail to complement experimental data. Additionally, the dependence of the Kelvin–Helmholtz wavelength on Mach number closely matches the relationship observed in experiments. This supports the notion that the observed instability is effectively two-dimensional and inviscid (like classical Kelvin–Helmholtz). The growth rate of the perturbations in the gas curtain was also found to be similar for different Mach numbers. The perturbation at the curtain foot, previously unreported in experiments, was found to have a similar relationship to Mach number as the Kelvin–Helmholtz instability. Both perturbation wavelengths are found to be proportional to layer width. Simulations were performed with the fast interfaces and transport in the atmosphere, an exascale ready, graphics processing unit-accelerated compressible flow solver developed at the University of New Mexico.

42 ENGINEERING↗

Decrease time-to-solution through improved linear-system setup and solve.

The goal of the ExaWind project is to enable predictive simulations of wind farms composed of many MW-scale turbines situated in complex terrain. Predictive simulations will require computational fluid dynamics (CFD) simulations for which the mesh resolves the geometry of the turbines, and captures the rotation and large deflections of blades. Whereas such simulations for a single turbine are arguably petascale class, multi-turbine wind farm simulations will require exascale-class resources.

17 WIND ENERGY↗

Decrease time-to-solution through improved linear-system setup and solve (Milestone Report)

The goal of the ExaWind project is to enable predictive simulations of wind farms composed of many MW-scale turbines situated in complex terrain. Predictive simulations will require computational fluid dynamics (CFD) simulations for which the mesh resolves the geometry of the turbines, and captures the rotation and large deflections of blades. Whereas such simulations for a single turbine are arguably petascale class, multi-turbine wind farm simulations will require exascale-class resources. We describe in this report our efforts to decrease the setup and solution time for the mass-continuity Poisson system with respect to the benchmark timing results reported in FY18 Q1. In particular, we investigate improving and evaluating two types of algebraic multigrid (AMG) preconditioners: Classical Ruge-Stfiben AMG (C-AMG) and smoothed-aggregation AMG (SA-AMG), which are implemented in the Hypre and Trilinos/MueLu software stacks, respectively.

17 WIND ENERGY↗

$\mathrm{RADICAL}$-Pilot and $\mathrm{PMIx}$/$\mathrm{PRRTE}$: Executing Heterogeneous Workloads at Large Scale on Partitioned $\mathrm{HPC}$ Resources

Execution of heterogeneous workflows on high-performance computing (HPC) platforms present unprecedented resource management and execution coordination challenges for runtime systems. Task heterogeneity increases the complexity of resource and execution management, limiting the scalability and efficiency of workflow execution. Re-source partitioning and distribution of tasks execution over portioned re-sources promises to address those problems but we lack an experimental evaluation of its performance at scale. Here this paper provides a performance evaluation of the Process Management Interface for Exascale (PMIx) and its reference implementation PRRTE on the leadership-class HPC plat-form Summit, when integrated into a pilot-based runtime system called RADICAL-Pilot. We partition resources across multiple PRRTE Distributed Virtual Machine (DVM) environments, responsible for launching tasks via the PMIx interface. We experimentally measure the work-load execution performance in terms of task scheduling/launching rate and distribution of DVM task placement times, DVM startup and termination overheads on the Summit leadership-class HPC platform. Integrated solution with PMIx/PRRTE enables using an abstracted, standardized set of interfaces for orchestrating the launch process, dynamic process management and monitoring capabilities. It extends scaling capabilities allowing to overcome a limitation of other launching mechanisms (e.g., JSM/LSF). Explored different DVM setup configurations provide insights on DVM performance and a layout to leverage it. Our experimental results show that heterogeneous workload of 65,500 tasks on 2048 nodes, and partitioned across 32 DVMs, runs steady with resource utilization not lower than 52%. While having less concurrently executed tasks resource utilization is able to reach up to 85%, based on results of heterogeneous workload of 8200 tasks on 256 nodes and 2 DVMs.

97 MATHEMATICS AND COMPUTING↗

Keeping LAMMPS cutting edge

Since its inception 30 years ago, LAMMPS has grown to be a world-class molecular dynamics code and a cornerstone of computational materials science research. This project aimed to keep LAMMPS at the forefront of molecular dynamics simulations by adapting LAMMPS to the latest developments in machine learning technology and hardware. Initially, the project set out to provide a unified implementation of active learning for efficient training data generation in LAMMPS, but the research trajectory pivoted to address more immediate and impactful opportunities. On the hardware side, recent record-breaking molecular dynamics simulations were developed on the Cerebras wafer-scale AI chip, and this project has developed an interface between LAMMPS and the hardware-specific molecular dynamics code to accelerate and simplify development and user adoption. On the software side, PyTorch’s Ahead-of-Time (AOT) compilation features promised increased performance for state-of-the-art equivariant neural network potentials, and this project laid the groundwork for their adoption in LAMMPS, resulting in a nearly 20x acceleration in extreme cases. Combined with a comprehensive benchmark study of LAMMPS across all current exascale systems, this project has reinforced LAMMPS’s role as a versatile, high-performance tool for current and future materials science applications.

36 MATERIALS SCIENCE↗

Determination of performance characteristics of scientific applications on IBM Blue Gene/Q

The IBM Blue Gene®/Q platform presents scientists and engineers with a rich set of hardware features such as 16 cores per chip sharing a Level 2 cache, a wide SIMD (single-instruction, multiple-data) unit, a five-dimensional torus network, and hardware support for collective operations. Especially important is the feature related to cores that have four “hardware threads,” which makes it possible to hide latencies and obtain a high fraction of the peak issue rate from each core. All of these hardware resources present unique performance-tuning opportunities on Blue Gene/Q. We provide an overview of several important applications and solvers and study them on Blue Gene/Q using performance counters and Message Passing Interface profiles. We also discuss how Blue Gene/Q tools help us understand the interaction of the application with the hardware and software layers and provide guidance for optimization. Furthermore, on the basis of our analysis, we discuss code improvement strategies targeting Blue Gene/Q. Information about how these algorithms map to the Blue Gene® architecture is expected to have an impact on future system design as we move to the exascale era.

97 MATHEMATICS AND COMPUTING↗

LHC Event Generation in the Exascale Era

MCFM is a dedicated Monte-Carlo simulation program for collider phenomenology at highest energies. Designed during the Tevatron era, it has successfully incorporated the latest developments needed for LHC precision calculations and remained on the forefront of collider phenomenology. The Fortran code includes interfaces to modern PDF and loop reduction libraries but has been unchanged structurally compared to the earlier versions. Parallel computing has been enabled using OpenMP and MPI. MCFM provides numerically highly stable one-loop amplitudes and superior phase-space efficiency, leading to excellent performance in NXLO calculations using jettiness or qT subtraction techniques for IR regularization.

Campbell, John [Fermilab]↗

Asynchronous Navier-Stokes Solver on 3D Unstructured Grids for the Exascale Era

This project has developed multiple fluid dynamics solvers for complex 3D flows using fully asynchronous distributed-memory task-parallel algorithms on top of the Charm++ runtime system. The algorithms solve the Euler or Navier-Stokes equations of compressible flows using unstructured tetrahedron meshes with optional solution-adaptive mesh-, and polynomial-order refinement. We have demonstrated excellent strong scaling up 50K compute cores and the benefits of Charm++’s automatic load balancing.

97 MATHEMATICS AND COMPUTING↗

Combining Physics and Machine Learning for the Next Generation of Molecular Simulation

Simulating molecules and atomic systems at quantum accuracy is a grand challenge for science in the 21 st century. Quantum-accurate simulations would enable the design of new medicines and the discovery of new materials. The defining problem in this challenge is that quantum calculations on large molecules, like proteins or DNA, are fundamentally impossible with current algorithms. In this work, we explore a range of different methods that aim to make large, quantum-accurate simulations possible. We show that using advanced classical models, we can accurately simulate ion channels, an important biomolecular system. We show how advanced classical models can be implemented in an exascale-ready software package. Lastly, we show how machine learning can learn the laws of quantum mechanics from data and enable quantum electronic structure calculations on thousands of atoms, a feat that is impossible for current algorithms. Altogether, this work shows that combining advances in physics models, computing, and machine learning, we are moving closer to the reality of accurately simulating our molecular world.

74 ATOMIC AND MOLECULAR PHYSICS↗

Simulation of ENSO Teleconnections to Precipitation Extremes over the United States in the High-Resolution Version of E3SM

Abstract We evaluate the simulated teleconnection of El Niño–Southern Oscillation (ENSO) to winter season precipitation extremes over the United States in a long (98 years) 1950 control high-resolution version (HR; 25-km nominal atmosphere model horizontal resolution) of the U.S. Department of Energy’s (DOE) Energy Exascale Earth System Model version 1 (E3SMv1). The model bias and spatial pattern of ENSO teleconnections to mean and extreme precipitation in HR overall are similar to the low-resolution model’s (LR; 110 km) historical simulation (four-member ensemble, 1925–59). However, over the southeastern United States (SE-U.S.), HR produces stronger El Niño–associated extremes, reducing LR’s model bias. Both LR and HR produce weaker than observed increase in storm track activity during El Niño events there, but HR improves the ENSO-associated variability of moisture transport over SE-U.S. During El Niño, stronger vertical velocities in HR produce stronger large-scale precipitation, causing larger latent heating of the troposphere that pulls in more moisture from the Gulf of Mexico into the SE-U.S. This positive feedback also contributes to the stronger mean and extreme precipitation response in HR. Over the Pacific Northwest, LR’s bias of stronger than observed La Niña associated extremes is amplified in HR. Both models simulate stronger than observed moisture transport from the Pacific Ocean into the region during La Niña years. The amplified HR bias there is due to stronger orographically driven vertical updrafts that create stronger large-scale precipitation, despite weaker La Niña–induced storm track activity. Significance Statement New high-resolution Earth system models (ESMs) solve mathematical equations of fluid flow at much smaller spatial scales than prevalent ESMs, and thus are prohibitively expensive to compute. However, they can be useful for simulating accurate details of regional climate extremes that are driven by naturally occurring climate oscillations like El Niño–Southern Oscillation (ENSO). Here, we evaluate the simulation of ENSO-driven precipitation extremes over the United States in the high-resolution version of the U.S. Department of Energy’s new Energy Exascale Earth System Model version 1. We find that the high-resolution model improves upon its low-resolution counterpart over the southeastern United States by producing a better transport of moisture into the region from the Gulf of Mexico during El Niño. Over the U.S. Pacific Northwest, the high-resolution model simulates the atmospheric flow in more detail over the complex mountainous terrain. However, it also brings in more moisture from the Pacific Ocean just like the low-resolution model. This causes it to produce precipitation extremes during La Niña years there that are stronger than that observed in the real world.

54 ENVIRONMENTAL SCIENCES↗

Report on local data recovery approaches suitable for weather and climate prediction (Deliverable 1.3) (V.1.0)

Numerical weather and climate prediction rates as one of the scientific applications whose accuracy improvements greatly depend on the growth of the available computing power. As the number of cores in top computing facilities pushes into the millions, increasing average frequency of hardware and software failures forces users to review their algorithms and systems in order to protect simulations from breakdown. This report surveys approaches for fault-tolerance in numerical algorithms and system resilience in parallel simulations from the perspective of numerical weather and climate prediction systems. A selection of existing strategies is analyzed, featuring interpolation-restart and compressed checkpointing for the numerics, in-memory checkpointing, user-level failure mitigation-based and backup-based methods for the systems. Numerical examples showcase the performance of the techniques in addressing faults, with particular emphasis on iterative solvers for linear systems, a staple of atmospheric fluid flow solvers. The potential impact of these strategies is discussed in relation to current development of numerical weather prediction algorithms and systems towards the exascale. Trade-offs between performance, efficiency and effectiveness of resiliency strategies are analyzed and some recommendations outlined for future developments.

97 MATHEMATICS AND COMPUTING↗

Extending Skel to Support the Development and Optimization of Next Generation I/O Systems

As the memory and storage hierarchy get deeper and more complex, it is important to have new benchmarks and evaluation tools that allow us to explore the emerging middleware solutions to use this hierarchy. Skel is a tool aimed at automating and refining this process of studying HPC I/O performance. It works by generating application I/O kernel/benchmarks as determined by a domain-specific model. This paper provides some techniques for extending Skel to address new situations and to answer new research questions. For example, we document use cases as diverse as using Skel to troubleshoot I/O performance issues for remote users, refining an I/O system model, and facilitating the development and testing of a mechanism for runtime monitoring and performance analytics. We also discuss data oriented extensions to Skel to support the study of compression techniques for Exascale scientific data management.

97 MATHEMATICS AND COMPUTING↗

Biomass Feedstock Flow Modeling at Exascale

Variable biomass feedstock handling in hoppers and screw conveyors can cause upstream failures and disrupt downstream conversion operations. Understanding how particle size, shape, and moisture content affect feedstock flowability is important for efficient biorefinery operation. We present the use of our open-source discrete element method (DEM) solver for predicting flows in large scale hoppers and assess its performance on state-of-the-art high performance computing platforms that use heterogenous computing architectures.

bioenergy↗