Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “EXASCALE”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 811 records · Page 45
All sourcesNASA NTRSDOE OSTI
DOE OSTI2022

A weighted state redistribution algorithm for embedded boundary grids

State redistribution is an algorithm that stabilizes cut cells for embedded boundary grid methods. This work extends the earlier algorithm in several important ways. First, state redistribution is extended to three spatial dimensions. Second, we discuss several algorithmic changes and improvements motivated by the more complicated cut cell geometries that can occur in higher dimensions. In particular, we introduce a weighted version with less dissipation in an easily generalizable framework. Third, we demonstrate that state redistribution can also stabilize a solution update that includes both advective and diffusive contributions. Notably, the stabilization algorithm is shown to be effective for incompressible as well as compressible reacting flows. Finally, we discuss the implementation of the algorithm for several exascale-ready simulation codes based on AMReX, demonstrating ease of use in combination with domain decomposition, hybrid parallelism and complex physics.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2024

Development of multi-scale computational frameworks to solve fusion materials science challenges

Over the past two decades, the US-DOE has funded multiple projects that rely on high-performance computing and exascale computing platforms to accelerate scientific discoveries and address grand scientific challenges, such as harnessing fusion energy. In this article, we review in detail one of these efforts aimed at enhancing our capability to model plasma-facing materials subject to plasma and high-energy ion/neutron irradiation. The plasma surface interactions project has built a multi-scale modeling framework where many of the plasma- and high-energy ion/neutron irradiation-induced effects occurring in tungsten are explored. Here, this knowledge is used to develop atomistically-informed, high-fidelity continuum and meso-scale models that can be validated against experiments. We review the developments within this project, with attention to experimental validation efforts, and specifically highlight activities associated with: helium bubble bursting and equation of state, and hydrogen-helium interactions in tungsten; atomistically-informed model development for beryllium-tungsten material mixing; coupling of scrape-of-layer plasma, sheath and material models; and coupling of stochastic cluster-dynamics and crystal plasticity models to address radiation effects in tungsten under stress. Finally, we present how the project is preparing for future computational architectures, for instance through efforts to adapt atomistic methods to exascale computing.

36 MATERIALS SCIENCE↗
DOE OSTI2020

Multi-GPU immersed boundary method hemodynamics simulations

Large-scale simulations of blood flow that resolve the 3D deformation of each comprising cell are increasingly popular owing to algorithmic developments in conjunction with advances in compute capability. Among different approaches for modeling cell-resolved hemodynamics, fluid structure interaction (FSI) algorithms based on the immersed boundary method are frequently employed for coupling separate solvers for the background fluid and the cells within one framework. GPUs can accelerate these simulations; however, both current pre-exascale and future exascale CPU-GPU heterogeneous systems face communication challenges critical to performance and scalability. In this paper, we describe, to our knowledge, the largest distributed GPU-accelerated FSI simulations of high hematocrit cell-resolved flows with over 17 million red blood cells. We compare scaling on a fat node system with six GPUs per node and on a system with a single GPU per node. Through comparison between the CPU- and GPU-based implementations, we identify the costs of data movement in multiscale multi-grid FSI simulations on heterogeneous systems and show it to be the greatest performance bottleneck on the GPU.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2021

Measurement and analysis of GPU-accelerated applications with HPCToolkit

To address the challenge of performance analysis on the US DOE’s forthcoming exascale supercomputers, Rice University has been extending its HPCToolkit performance tools to support measurement and analysis of GPU-accelerated applications. To help developers understand the performance of accelerated applications as a whole, HPCToolkit’s measurement and analysis tools attribute metrics to calling contexts that span both CPUs and GPUs. To measure GPU-accelerated applications efficiently, HPCToolkit employs a novel wait-free data structure to coordinate monitoring and attribution of GPU performance. To help developers understand the performance of complex GPU code generated from high-level programming models, HPCToolkit constructs sophisticated approximations of call path profiles for GPU computations. To support fine-grained analysis and tuning, HPCToolkit uses PC sampling and instrumentation to measure and attribute GPU performance metrics to source lines, loops, and inlined code. To supplement fine-grained measurements, HPCToolkit can measure GPU kernel executions using hardware performance counters. To provide a view of how an execution evolves over time, HPCToolkit can collect, analyze, and visualize call path traces within and across nodes. Finally, on NVIDIA GPUs, HPCToolkit can derive and attribute a collection of useful performance metrics based on measurements using GPU PC samples. Here, we illustrate HPCToolkit’s new capabilities for analyzing GPU-accelerated applications with several codes developed as part of the Exascale Computing Project.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2019

Evaluation of Clouds in Version 1 of E3SM Atmosphere Model with Satellite and Simulators

This study systematically evaluates clouds simulated by the Energy Exascale Earth System Model (E3SM) Atmosphere Model version one (EAMv1) against satellite and ground-based cloud observations. Both low (1°) and high (0.25°) resolution EAMv1 configurations generally underestimate clouds in low and midlatitutes and overestimate clouds in the Arctic although the error is smaller in the high-resolution model. The underestimate of clouds is due to the underestimate of optically thin to intermediate clouds. EAMv1 overestimates the optically intermediate to thick clouds. Other model errors include the largely under-predicted stratocumulus along the coasts and high clouds over the tropical deep convection regions. The underestimate of thin clouds results in too much LW radiation being emitted to space and too little SW radiation being reflected back to space while the overestimate of optically intermediate and thick clouds leads to too little LW radiation being emitted to space and too much SW radiation being reflected back to space. EAMv1 shows better skill in reproducing the observed distribution of clouds and their properties and has smaller radiatively relevant errors in the distribution of clouds than most of the CFMIP2 models. It produces more supercooled liquid cloud fraction than CAM5 and most CMIP5 models owing to a new ice nucleation scheme and a reduction of ice deposition growth rate. It simulates the diurnal variation of clouds during the summer season at SGP qualitatively well, with both the evolution of shallow clouds and the nocturnal peak in high clouds well captured, however, it largely overestimates the observed magnitude.

Zhang, Yuying↗
DOE OSTI2019

The Summertime Precipitation Bias in E3SM Atmosphere Model Version 1 over the Central United States

This study analyzes the summertime precipitation bias over the Central United States and its relationship to the simulated large-scale environment and the convection scheme in the Energy Exascale Earth System Model Atmosphere Model version 1. This relationship is mainly examined in a set of short-term hindcasts initialized with realistic large-scale conditions for the summer of 2011. Besides the uniform 1° model resolution, we adopt Regionally Refined Meshes to increase the model resolution to 0.25°over the contiguous United States. Additional five-year Atmospheric Model Intercomparison Project simulations are conducted to confirm that the results from the hindcasts are consistent with the climate runs. We find that the summertime dry precipitation bias over the Great Plains and the wet bias over the Rockies cannot be reduced simultaneously by changing resolution or tuning parameters. As for the diurnal cycle, Energy Exascale Earth System Model Atmosphere Model version 1 captures the general diurnal variation of the large-scale moisture transport and the large-scale upward motion over the Central United States. However, the diurnal cycle of precipitation over the Great Plains is out of phase with the diurnal variation of the large-scale environment because the convective precipitation dominates the total precipitation and its diurnal cycle, and it does not directly respond to the local moisture convergence and the large-scale upward motion. These results reemphasize the importance of improving the coupling of the convection to the large-scale environment in reducing the summer precipitation bias over the Central United States in climate models with the resolution of ~0.25°.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2019

Flood Inundation Generation Mechanisms and Their Changes in 1953-2004 in Global Major River Basins

Despite the serious threats posed by floods, the driving mechanisms of floods are still not well understood. Here, we apply a physically-based inundation model coupled with a river routing model (Model for Scale Adaptive River Transport, MOSART) within the Energy Exascale Earth System Model (E3SM) framework to investigate flood inundation dynamics. After calibration using observed streamflow and satellite-derived flood extent, the model is used to simulate global flood inundation from 1953 to 2004. The mean date and seasonality of annual maximum flood, defined based on flood extent, exhibit significant regional differences across 16 major basins. Generally, soil moisture and monthly maximum daily rainfall (MMR) are the dominant drivers of flood in tropical basins while monthly maximum daily snowmelt (MMS) is the dominant driver in high latitude basins. From 1953-1982 to 1975-2004, significant changes in flood generation mechanisms are found in some basins such as Amazon, Lena, Yenisey, and Kolyma. Analysis of the rainfall seasonality and water balance at grid scale reveals a stronger rainfall seasonality in the Amazon during the later period that increases the synchrony between extreme rainfall and wet soil. With high antecedent soil moisture coinciding with rainfall, MMR contributes more to flood in the later period. Fewer extreme rainfall events and increasing soil moisture reduced the contribution of MMR and increased the role of MMS in floods in the Lena and Yenisey basins, respectively. Lastly, increased soil moisture and frequency of large MMS reduced the contribution of the latter to floods in the Kolyma basin.

Mao, Yuna↗
DOE OSTI2020

Exploring topography-based methods for downscaling subgrid precipitation for use in Earth System Models

Topography exerts major control on land surface processes. To improve representation of topographic impacts on land surface processes, a new topography-based subgrid structure has been introduced to the Energy Exascale Earth System Model to represent the subgrid heterogeneity of surface elevation. To take advantage of the new subgrid structure for improving land surface modeling, this study explores four variations of topography-based methods for downscaling grid precipitation to the corresponding subgrids. In the first three methods, the deviation of the subgrid precipitation from the grid’s is equal to the grid precipitation multiplied by the ratio of the elevation difference between the subgrid and grid mean to a specified elevation equals to the grid elevation, the difference between the maximum and minimum subgrid elevation, and the maximum subgrid elevation, respectively. The second method limits the ratio to 0.5 to avoid extreme values on mountains and the third method accounts for the slope effect. The fourth method is similar to the third method except that the Froude Number is used to limit the numerator in a blocking regime of the ambient flow. The downscaled precipitation is evaluated using the PRISM precipitation data over the U.S. using statistical metrics. Results show that by accounting for topographic slope besides elevation, the third and fourth methods show clear advantages over the first and second methods. Furthermore, introducing the Froude Number in the fourth method improves downscaling skill and shows consistent advantages over the third method in areas with larger subgrid heterogeneity across different grids sizes.

Tesfa, Teklu K.↗
DOE OSTI2021

Effects of organized convection parameterization on the MJO and precipitation in E3SMv1. Part I: Mesoscale heating

Mesoscale organization of convection is typically not represented in global circulation models, and hence its influence on the global circulation is not accounted for. The heating component of a parameterization that represents the dynamical and physical effects of circulations associated with organized convection, referred to as the multiscale coherent structure parameterization (MCSP), is implemented in the Energy Exascale Earth System Model version 1 (E3SMv1). Numerical simulations are conducted to assess its impact on the simulated climate. Besides E3SMv1 simulations, we performed high-resolution (1 km) simulations using the Weather Research and Forecasting (WRF) Model to determine the temperature tendencies induced by mesoscale convective systems embedded in deep convection. We tuned the free parameters of the MCSP based on the WRF simulations. MCSP heating enhances Kevin wave spectra in E3SMv1, improves the representation of the Madden-Julian Oscillation, and impacts on mean precipitation that vary with locations and seasons, including a reduction of biases over the tropical Pacific.

Climate Modeling↗
DOE OSTI2021

Subtropical Eastern North Pacific SST bias in earth system models

Karthik Balaguru (PNNL), Luke Van Roekel (LLNL), L. Ruby Leung (PNNL) and Milena Veneziani (LLNL) Abstract: This study systematically evaluates the warm SST bias in the Subtropical Eastern North Pacific (SENP), a problem plaguing most CMIP6 models, using the Energy Exascale Earth System Model version 1 (E3SM). Compared to the model at its standard resolution (1o atmosphere, 30-60 km ocean), the SST bias improves in several regions at high resolution (0.25o atmosphere, 18-6 km ocean) except in subtropical eastern boundary regions, such as the SENP. A mixed layer heat budget analysis reveals that errors in the surface radiative fluxes and horizontal heat advection are primarily responsible for the bias. Analysis of HighResMIP models indicates that the lack of improvement in the SENP SST bias with resolution is not native to E3SM alone. A larger CMIP6 inter-model correlation of SST bias with biases in surface winds than radiation highlights the important role of large-scale circulation biases in the SENP SST bias in coupled models.

Balaguru, Karthik↗
DOE OSTI2024

Mesoscale Convective Systems Represented in High Resolution E3SMv2 and Impact of New Cloud and Convection Parameterizations

Mesoscale convective systems (MCSs) play an important role in modulating the global hydrological cycle, general circulation, and radiative energy budget. In this study, we evaluate MCS simulations in the second version of U.S. Department of Energy (DOE) Energy Exascale Earth System Model (E3SMv2). E3SMv2 atmosphere model (EAMv2) is run at the uniform 0.25? horizontal resolution. We track MCSs consistently in the model and observations using the PyFLEXTRKR algorithm, which defines MCS based on both cloud-top brightness temperature (Tb) and surface precipitation. Results from using Tb only to define MCS, commonly used in previous studies, are also discussed. Furthermore, sensitivity experiments are performed to examine the impact of new cloud and convection parameterizations developed for EAMv3 on simulated MCSs. Our results show that EAMv2 simulated MCS precipitation is largely underestimated in the tropics and contiguous United States. This is mainly attributed to the underestimated precipitation intensity in EAMv2. In contrast, the simulated MCS frequency becomes more comparable to observations if MCSs are defined only based on cloud-top Tb. The Tb-based MCS tracking method, however, includes many cloud systems with very weak precipitation which conflicts with the MCS definition. This result illustrates the importance of accounting for precipitation in evaluating simulated MCSs. We also find that the new physics parameterizations help increase the relative contribution of convective precipitation to total precipitation in the tropics, but the simulated MCS properties are generally not improved. This suggests that simulating MCSs will remain a challenge for the next version of E3SM.

Zhang, Meng↗
DOE OSTI2021

Modeling of a chain of three plasma accelerator stages with the WarpX electromagnetic PIC code on GPUs

The fully electromagnetic particle-in-cell code WarpX is being developed by a team of the U.S. DOE Exascale Computing Project (with additional non-U.S. collaborators on part of the code) to enable the modeling of chains of tens to hundreds of plasma accelerator stages on exascale supercomputers, for future collider designs. The code is combining the latest algorithmic advances (e.g., Lorentz boosted frame and pseudo-spectral Maxwell solvers) with mesh refinement and runs on the latest computer processing unit and graphical processing unit (GPU) architectures. In this paper, we summarize the strategy that was adopted to port WarpX to GPUs, report on the weak parallel scaling of the pseudo-spectral electromagnetic solver, and then present solutions for decreasing the time spent in data exchanges from guard regions between subdomains. In Sec. IV, we demonstrate the simulations of a chain of three consecutive multi-GeV laser-driven plasma accelerator stages.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗
DOE OSTI2024

Parallel interior-point solver for block-structured nonlinear programs on SIMD/GPU architectures

Here, we investigate how to port the standard interior-point method to new exascale architectures for block-structured nonlinear programs with state equations. Computationally, we decompose the interior-point algorithm into two successive operations: the evaluation of the derivatives and the solution of the associated Karush-Kuhn-Tucker (KKT) linear system. Our method accelerates both operations using two levels of parallelism. First, we distribute the computations on multiple processes using coarse parallelism. Second, each process uses SIMD/GPU accelerators locally to accelerate the operations using fine-grained parallelism. The KKT system is reduced by eliminating the inequalities and the state variables from the corresponding equations. We demonstrate our method's capability on the supercomputer Polaris, a testbed for the future exascale Aurora system. Each node is equipped with four GPUs, a setup amenable to our two-level approach. Our experiments on the stochastic optimal power flow problem show that the reduction method is 50x faster than the sparse linear solver HSL MA57 running in serial on the CPU, and 6x faster than Pardiso running in parallel on CPU on the same number of processes.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2020

The future of computing beyond Moore’s Law

Moore’s Law is a techno-economic model that has enabled the information technology industry to double the performance and functionality of digital electronics roughly every 2 years within a fixed cost, power and area. Advances in silicon lithography have enabled this exponential miniaturization of electronics, but, as transistors reach atomic scale and fabrication costs continue to rise, the classical technological driver that has underpinned Moore’s Law for 50 years is failing and is anticipated to flatten by 2025. Here, this article provides an updated view of what a post-exascale system will look like and the challenges ahead, based on our most recent understanding of technology roadmaps. It also discusses the tapering of historical improvements, and how it affects options available to continue scaling of successors to the first exascale machine. Lastly, this article covers the many different opportunities and strategies available to continue computing performance improvements in the absence of historical technology drivers. This article is part of a discussion meeting issue ‘Numerical algorithms for high-performance computational science’.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2025

Modeling Offshore Wind Farm Performance in Coastal Low-Level Jets Using Coupled Mesoscale-Microscale Large Eddy Simulations

Accurately predicting wind farm reliability under complex offshore atmospheric conditions remains a key challenge, particularly during noncanonical meteorological events such as coastal low-level jets (LLJs). LLJs, characterized by strong nonmonotonic vertical shear and directional veer, depart significantly from the simplified inflow assumptions embedded in conventional design standards, low-fidelity engineering models, and microscale large eddy simulations of the atmospheric boundary layer. In this work, we use the virtual wind farm framework—an exascale, graphics processing unit–accelerated large eddy simulation platform coupled with high-fidelity aeroservoelastic turbine models and advanced mesoscale-microscale coupling via the ExaWind software stack—to investigate turbine responses under realistic LLJ forcing. Simulations are performed over the U.S. North Atlantic offshore domain with the use of meteorological inputs from New York State Energy Research and Development Authority buoy data, focusing on a representative LLJ case impacting the International Energy Agency 15 MW reference turbine. Our results show that LLJs can cause up to 50% power deficits in downstream turbine rows and significantly amplify low-speed shaft and tower loads through nonlinear coupling between complex inflow characteristics and turbine structural dynamics. Two primary mechanisms drive these load amplifications: (1) unique LLJ inflow features—including veer and vertical/lateral shear—and (2) the downstream evolution of the flow under stable thermal stratification, which suppresses turbulence mixing and alters wake recovery. These mechanisms produce streamwise variations in turbine loading not captured by standard hub height–based metrics or existing design load case (DLC) definitions. This study highlights the critical role of rotor-scale flow gradients in driving fatigue and system-level aeroelastic responses, challenging current DLC and control strategies. We advocate the integration of full-flow field, environment-aware wind inputs into load modeling and control algorithms. By leveraging exascale computing to resolve mesoscale-microscale coupling, this work lays the groundwork for next-generation offshore wind turbine design and operation in meteorologically complex marine environments.

17 WIND ENERGY↗
DOE OSTI2022

Fast Gaussian Process Estimation for Large-Scale In Situ Inference using Convolutional Neural Networks

Exascale computing will bring with it significant I/O limitations. One foreseeable consequence of such restrictions is that the user can save only a small fraction of complex simulation data to disk for subsequent analysis. An alternative is to fit statistical models to data in situ, that is, inside the simulation as it runs. This option requires extremely fast statistical estimation to avoid slowing down the simulation. Gaussian processes (GPs) have state-of-the-art predictive performance for modeling spatial data. However, standard estimation methods for GPs scale quite poorly to large data sets as parameter estimation requires inverting a covariance matrix to the size of the data set. In the presented work, we use a convolutional neural network (CNN) to predict the GP parameters for a spatial data set, from a simulation or otherwise, rather than optimize the parameters directly. Here, our presented case study models spatial data from E3SM, the Department of Energy’s Exascale climate model. The CNN is trained on synthetic data simulated from GP models with known parameters and then applied to data from the climate simulation. In the presented examples, the neural network scheme produces parameter estimates that compare well with standard methods such as maximum likelihood estimation in predictive performance but is obtained four orders of magnitude faster.

big data↗
DOE OSTI2025

JACC.shared: Leveraging HPC Metaprogramming and Performance Portability for Computations That Use Shared Memory GPUs

In this work, we present JACC.shared, a new feature of Julia for ACCelerators (JACC), which is the performanceportable and metaprogramming model of the just-in-time and LLVM-based Julia language. This new feature allows JACC applications to leverage the high-performance computing (HPC) capabilities of high-bandwidth, on-chip GPU memory. Historically, exploiting high-bandwidth, shared-memory GPUs has not been a priority for high-level programming solutions. JACC.shared covers that gap for the first time, thereby providing a highlevel, portable, and easy-to-use solution for programmers to exploit this memory and supporting all current major accelerator architectures. Well-known HPC and AI workloads, such as multi/hyperspectral imaging and AI convolutions, have been used to evaluate JACC.shared on two exascale GPU architectures hosted by some of the most powerful US Department of Energy supercomputers: Perlmutter (NVIDIA A100) and Frontier (AMD MI250X). The performance evaluation reports speedup of up to 3.5× by adding only one line of code to the base codes, thus providing important accelerators in a simple, portable, and transparent way and elevating the programming productivity and performance-portability capabilities for Julia/JACC HPC, AI, and scientific applications.

Valero Lara, Pedro [ORNL] (ORCID:0000000214794310)↗