Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “EXASCALE”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,009 records · Page 56
All sourcesNASA NTRSDOE OSTI
DOE OSTI2017

Exploring dynamic load imbalance solutions with the CoMD proxy application

Proxy applications are developed to simplify studying parallel performance of scientific simulations and to test potential solutions for performance problems. However, proxy applications are typically too simple to allow work migration or to represent the load imbalance of their parent applications. To study the ability of load balancing solutions to balance work effectively, we enable work migration in one of the Exascale Co-design Center for Materials in Extreme Environments (ExMatEx) [1] applications, CoMD. We design a methodology to parameterize three key aspects necessary for studying load imbalance correction: (1) the granularity with which work can be migrated; (2) the initial load imbalance; (3) the dynamic load imbalance (how quickly the load changes over time). Finally, we present a study of the impact of flexibility in work migration in CoMD on load balance and the associated rebalancing costs for a wide range of initial and dynamic load imbalance scenarios.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2020

End-to-end online performance data capture and analysis for scientific workflows

With the increased prevalence of employing workflows for scientific computing and a push towards exascale computing, it has become paramount that we are able to analyze characteristics of scientific applications to better understand their impact on the underlying infrastructure and vice-versa. Such analysis can help drive the design, development, and optimization of these next generation systems and solutions. Here, we present the architecture, integrated with existing well-established and newly developed tools, to collect online performance statistics of workflow executions from various, heterogeneous sources and publish them in a distributed database (Elasticsearch). Using this architecture, we are able to correlate online workflow performance data, with data from the underlying infrastructure, and present them in a useful and intuitive way via an online dashboard. We have validated our approach by executing two classes of real-world workflows, both under normal and anomalous conditions. The first is an I/O-intensive genome analysis workflow; the second, a CPU- and memory-intensive material science workflow. Based on the data collected in Elasticsearch, we are able to demonstrate that we can correctly identify anomalies that we injected. The resulting end-to-end data collection of workflow performance data is an important resource of training data for automated machine learning analysis.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2024

Software stewardship and advancement of a high-performance computing scientific application: QMCPACK

Here, we provide an overview of the software engineering efforts and their impact in QMCPACK, a production-level ab-initio Quantum Monte Carlo open-source code targeting high-performance computing (HPC) systems. Aspects included are: (i) strategic expansion of continuous integration (CI) targeting CPUs, using GitHub Actions own runners, and NVIDIA and AMD GPUs used in pre-exascale systems, (ii) incremental reduction of memory leaks using sanitizers, (iii) incorporation of Docker containers for CI and reproducibility, and (iv) refactoring efforts to improve maintainability, testing coverage, and memory lifetime management. We quantify the value of these improvements by providing metrics to illustrate the shift towards a predictive, rather than reactive, maintenance approach. Our goal, in documenting the impact of these efforts on QMCPACK, is to contribute to the body of knowledge on the importance of research software engineering (RSE) for the stewardship and advancement of community HPC codes to enable scientific discovery at scale.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2020

Spatiotemporal dynamics of ecosystem fires and biomass burning-induced carbon emissions in China over the past two decades

Fire is a major type of disturbance that has important influences on ecosystem dynamics and carbon cycles. Yet our understanding of ecosystem fires and their carbon cycle consequences is still limited, largely due to the difficulty of large-scale fire monitoring and the complex interactions between fire, vegetation, climate, and anthropogenic factors. Here, using data from satellite-derived fire observations and ecosystem model simulations, we performed a comprehensive investigation of the spatial and temporal dynamics of China's ecosystem fire disturbances and their carbon emissions over the past two decades (1997–2016). Satellite-derived results showed that on average about 3.47 - 4.53 x 10 4 km 2 of the land was burned annually during the past two decades, among which annual burned forest area was about 0.81 - 1.25 × 10 4 km 2 , accounting for 0.33-0.51% of the forest area in China. Biomass burning emitted about 23.02 TgC per year. Compared to satellite products, simulations from the Energy Exascale Earth System Land Model (ELM) strongly overestimated China's burned area and fire-induced carbon emissions. Annual burned area and fire-induced carbon emissions were high for boreal forest in Northeast China's Daxing'anling region and subtropical dry forest in South Yunnan, as revealed by both the satellite product and the model simulations. Our results suggest that climate and anthropogenic factors play critical roles in controlling the spatial and seasonal distribution of China's ecosystem fire disturbances. Our findings highlight the importance of multiple complementary approaches in assessing ecosystem fire disturbance and its carbon consequences. Further studies are required to improve the methods of observing and modelling China's ecosystem fire disturbances, which will provide valuable information for fire management and ecosystem sustainability in an era when both human activities and the natural environment are rapidly changing.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2020

Development of a CPU/GPU portable software library for Lagrangian–Eulerian simulations of liquid sprays

The Lagrangian–Eulerian method is widely used for simulations of fuel sprays in turbulent combustion because of the advantage of treating the spray droplets as discrete points. One challenge of the Lagrangian–Eulerian method is the intense computational requirement when tracking the large number of Lagrangian particles needed for high fidelity. We have developed a performance-portable library, Grit, to track the Lagrangian particles in parallel on central processing unit (CPU) and graphics processing unit (GPU) accelerated high performance computing (HPC) architectures. Grit is a C++ library which employs Message Passing Interface (MPI) for distributed memory parallelism and Kokkos programming model for on-node shared memory parallelism with performance portability across different architectures of GPUs and multi-core/manycore CPUs. The parallel algorithms, key parallel kernels, and their performances on the pre-exascale supercomputer, Summit, are presented. Grit is coupled with a direct numerical simulation (DNS) solver, S3D, for multiphase simulations. A conservative formulation has been developed and implemented in Grit for phase coupling with thermodynamic consistency. The formulation separates the conservation of mass, momentum and energy from the physical models to prevent accidental violation of conservation laws due to inconsistent models. The formulation also enforces consistent definitions of enthalpies of the fuel for both phases and the latent heat of evaporation. Finally, simulations of turbulent particle-laden flow and the evaporation of dilute turbulent spray jet are performed to verify the software implementation, and to demonstrate the scalability of Grit for large-scale multiphase simulations.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2017

A general CFD framework for fault-resilient simulations based on multi-resolution information fusion

We develop a general CFD framework for multi-resolution simulations to target multiscale problems but also resilience in exascale simulations, where faulty processors may lead to gappy, in space-time, simulated fields. We combine approximation theory and domain decomposition together with statistical learning techniques, e.g. coKriging, to estimate boundary conditions and minimize communications by performing independent parallel runs. To demonstrate this new simulation approach, we consider two benchmark problems. First, we solve the heat equation (a) on a small number of spatial “patches” distributed across the domain, simulated by finite differences at fine resolution and (b) on the entire domain simulated at very low resolution, thus fusing multi-resolution models to obtain the final answer. Second, we simulate the flow in a lid-driven cavity in an analogous fashion, by fusing finite difference solutions obtained with fine and low resolution assuming gappy data sets. We investigate the influence of various parameters for this framework, including the correlation kernel, the size of a buffer employed in estimating boundary conditions, the coarseness of the resolution of auxiliary data, and the communication frequency across different patches in fusing the information at different resolution levels. In addition to its robustness and resilience, the new framework can be employed to generalize previous multiscale approaches involving heterogeneous discretizations or even fundamentally different flow descriptions, e.g. in continuum-atomistic simulations.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
DOE OSTI2020

Pass-efficient methods for compression of high-dimensional turbulent flow data

The future of high-performance computing, specifically on future Exascale computers, will presumably see memory capacity and bandwidth fail to keep pace with data generated, for instance, from massively parallel partial differential equation (PDE) systems. Current strategies proposed to address this bottleneck entail the omission of large fractions of data, as well as the incorporation of in situ compression algorithms to avoid overuse of memory. To ensure that post-processing operations are successful, this must be done in a way that a sufficiently accurate representation of the solution is stored. Moreover, in situations where the input/output system becomes a bottleneck in analysis, visualization, etc., or the execution of the PDE solver is expensive, the number of passes made over the data must be minimized. In the interest of addressing this problem, this work focuses on the utility of pass-efficient, parallelizable, low-rank, matrix decomposition methods in compressing high-dimensional simulation data from turbulent flows. Additionally, a particular emphasis is placed on using coarse representation of the data – compatible with the PDE discretization grid – to accelerate the construction of the low-rank factorization. This includes the presentation of a novel single-pass matrix decomposition algorithm for computing the so-called interpolative decomposition. The methods are described extensively and numerical experiments on two turbulent channel flow data are performed. In the first (unladen) channel flow case, compression factors exceeding 400 are achieved while maintaining accuracy with respect to first- and second-order flow statistics. In the particle-laden case, compression factors of 100 are achieved and the compressed data is used to recover particle velocities. These results show that these compression methods can enable efficient computation of various quantities of interest in both the carrier and disperse phases.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
DOE OSTI2024

Stabilized bases for high-order, interpolation semi-Lagrangian, element-based tracer transport

In a computational fluid model of the atmosphere, the advective transport of trace species, or tracers, can be computationally expensive. For efficiency, models often use semi-Lagrangian advection methods. High-order interpolation semi-Lagrangian (ISL) methods, in particular, can be extremely efficient, if the problem of property preservation specific to them can be addressed. Atmosphere models often use geometrically and logically nonuniform grids for efficiency and, as a result, element-based discretizations. Such grids and discretizations make stability a particular problem for ISL methods. Generally, high-order, element-based ISL methods that use the natural polynomial interpolant associated with a nodal finite-element discretization are unstable. Here, we derive new bases having order of accuracy up to nine, with positive nodal weights, that stabilize the element-based ISL method. We use these bases to construct the linear advection operator in the property-preserving Interpolation Semi-Lagrangian Element-based Transport (Islet) method. Then we discuss key software implementation details. Finally, we show performance results for the Energy Exascale Earth System Model's atmosphere dynamical core, comparing the original and new transport methods. These simulations used up to 27,600 Graphical Processing Units (GPU) on the Oak Ridge Leadership Computing Facility's Summit supercomputer.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2024

Quantifying the long-term changes of terrestrial water storage and their driving factors

Global warming is expected to cause changes in terrestrial water storage (TWS) across the land surface, with widespread impacts on ecosystems and society. Although extensive research has been performed to analyze TWS changes and possible drivers during the post-2000 period, longer-term evolution of TWS and associated environmental forcings remain relatively unexplored. In this study, we evaluated the performance of the Energy Exascale Earth System model (E3SM) land model ELM version 1 (ELM v1) in simulating global TWS, and used factorial simulations of ELMv1 to quantify global TWS changes and their drivers during 1948–2012. We found that ELM’s agreed best with existing satellites and reconstruction datasets in temperate regions unaffected by irrigation. Biome- and climate zone-averaged TWS mainly increased at rates between 0 and 10 mm/year over 1948–2012, but the second half of that period saw smaller positive trends than the first half or even negative trends. Climate change explained >80 % of the TWS trends across most biomes and climate zones, followed by land use and land cover change. The physiological and phenological effects of CO2 primarily induced noticeable TWS trends in the more humid biomes and climate zones across different latitudes. In contrast, nitrogen deposition and aerosol deposition generally had smaller and negative impacts across the biomes and climate regions. Among the meteorological drivers analyzed, the long-term average imbalance between precipitation (P), evapotranspiration (E), and runoff (Q) contributed >50 % of the TWS trends in most biomes and climate zones, with nonlinearity being induced by spatially heterogenous changes in E/P and Q/P ratios. The accumulated detrended anomalies in P, E, and Q also often contributed substantially, while the trends difference between P, E, and Q contributed little. Together, these findings unveiled an intensification of the global TWS and its diverse patterns of climate change and different non-withdrawal human-induced alterations, contributing to a more comprehensive understanding and projection of the global water cycle.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2026

Storylines for the 1997 New Year’s Flood: The role of watershed antecedent conditions and future warming in shaping discharge in the Truckee River watershed

The 1997 New Year’s flood was among the most devastating floods in the Truckee River watershed located in western Nevada. This event resulted from complex interactions of flood drivers, such as extreme precipitation, wet antecedent watershed conditions, warm temperatures and rapid snowmelt. We leveraged simulated forcings from the regionally refined mesh capabilities of the Energy Exascale Earth System Model (RRM-E3SM) and a process-based hydrological model to recreate the 1997 New Year’s flood for the Truckee River watershed across four climate warming levels ranging from the current temperatures to + 4° C. For each scenario, we conducted ensemble simulations with the same forcing but with 100 different seasonal watershed antecedent conditions, which were randomly sampled from long-term hydrological simulations. The results show that the 1997 New Year’s flood can be reproduced or exceeded consistently only when the antecedent watershed conditions are wet, specifically when streamflows are above the 75th percentile of the climatological value. There is negligible change in ensemble mean peakflows for Truckee River near Reno; however, there are increases of 18% and 14% under the warming levels of + 3° C and + 4° C, respectively. The increases in peakflows under future climate warming are attributed to wetter antecedent watershed conditions and enhanced snowmelt. Furthermore, the largest increases in peakflows occur at small, high-elevation headwater basins along the Sierra Nevada crest. This study highlights that changes in extreme flood events will result from the complex interplay of multiple flood drivers. It also demonstrates the potential of storyline approaches to analyze future realizations of these extreme events under different climate scenarios.

Climate change impact study↗
DOE OSTI2026

Systematic Evaluation of Atmospheric Forcing, Surface Datasets, and Mesh Effects on Kilometer-Scale Land Surface and River Modeling

Earth system models are advancing toward kilometer-scale resolution to capture local climate impacts and extremes. High-resolution land and river modeling depends on multiple factors, including mesh, surface datasets, and atmospheric forcing, but their relative effects at kilometer scales remain unquantified. We evaluated five Energy Exascale Earth System Model land and river configurations over the Mid-Atlantic region using two mesh (1/8° structured versus variable-resolution unstructured mesh), two surface datasets (default versus newly developed), and three atmospheric forcings (NLDAS2, MSWX, GSWP). Evaluation against satellite, reanalysis, and in situ benchmarks across water, energy, and carbon cycles quantifies how these factors affect model performance. Forcing selection produces the largest bias reductions (12-99% across variables), followed by surface datasets (7-75%) and mesh (up to 21%). Forcing effects vary by variable, with MSWX reducing biases for snow water equivalent, evapotranspiration, albedo, temperature, and gross primary productivity, GSWP for snow cover and runoff, and NLDAS for soil moisture and streamflow. The use of newly developed surface datasets improves gross primary productivity (58% bias reduction) and evapotranspiration but increase soil moisture and albedo biases due to current modeling limitations. Variable-resolution unstructured mesh improves the simulation of small-basin streamflow through better capturing drainage networks, though mesh minimally affects other land variables. These findings provide important guidance for high-resolution modeling development and actionable science.

Land and River modeling↗
DOE OSTI2021

Translational research in the MPICH project

The MPICH project is an example of translational research in computer science before that term was well known or even coined. The project began in 1992 as an effort to develop a portable, high-performance implementation of the emerging Message-Passing Interface (MPI) Standard. It has enabled the widespread adoption of MPI as a way to write scalable parallel applications on systems of all sizes including upcoming exascale supercomputers. In this paper, we describe how the translational research process was used in MPICH, how that led to its success, the challenges encountered and lessons learned, and how the process could be applied to other similar projects.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2022

Demonstrating the viability of Lagrangian in situ reduction on supercomputers

Performing exploratory analysis and visualization of large-scale time-varying computational science applications is challenging due to inaccuracies that arise from under-resolved data. In recent years, Lagrangian representations of the vector field computed using in situ processing are being increasingly researched and have emerged as a potential solution to enable exploration. However, prior works have offered limited estimates of the encumbrance on the simulation code as they consider “theoretical” in situ environments. Further, the effectiveness of this approach varies based on the nature of the vector field, benefitting from an in-depth investigation for each application area. With this study, an extended version of Sane et al. (2021), we contribute an evaluation of Lagrangian analysis viability and efficacy for simulation codes executing at scale on a supercomputer. We investigated previously unexplored cosmology and seismology applications as well as conducted a performance benchmarking study by using a hydrodynamics mini-application targeting exascale computing. Here, to inform encumbrance, we integrated in situ infrastructure with simulation codes, and evaluated Lagrangian in situ reduction in representative homogeneous and heterogeneous HPC environments. To inform post hoc accuracy, we conducted a statistical analysis across a range of spatiotemporal configurations as well as a qualitative evaluation. Additionally, our study contributes cost estimates for distributed-memory post hoc reconstruction. In all, we demonstrate viability for each application — data reduction to less than 1% of the total data via Lagrangian representations, while maintaining accurate reconstruction and requiring under 10% of total execution time in over 90% of our experiments.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2024

Development of a multi-layer canopy model for E3SM Land Model with support for heterogeneous computing

The vertical structure of vegetation canopies creates micro-climates. However, the land components of most Earth System Models, including the Energy Exascale Earth System Model (E3SM), typically neglect vertical canopy structure by using a single layer big-leaf representation to simulate water, CO 2 , and energy exchanges between the land and the atmosphere. In this study, we developed a Multi-Layer Canopy Model for the E3SM Land Model to resolve the micro-climate created by vegetation canopies. The model developed in this study re-implements the CLM-ml_v1 to support heterogeneous computing architectures consisting of CPUs and GPUs and includes three additional optimization-based stomatal conductance models. The use of Portable, Extensible Toolkit for Scientific Computation provides a speedup of 25–50 times on a GPU relative to a CPU. The numerical implementation of the model was verified against CLM-ml_v1 for a month-long simulation using data from the Ameriflux US-University of Michigan Biological Station site. Model structural uncertainty was explored by performing control simulations for five stomatal conductance models that exclude and include the control of plant hydrodynamics (PHD) on photosynthesis. The bias in simulated sensible and latent heat fluxes was lower when PHD was accounted for in the model. Additionally, six idealized simulations were performed to study the impact of three environmental variables (i.e. air temperature, atmospheric CO 2 , and soil moisture) on canopy processes (i.e. net CO 2 assimilation, leaf temperature, and leaf water potential). Increasing air temperature reduced net CO 2 assimilation and increased air temperature. Net CO 2 assimilation increased at higher atmospheric CO 2 , while decreasing soil moisture resulted in lower leaf water potential.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2026

An uncertainty visualization framework for large-scale cardiovascular flow simulations: A case study on aortic stenosis

We present a generalizable uncertainty quantification (UQ) and visualization framework for lattice Boltzmann method simulations of high Reynolds number vascular flows, demonstrated on a patient-specific stenosed aorta. The framework combines EasyVVUQ for parameter sampling with large-eddy simulation turbulence modeling in HemeLB, and executes ensembles on the Frontier exascale supercomputer. Spatially resolved metrics, including entropy and isosurface-crossing probability, are used to map uncertainty in pressure and wall shear stress fields directly onto vascular geometries. Two sources of model variability are examined: inlet peak velocity and the Smagorinsky constant. Inlet velocity variation produces high uncertainty downstream of the stenosis where turbulence develops, while upstream regions remain stable. Smagorinsky constant variation has little effect on the large-scale pressure field but increases WSS uncertainty in localized high-shear regions. In both cases, the stenotic throat manifests low entropy, indicative of robust identification of elevated WSS. By linking quantitative UQ measures to three-dimensional anatomy, the framework improves interpretability over conventional 1D UQ plots and supports clinically relevant decision-making, with broad applicability to vascular flow problems requiring both accuracy and spatial insight.

Hemodynamics↗
DOE OSTI2020

An empirical study of I/O separation for burst buffers in HPC systems

To meet the exascale I/O requirements for the High-Performance Computing (HPC), a new I/O subsystem, Burst Buffer, based on solid state drives (SSD), has been developed. However, the diverse HPC workloads and the bursty I/O pattern cause severe data fragmentation that requires costly garbage collection (GC) and increases the number of bytes written to the SSD. To address this data fragmentation challenge, a new multi-stream feature has been developed for SSDs. In this work, we develop an I/O Separation scheme called BIOS to leverage this multi-stream feature to group the I/O streams based on the user IDs. We propose a stream-aware scheduling policy based on burst buffer pools in the workload manager, and integrate the BIOS with the workload manager to optimize the I/O separation scheme in burst buffer. We evaluate the proposed framework with a burst buffer I/O traces from Cori Supercomputer including a diverse set of applications. Experimental results show that the BIOS could improve the performance by 1.44x on average and reduce the Write Amplification Factor (WAF) by up to 1.20x. Finally, these demonstrate the potential benefits of the I/O separation scheme for solid state storage systems.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2021

Feasibility of full-core pin resolved CFD simulations of small modular reactor with momentum sources

Complex flow structure interactions and heat transfer processes take place in nuclear reactor cores. Given the extreme pressure/temperature and radioactive conditions inside the core, numerical simulations offer an attractive and sometimes more feasible approach to study the related flow and heat transfer phenomena in addition to the experiments. Under the Exascale Computing Project, the full-core simulation of a small modular reactor (SMR) has been pursued coupling Computational Fluid Dynamics (CFD) and neutronics. A key aspect of the modeling of SMR fuel assemblies is the presence of spacer grids and the mixing promoted by mixing vanes or the equivalent. A reduced order methodology is adopted based on momentum sources to mimic the mixing of the vanes. The momentum sources have been carefully calibrated with detailed Large Eddy Simulations (LES) of spacer grids performed with Nek5000. Modeling the spacer grid and mixing vanes (SGMV) effect without body-fitted computational grid avoids the excessive costs in resolving the local geometric details, and thus supports the simulation to be scaled up to the full core. Besides the progress on momentum source modeling, this paper also features the first full-core pin resolved CFD simulation ever performed to the authors' knowledge. This represents a significant advancement in capability for the CFD of nuclear reactors, which will hopefully serve as an inspiration for further integrating high-fidelity numerical simulations in actual engineering designs.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗
DOE OSTI2022

High performance sparse multifrontal solvers on modern GPUs

Here, we have ported the numerical factorization and triangular solve phases of the sparse direct solver STRUMPACK to GPU. STRUMPACK implements sparse LU factorization using the multifrontal algorithm, which performs most of its operations in dense linear algebra operations on so-called frontal matrices of various sizes. Our GPU implementation off-loads these dense linear algebra operations, as well as the sparse scatter–gather operations between frontal matrices. For the larger frontal matrices, our GPU implementation relies on vendor libraries such as cuBLAS and cuSOLVER for NVIDIA GPUs and rocBLAS and rocSOLVER for AMD GPUs. For the smaller frontal matrices we developed custom CUDA and HIP kernels to reduce kernel launch overhead. Overall, high performance is achieved by identifying submatrix factorizations corresponding to sub-trees of the multifrontal assembly tree which fit entirely in GPU memory. The multi-GPU setting uses SLATE (Software for Linear Algebra Targeting Exascale) as a modern GPU-aware replacement for ScaLAPACK. On 4 nodes of SUMMIT the code runs ~10X faster when using all 24 V100 GPUs compared to when it only uses the 168 POWER9 cores. On 8 SUMMIT nodes, using 48 V100 GPUs, the sparse solver reaches over 50TFlop/s. Compared to SuperLU, on a single V100, for a set of 17 matrices our implementation is faster for all but one matrix, and is on average 5X (median 4X) faster

97 MATHEMATICS AND COMPUTING↗