Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “exascale computing”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 631 records · Page 35

On the energy footprint of I/O management in Exascale HPC systems

The advent of unprecedentedly scalable yet energy hungry Exascale supercomputers poses a major challenge in sustaining a high performance-per-watt ratio. With I/O management acquiring a crucial role in supporting scientific simulations, various I/O management approaches have been proposed to achieve high performance and scalability. But, the details of how these approaches affect energy consumption have not been studied yet. Therefore, this paper aims to explore how much energy a supercomputer consumes while running scientific simulations when adopting various I/O management approaches. In particular, we closely examine three radically different I/O schemes including time partitioning, dedicated cores, and dedicated nodes. To accomplish this, we implement the three approaches within the Damaris I/O middleware and perform extensive experiments with one of the target HPC applications of the Blue Waters sustained-petaflop supercomputer project: the CM1 atmospheric model. Our experimental results obtained on the French Grid'5000 platform highlight the differences among these three approaches and illustrate in which way various configurations of the application and of the system can impact performance and energy consumption. Moreover, we propose and validate a mathematical model that estimates the energy consumption of a HPC simulation under different I/O approaches. This proposed model gives hints to pre-select the most energy-efficient I/O approach for a particular simulation on a particular HPC system and therefore provides a step towards energy-efficient HPC simulations in Exascale systems. To the best of our knowledge, our work provides the first in-depth look into the energy-performance tradeoffs of I/O management approaches.

97 MATHEMATICS AND COMPUTING↗

Multiscale Nuclear-Electronic Orbital Quantum Dynamics in Complex Environments

Many renewable energy conversion processes rely on the movement of protons as well as electrons through either electrocatalysis or photoexcitation. The simulation of such processes requires a quantum mechanical description of coupled nuclear-electronic dynamics in a solvent or heterogeneous chemical environment. The overall objective of this project is the development of theoretical and computational capabilities for simulating nuclear-electronic quantum dynamics in complex environments and the creation of high-performance, open-source software. This multiscale framework will enable simulations of the real-time dynamics of nonequilibrium excited state proton-coupled electron transfer, quantum decoherence, vibronic energy transfer, and ultrafast radiolysis, as well as their associated time-resolved multidimensional spectroscopies. An important outcome of this project will be a sustainable, reusable, and interoperable open-source software ecosystem. This software will be designed for emerging exascale and future national leadership computers. Another key outcome will be a multiscale quantum dynamics method and software enabling simulations of nonequilibrium nuclear-electronic quantum dynamics in complex environments.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

High Fidelity Computational Model for Fluidized Bed Experiments

MFiX was integrated with preconditioners and linear solver packages in Trilinos via MFiX, Fortran, C, and C++ wrappers. The MFiX wrapper interpreted the structure of the matrix and vector from MFiX while the Fortran wrapper transfered this information to a low-level language, C. The C-wrapper transformed the memory semantic between Fortran and C++ language. The C++ wrapper passed the matrix and vector to a Trilinos package and got the solution. This solution was transferred to MFiX via C++, C, Fortran, MFiX wrapper. The framework was used to integrate first, second, and third generation linear solvers in Trilinos with MFiX.

20 FOSSIL-FUELED POWER PLANTS↗

High Performance Computing - Power Application Programming Interface Specification

Achieving practical exascale supercomputing will require massive increases in energy efficiency. The bulk of this improvement will likely be derived from hardware advances such as improved semiconductor device technologies and tighter integration, hopefully resulting in more energy efficient computer architectures. Still, software will have an important role to play. With every generation of new hardware, more power measurement and control capabilities are exposed. Many of these features require software involvement to maximize feature benefits. This trend will allow algorithm designers to add power and energy efficiency to their optimization criteria. Similarly, at the system level, opportunities now exist for energy-aware scheduling to meet external utility constraints such as time of day cost charging and power ramp rate limitations. Finally, future architectures might not be able to operate all components at full capability for a range of reasons including temperature considerations or power delivery limitations. Software will need to make appropriate choices about how to allocate the available power budget given many, sometimes conflicting considerations.

97 MATHEMATICS AND COMPUTING↗

Checkpoint-Restart and Platform Design Criteria

Long computations often use checkpoint/restart to survive the inevitable failures. Optimal checkpoint/restart strategy can be derived in terms of special functions, and approximated within the regime of interest. As a further step under nominal assumptions, platforms can be optimized for maximum progress rate, computed in terms of non-dimensional parameter JMTTI/Delta, where JMTTI is the Job Mean Time To Interrupt and Delta is the time it takes to write (or read) a checkpoint. Scaling to a larger platform of the same technology requires scaling checkpoint bandwidth quadratically to maintain the same progress rate. This was observed during the design of LANL’s Roadrunner in 2007, led to burst buffers proposed in 2009 and introduced on Trinity in 2015. The need to compute reliably continues to impact the design of future exascale platforms.

97 MATHEMATICS AND COMPUTING↗

Planning for Pre-Exascale Platform Environment (Fiscal Year 2015 Level 2 Milestone 5216)

This Plan for ASC Pre-Exascale Platform Environments document constitutes the deliverable for the fiscal year 2015 (FY15) Advanced Simulation and Computing (ASC) Program Level 2 milestone Planning for Pre-Exascale Platform Environment. It acknowledges and quantifies challenges and recognized gaps for moving the ASC Program towards effective use of exascale platforms and recommends strategies to address these gaps. This document also presents an update to the concerns, strategies, and plans presented in the FY08 predecessor document that dealt with the upcoming (at the time) petascale high performance computing (HPC) platforms. With the looming push towards exascale systems, a review of the earlier document was appropriate in light of the myriad architectural choices currently under consideration. The ASC Program believes the platforms to be fielded in the 2020s will be fundamentally different systems that stress ASC’s ability to modify codes to take full advantage of new or unique features. In addition, the scale of components will increase the difficulty of maintaining an errorfree system, thus driving new approaches to resilience and error detection/correction. The code revamps of the past, from serial- to vector-centric code to distributed memory to threaded implementations, will be revisited as codes adapt to a new message passing interface (MPI) plus “x” or more advanced and dynamic programming models based on architectural specifics. Development efforts are already underway in some cases, and more difficult or uncertain aspects of the new architectures will require research and analysis that may inform future directions for program choices. In addition, the potential diversity of system architectures may require parallel if not duplicative efforts to analyze and modify environments, codes, subsystems, libraries, debugging tools, and performance analysis techniques as well as exploring new monitoring methodologies. It is difficult if not impossible to selectively eliminate some of these activities until more information is available through simulations of potential architectures, analysis of systems designs, and informed study of commodity technologies that will be the constituent parts of future platforms.

97 MATHEMATICS AND COMPUTING↗

Improving MPI Collective I/O for High Volume Non-Contiguous Requests With Intra-Node Aggregation

Two-phase I/O is a well-known strategy for implementing collective MPI-IO functions. It redistributes I/O requests among the calling processes into a form that minimizes the file access costs. As modern parallel computers continue to grow into the exascale era, the communication cost of such request redistribution can quickly overwhelm collective I/O performance. In this study, this effect has been observed from parallel jobs that run on multiple compute nodes with a high count of MPI processes on each node. Furthermore, to reduce the communication cost, we present a new design for collective I/O by adding an extra communication layer that performs request aggregation among processes within the same compute nodes. This approach can significantly reduce inter-node communication contention when redistributing the I/O requests. We evaluate the performance and compare it with the original two-phase I/O on Cray XC40 parallel computers (Theta and Cori) with Intel KNL and Haswell processors. Using I/O patterns from two large-scale production applications and an I/O benchmark, we show our proposed method effectively reduces the communication cost and hence maintains the scalability for a large number of processes.

97 MATHEMATICS AND COMPUTING↗

Advanced Technology and Mitigation (ATDM) SPARC Re-Entry Code Fiscal Year 2017 Progress and Accomplishments for ECP.

The SPARC (Sandia Parallel Aerodynamics and Reentry Code) will provide nuclear weapon qualification evidence for the random vibration and thermal environments created by re-entry of a warhead into the earth’s atmosphere. SPARC incorporates the innovative approaches of ATDM projects on several fronts including: effective harnessing of heterogeneous compute nodes using Kokkos, exascale-ready parallel scalability through asynchronous multi-tasking, uncertainty quantification through Sacado integration, implementation of state-of-the-art reentry physics and multiscale models, use of advanced verification and validation methods, and enabling of improved workflows for users. SPARC is being developed primarily for the Department of Energy nuclear weapon program, with additional development and use of the code is being supported by the Department of Defense for conventional weapons programs.

45 MILITARY TECHNOLOGY, WEAPONRY, AND NATIONAL DEF↗

Collaborative Research: Advancing Arctic Climate Projection Capability at Seasonal to Decadal Scales (Final Technical Report)

The Regional Arctic System Model (RASM) at process resolving configurations has been used to (i) advance understanding of physical processes and feedbacks involved in Arctic amplification and (ii) understand and potentially reduce uncertainty in prediction of arctic climate change at seasonal to decadal scales. RASM consists the atmosphere (Weather and Research Forecasting model, WRF), ocean (Parallel Ocean Program, POP), sea ice (CICE), land hydrology (Variable Infiltration Capacity model, VIC), river routing scheme (RVIC), marine biogeochemistry components and the coupling framework (CPL7). Its domain is pan-Arctic, with the atmosphere and land components configured on a 50-km or 25-km grid and four configurations of the ocean and sea ice components: 1/12°(~9.3km) or 1/48°(~2.4km) and 45 or 60 vertical layers. These RASM configurations have been motivated by the emerging exascale capability for high performance computing to improve model fidelity. The dynamical downscaling of reanalysis allows comparison of RASM results with observations in place and time to: (i) advance system level understanding of physical processes and coupling involved in an event, (ii) optimize model parameter space, (iii) diagnose and reduce model biases and (iv) produce realistic and consistent across all the components initial conditions for predictions and predictability studies, which are all unique capabilities not available in global Earth System Models (ESMs). An evaluation of RASM 1.0 (Cassano et al. 2017) revealed that it had a cold bias over the oceans and a warm bias over land areas due largely to cloud and radiation biases in the model, with too little cloud cover simulated over land and too much cloud cover simulated over sub-polar oceans. This study has motivated an upgrade to WRF version 3.7.1 in RASM and allowed for the inclusion of the radiative impact of convective clouds. A variety of atmospheric physics parameterizations were evaluated against observations (e.g. data from the Arctic Clouds in Summer Experiment (ACSE); Sedlar et al. 2020) to identify an optimal suite of WRF physics options in RASM. The RASM with the optimized WRF physics were used to study the impact of strong mesoscale winds over the ocean around the southern tip of Greenland (DuVivier and Cassano 2016) and their impact on oceanic convection (DuVivier et al. 2017a). Data from the PolarWinds field campaign were used to evaluate WRF boundary layer physics and resolution impacts on the simulation of a Greenland barrier wind event (DuVivier et al. 2017b). The RVIC streamflow routing model has been implemented in RASM to realistically represent high-resolution streamflow processes (Hamman et al. 2017) and to couple the land buoyancy fluxes to the ocean. The RASM-RVIC high-resolution data set of all coastal freshwater fluxes for the Arctic drainage basin and surrounding areas for 1979-2014 was published as a separate product (https://doi.org/10.5281/zenodo.293037). The fidelity of atmospheric momentum transfer to and the response of polar marine Ekman layer in RASM and Community Earth System Model (CESM) was investigated by Roberts et al. (2015). The increased frequency of oceanic flux exchange in CESM, following the RASM guidance, caused a considerable increase in the median inertial ice speed across the Southern Ocean and parts of the Arctic. A comprehensive evaluation of the RASM1.0 atmosphere-ocean-sea ice-land interface was completed by Brunke et al. (2018). RASM was also demonstrated for its capability to simulate extreme events in agreement with observations in space and time (Lee et al. submitted). In particular, the development of three open water events, known as polynyas, have been simulated north of Greenland in February of 2011, 2017 and 2018, in agreement with satellite observations for the past four decades. The optimized RASM sea ice results have been favorably evaluated against satellite observations and a subset of eleven CMIP6 models (Watts et al. submitted). In a complementary project, Jin et al. (2018) have shown that RASM with higher-resolution and new sea-ice processes contributed to lower model errors in sea-ice conditions, concentrations of nutrients and ice algae, in comparison to results from the coarse-resolution (1°) CESM. In two other complementary studies, RASM results were used (i) to explain areas of concentrated use by bowhead whales, the seasonal progression in the use, and the physical environment within those areas (Citta et al. 2015) and (ii) for a synthesis of fall bowhead whales distribution and migration in the Bering-Chukchi-Beaufort (BCB) Sea to investigate whale movements and feeding to the local ocean hydrography and currents (Citta et al. 2018). However, the multi-decadal output from the CESM Large Ensemble yielded unrealistic forcing. Instead, the shorter NCEP CFSv2 9-month forecasts were successfully tested and afforded an increased ensemble size (~30) to demonstrate gains of dynamical downscaling at sub-seasonal to intra-annual time scales. The improved model physics and coupling among RASM model components have yielded more realistic representation of the sea ice cover and consistent across all model components initial conditions. Consequently, RASM demonstrates significant gains compared to simulation of sea ice in the NCEP reanalysis. In addition, RASM 6-month ensemble forecasts yield very realistic sea ice distribution, which demonstrates both significant gains of dynamical downscaling and the continued impact of the initial conditions on forecasts out to 6 months (https://nps.edu/web/rasm/predictions). A follow up study using RASM for dynamical downscaling of the more realistic CESM initialized Decadal Prediction Large Ensemble output is currently ongoing as part of the DOE RGMA HiLAT-RASM project.

54 ENVIRONMENTAL SCIENCES↗

Data Transfer Tool (HPC Annual Report V.3.1)

Sandia has been developing and supporting data transfer tools for over 20 years and has the expertise to take DOE into the Extreme Scale era. In looking at Exascale and beyond (Extreme Scale Computing), data sets can be thousands of 500TBs in size, a single file can be in the 100TB range, and billions of files are expected. Huge bursts of data need to be transferred, even today. While data archiving is often not thought about, it is an integral part of the full data management path when data is generated on HPC systems. In order to move generated data to its final resting place (data archive) or to transfer between file systems, a capable data transfer tool is required.

97 MATHEMATICS AND COMPUTING↗

Providing Thermal Stability for an Exascale Supercomputer: A Case Study of Frontier's Cooling System

High performance computing (HPC) systems frequently produce large dynamic power swings, even under typical operating conditions, that can present a significant challenge for their direct-liquid cooling systems. Further, the primary cooling loops that must remove this waste heat have response times measured in minutes while the underlying HPC component thermal stress is measured in seconds. The per-socket power demand for both compute processing units (CPUs) and graphic processing units ( GPUs) continues to increase with each successive generation while case temperatures are declining. New HPC systems are expected to exacerbate the challenge of these dynamic power swings and the impact on effective and timely cooling systems. This paper describes the cooling and controls system for Oak Ridge National Laboratory’s Frontier Supercomputer, the first sustained exascale system, as a case study for this situation. The cooling and control system for Frontier demonstrates specific success, but with a number of trade-offs and decisions that suggest further design and operating optimizations for the community at large to consider.

42 ENGINEERING↗

Turbulence theories and statistical closure approaches

When discussing research in physics and in science more generally, it is common to ascribe equal importance to the three components of the scientific trinity: theoretical, experimental, and computational studies. This review will explore the future of modern turbulence theory by tracing its history, which began in earnest with Kolmogorov’s 1941 analysis of turbulence cascade and inertial range [A.N. Kolmogorov, Dokl. Akad. Nauk SSSR, 30, 299, (1941); 32, 19, (1941)]. The 80th Anniversary of Kolmogorov’s landmark study is a welcome opportunity to survey the achievements and evaluate the future of the theoretical approach of turbulence research. Over the years, turbulence theories have been critically important in laying the foundation of our understanding of the nature of turbulent flows. In particular, the Direct Interaction Approximation (DIA) [R.H. Kraichnan, J. Fluid Mech., 5, 497 (1959)] and its subsequent development, known as the statistical closure approach, can be identified as perhaps the most profound single advancement. The remarkable success of the statistical closure has furnished a platform to study such essential concepts as the energy transfer process and interacting scales, and the roles of the straining and sweeping motions. More recently, the quasi-Lagrangian formulation of V. L’vov & I. Procaccia and Kraichnan’s solvable passive scalar model provided powerful ways to explore another fundamental aspect of turbulent flows, the phenomena of intermittency, and the associated anomalous scaling exponents. In the meantime, the theory of fluid equilibria has been developed to describe the large-scale structures that can emerge from turbulent cascades of two-dimensional and geophysical flows at a later time. And yet, despite all these successes, analytical treatments suffer from mathematical complexities. As a result, the utility of theoretical approaches has been limited to relatively idealized flows. On the other hand, in recent decades, computational abilities and experimental facilities have reached an unprecedented scale. Looking beyond the horizon, the imminent deployment of exascale supercomputers will generate complete datasets of the entire flow field of key benchmark flows, allowing researchers to extract additional measurements concerning fully developed, complex turbulent flow fields far beyond those available from the statistical closure theories. Some other developments that could potentially influence the future course of turbulence theories include the advancement of machine learning, artificial intelligence, and data science; likely disruptions arising from the advent of quantum computation; and the increasingly prominent role of turbulence research in providing more accurate climate scientific data. Finally, turbulence theorists can leverage these developments by asking the right questions and developing advanced, sophisticated frameworks that will be able to predict and correlate vast amounts of data from the other two components of the trinity.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

CALORIE: A Constraint Language and Optimizing Runtime for Exascale Power Management (Final Report)

This final technical report summarizes the key accomplishments on the CALORIE project, a DOE Early Career award received by PI Henry Hoffmann at the University of Chicago. CALORIE’s main goal was to create principled methodologies, tools, and practices to help scientists and high-performance computing (HPC) operators maximize the performance and insights obtained from scientific computing applications in the face of exascale power constraints. The project accomplished these goals by completing the following objectives: • Designing a language for describing application goals (including constraints and objectives) and system capabilities. The application goals will include things like which simulations or simulations plus in situ analysis will be run together, what the power constraints are, and what requirements there are for in situ analysis (for example, a desired frame rate for visualization). The system capabilities include all components that can be adjusted to tradeoff power and performance. • Designing a runtime system that takes specified goals and capabilities and dynamically determines what capabilities to use to meet the goals. This runtime adapts to changes in goals, application behavior, or available capabilities to automatically maintain the goals despite unexpected disturbances. • Developing a foundational understanding of how power constraints affect the problem of scheduling applications in large-scale systems. This report provides an overview of the accomplishments related to each of these key objectives.

97 MATHEMATICS AND COMPUTING↗

Enabling Combustion Science Simulations for Future Exascale Machines

Reacting flow simulations for combustion applications require extensive computing capabilities. Leveraging the AMReX library, the Pele suite of combustion simulation tools targets the largest supercomputers available and future exascale machines. We introduce PeleC, the compressible solver in the Pele suite, and detail its capabilities, including complex geometry representation, chemistry integration, and discretization. We present a comparison of development efforts using both OpenACC and AMReX's C++ performance portability framework for execution on multiple GPU architectures. We discuss relevant details that have allowed PeleC to achieve high performance and scalability. PeleC's performance characteristics are measured through relevant simulations on multiple supercomputers. The success of PeleC's design for exascale is exhibited through demonstration of a 160 billion cell simulation and weak scaling onto 100\% of Summit, an NVIDIA-based GPU supercomputer at Oak Ridge National Laboratory. Our results provide confidence that PeleC will enable future combustion science simulations with unprecedented fidelity.

combustion↗

FFT, FMM, and multigrid on the road to exascale: Performance challenges and opportunities

FFT, FMM, and multigrid methods are widely used fast and highly scalable solvers for elliptic PDEs. However, emerging large-scale computing systems are introducing challenges in comparison to current petascale computers. Recent efforts (Dongarra et al. 2011) have identified several constraints in the design of exascale software that include massive concurrency, resilience management, exploiting the high performance of heterogeneous systems, energy efficiency, and utilizing the deeper and more complex memory hierarchy expected at exascale. In this paper, we perform a model-based comparison of the FFT, FMM, and multigrid methods in the context of these projected constraints. Finally, we use performance models to offer predictions about the expected performance on upcoming exascale system configurations based on current technology trends.

97 MATHEMATICS AND COMPUTING↗

Biological and Environmental Research Exascale Requirements Review. An Office of Science review sponsored jointly by Advanced Scientific Computing Research and Biological and Environmental Research, March 28-31, 2016, Rockville, Maryland

Understanding the fundamentals of genomic systems or the processes governing impactful weather patterns are examples of the types of simulation and modeling performed on the most advanced computing resources in America. High-performance computing and computational science together provide a necessary platform for the mission science conducted by the Biological and Environmental Research (BER) office at the U.S. Department of Energy (DOE). This report reviews BER’s computing needs and their importance for solving some of the toughest problems in BER’s portfolio. BER’s impact on science has been transformative. Mapping the human genome, including the U.S.-supported international Human Genome Project that DOE began in 1987, initiated the era of modern biotechnology and genomics-based systems biology. And since the 1950s, BER has been a core contributor to atmospheric, environmental, and climate science research, beginning with atmospheric circulation studies that were the forerunners of modern Earth system models (ESMs) and by pioneering the implementation of climate codes onto high-performance computers. See http://exascaleage.org/ber/ for more information.

54 ENVIRONMENTAL SCIENCES↗

Large-scale GW calculations on pre-exascale HPC systems

The ab initio GW approach is a rigorous Green’s-function-based framework that can be employed to compute electronic excitation properties of a wide variety of materials such as extended systems, molecules, as well as confined and nanostructured materials with a very satisfactory accuracy. However, GW calculations on complex systems are often hindered by the high computational cost associated with the method. Here, in this study, we demonstrate how to significantly speedup GW calculations with a novel algorithm for the computationally intense kernel based on a non-blocking chunked cyclic communication scheme that minimizes latency in MPI messages, allows for overlapping of communication and computation, and improves cache usage in matrix multiplication operations. The optimized version of the code, implemented in the BerkeleyGW software package, is capable of scaling well to the full Cori computer at NERSC (Cray XC40) and achieves over 11 Peta FLOP/s of sustained performance. We showcase our work by performing large-scale GW calculations of defect structures on silicon, which require simulation cells containing over 1700 atoms, and which can now be efficiently executed in just a few minutes on large pre-exascale high-performance computing systems.

97 MATHEMATICS AND COMPUTING↗