Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Parallel Supercomputers”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 127 records · Page 7

Parallel Multivariate Spatio-Temporal Clustering of Large Ecological Datasets on Hybrid Supercomputers

A proliferation of data from vast networks of remote sensing platforms (satellites, unmanned aircraft systems (UAS), airborne etc.), observational facilities (meteorological, eddy covariance etc.), state-of-the-art sensors, and simulation models offer unprecedented opportunities for scientific discovery. Unsupervised classification is a widely applied data mining approach to derive insights from such data. However, classification of very large data sets is a complex computational problem that requires efficient numerical algorithms and implementations on high performance computing (HPC) platforms. Additionally, increasing power, space, cooling and efficiency requirements has led to the deployment of hybrid supercomputing platforms with complex architectures and memory hierarchies like the Titan system at Oak Ridge National Laboratory. The advent of such accelerated computing architectures offers new challenges and opportunities for big data analytics in general and specifically, large scale cluster analysis in our case. Although there is an existing body of work on parallel cluster analysis, those approaches do not fully meet the needs imposed by the nature and size of our large data sets. Moreover, they had scaling limitations and were mostly limited to traditional distributed memory computing platforms. We present a parallel Multivariate Spatio-Temporal Clustering (MSTC) technique based on k-means cluster analysis that can target hybrid supercomputers like Titan. We developed a hybrid MPI, CUDA and OpenACC implementation that can utilize both CPU and GPU resources on computational nodes. We describe performance results on Titan that demonstrate the scalability and efficacy of our approach in processing large ecological data sets.

Sreepathi, Sarat↗

Domain decomposition in the GPU-accelerated Shift Monte Carlo code

The GPU solver within the Shift continuous-energy Monte Carlo neutron transport code has been extended to provide domain decomposition in addition to domain replication to enable the solution of problems with memory requirements exceeding the capacity of a single GPU. The strategy follows the Multiple Set, Overlapping Domain (MSOD) approach that is used in Shift’s CPU solver and integrates into the event-based algorithm used for Shift’s GPU solver. Furthermore, the ability to assign processors to spatial domains non-uniformly has been maintained. In this work, two different approaches for communicating particle data between domains are considered, and multiple criteria for load balancing problems have been investigated. Numerical results are presented for both fresh and depleted small modular nuclear reactor (SMR) cores. A parallel efficiency of approximately 80% was achieved with up to 16 spatial domains measured relative to full domain replication. A scaling study on the Summit supercomputer demonstrates a weak scaling parallel efficiency of over 90% on over 24000 GPUs.

97 MATHEMATICS AND COMPUTING↗

Collaborative Research: A Model of Partially Ionized Plasma Flows with Kinetic Treatment of Neutral Atoms and Nonthermal Ions

Interactions of flows of partially ionized, magnetized plasma are frequently accompanied by the presence of both thermal and non-thermal (pickup) ion components. Such interactions cannot be modeled using traditional MHD equations and require more advanced approaches to treat them. If a nonthermal component of ions is formed due to charge exchange and collisions between the thermal (core) ions and neutrals, it experiences the action of magnetic field, its distribution function is isotropized, and it soon acquires the velocity of the ambient plasma without being thermodynamically equilibrated. This situation, e. g., takes place in the outer heliosphere –- the part of interstellar space beyond the solar system whose properties are determined by the solar wind interaction with the local interstellar medium. This is also possible in laboratory, at million degrees and above, when plasma is conducting electricity far too well, which makes Ohmic heating ineffective. To attain the target temperatures one needs additional heating eventually playing a dominant role. Among such sources is a so-called neutral particle beam heating. This is a wide-spread technique (Joint European Torus and International Thermonuclear Experimental Reactor experiments) based on the injection of powerful beams of neutral atoms into ohmically preheated plasma. In this project we have investigated the energy and density separation between the thermal and nonthermal components in the solar wind and interstellar plasmas. A new model has been developed in which we solve the ideal MHD equations for mixture of all ions and the kinetic Boltzmann equation to describe the transport of neutral atoms. As a separate capability, we can treat the flow of neutral atoms in a multi-component fashion, where neutral atoms born in each thermodynamically distinct region are governed by the Euler gas dynamic equations. We also describe the behavior of pickup ions either kinetically, using the Fokker--Planck equation, or as a separate fluid. Our numerical simulations have demonstrated that pickup ions play a major role in the interaction of the solar wind and (partially ionized) interstellar medium plasmas. Our teams have investigated the stability of the surface (the heliopause) that separates the solar wind from the local interstellar medium, the transport of galactic cosmic rays, the properties of the heliotail flow, and modifications to the bow wave in front of the heliopause due to charge exchange between the neutral H atoms born in the solar wind and interstellar ions. Modeling results have been validated against observational data, such as obtained by the Interstellar Boundary Explorer (IBEX), and made it possible to shed light on the structure of energetic neutral atom maps created by this spacecraft.. We have also demonstrated that charge-exchange modulated heliosphere is a source of anisotropy of the multi-TeV cosmic ray flux observed in a number of Earth-bound air shower experiments. Newly developed codes are implemented within a Multi-Scale Fluid-Kinetic Simulation Suite (MS-FLUKSS), a publicly available code being developed by our team for over 12 years. MS-FLUKSS scales well up to 160,000 computing cores and has been ported on major supercomputers in the country. Efficient parallelization and data choreography in the continuum simulation modules are provided by Chombo, an adaptive mesh refinement framework managed by Phillip Colella's team at LBNL. We have implemented in-house, hybrid (MPI+OpenMP) parallelization of the kinetic modules that solve the Boltzmann equation with a Monte Carlo method. Currently, the kinetic modules are being rewritten to take advantage of the modern CPU-GPU supercomputer architecture. The scope of the project allowed us to enhance plasma research and education in such broad, multidisciplinary field as physics of partially ionized plasma and its application to space physics and fusion science. Besides the impact on the modeling of complex physical systems, our approach to computational resource management for complex codes utilizing multiple algorithm technologies appears to be a major advance on current approaches. The development of sophisticated resource management will be essential for all future modeling efforts that incorporate a diversity of scales and physical processes. Our effort provided leadership in promoting computational science and plasma physics within the UAH and FIT campuses and, through the training of a broad spectrum of scientists and engineers, fostering new technologies across the country.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗

Collaborative Research: A Model of Partially Ionized Plasma Flows with Kinetic Treatment of Neutral Atoms and Nonthermal Ions

Interactions of flows of partially ionized, magnetized plasma are frequently accompanied by the presence of both thermal and non-thermal (pickup) ion components. Such interactions cannot be modeled using traditional MHD equations and require more advanced approaches to treat them. If a nonthermal component of ions is formed due to charge exchange and collisions between the thermal (core) ions and neutrals, it experiences the action of magnetic field, its distribution function is isotropized, and it soon acquires the velocity of the ambient plasma without being thermodynamically equilibrated. This situation, e. g., takes place in the outer heliosphere - the part of interstellar space beyond the solar system whose properties are determined by the solar wind interaction with the local interstellar medium. This is also possible in laboratory, at million degrees and above, when plasma is conducting electricity far too well, which makes Ohmic heating ineffective. To attain the target temperatures one needs additional heating eventually playing a dominant role. Among such sources is a so-called neutral particle beam heating. This is a wide-spread technique (Joint European Torus and International Thermonuclear Experimental Reactor experiments) based on the injection of powerful beams of neutral atoms into ohmically preheated plasma. In this project we have investigated the energy and density separation between the thermal and nonthermal components in the solar wind and interstellar plasmas. A new model has been developed in which we solve the ideal MHD equations for mixture of all ions and the kinetic Boltzmann equation to describe the transport of neutral atoms. As a separate capability, we can treat the flow of neutral atoms in a multi-component fashion, where neutral atoms born in each thermodynamically distinct regions are governed by the Euler gas dynamic equations. We also describe the behavior of pickup ions either kinetically, using the Fokker–Planck equation, or as a separate fluid. Our numerical simulations have demonstrated that pickup ions play a major role in the interaction of the solar wind and (partially ionized) interstellar medium plasmas. Our teams have investigated the stability of the surface (the heliopause) that separates the solar wind from the local interstellar medium, the transport of galactic cosmic rays, the properties of the heliotail flow, and modifications to the bow wave in front of the heliopause due to charge exchange between the neutral H atoms born in the solar wind and interstellar ions. Modeling results have been validated against observational data, such as obtained by the Interstellar Boundary Explorer (IBEX), and made it possible to shed light on the structure of energetic neutral atom maps created by this spacecraft.. We have also demonstrated that charge-exchange modulated heliosphere is a source of anisotropy of the multi-TeV cosmic ray flux observed in a number of Earth-bound air shower experiments. Newly developed codes are implemented within a Multi-Scale Fluid-Kinetic Simulation Suite (MS-FLUKSS), a publicly available code being developed by our team for over 12 years. MS-FLUKSS scales well up to 160,000 computing cores and has been ported on major supercomputers in the country. Efficient parallelization and data choreography in the continuum simulation modules are provided by Chombo, an adaptive mesh refinement framework managed by Phillip Colella’s team at LBNL. We have implemented in-house, hybrid (MPI+OpenMP) parallelization of the kinetic modules that solve the Boltzmann equation with a Monte Carlo method. Currently, the kinetic modules are being rewritten to take advantage of the modern CPU-GPU supercomputer architecture. The scope of the project allowed us to enhance plasma research and education in such broad, multidis- ciplinary field as physics of partially ionized plasma and its application to space physics and fusion science. Besides the impact on the modeling of complex physical systems, our approach to computational resource management for complex codes utilizing multiple algorithm technologies appears to be a major advance on current approaches. The development of sophisticated resource management will be essential for all future modeling efforts that incorporate a diversity of scales and physical processes. Our effort provided leadership in promoting computational science and plasma physics within the UAH and FIT campuses and, through the training of a broad spectrum of scientists and engineers, foster new technologies across the country.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗

Factors impacting performance of multithreaded sparse triangular solve.

As computational science applications grow more parallel with multi-core supercomputers having hundreds of thousands of computational cores, it will become increasingly difficult for solvers to scale. Our approach is to use hybrid MPI/threaded numerical algorithms to solve these systems in order to reduce the number of MPI tasks and increase the parallel efficiency of the algorithm. However, we need efficient threaded numerical kernels to run on the multi-core nodes in order to achieve good parallel efficiency. In this paper, we focus on improving the performance of a multithreaded triangular solver, an important kernel for preconditioning. We analyze three factors that affect the parallel performance of this threaded kernel and obtain good scalability on the multi-core nodes for a range of matrix sizes.

99 GENERAL AND MISCELLANEOUS↗

Factors impacting performance of multithreaded triangular solve.

As computational science applications grow more parallel with multi-core supercomputers having hundreds of thousands of computational cores, it will become increasingly difficult for solvers to scale. Our approach is to use hybrid MPI/threaded numerical algorithms to solve these systems in order to reduce the number of MPI tasks and increase the parallel efficiency of the algorithm. However, we need efficient threaded numerical kernels to run on the multi-core nodes in order to achieve good parallel efficiency. In this paper, we focus on improving the performance of a multithreaded triangular solver, an important kernel for preconditioning. We analyze three factors that affect the parallel performance of this threaded kernel and obtain good scalability on the multi-core nodes for a range of matrix sizes.

99 GENERAL AND MISCELLANEOUS↗

SST-GPU: An Execution -Driven CUDA Kernel Scheduler and Streaming-Multiprocessor Compute Model

Programmable accelerators have become commonplace in modern computing systems. Advances in programming models and the availability of massive amounts of data have created a space for massively parallel acceleration where the context for thousands of concurrent threads are resident on-chip. These threads are grouped and interleaved on a cycle-by-cycle basis among several massively parallel computing cores. The design of future supercomputers relies on an ability to model the performance of these massively parallel cores at scale. To address the need for a scalable, decentralized GPU model that can model large GPUs, chiplet-based GPUs and multi-node GPUs, this report details the first steps in integrating the open-source, execution driven GPGPU-Sim into the SST framework. The first stage of this project, creates two elements: a kernel scheduler SST element accepts work from SST CPU models and schedules it to an SM-collection element that performs cycle-by-cycle timing using SSTs Mem Hierarchy to model a flexible memory system.

97 MATHEMATICS AND COMPUTING↗

Scalable I/O Systems via Node-Local Storage: Approaching 1 TB/sec File I/O

In the race to PetaFLOP-speed supercomputing systems, the increase in computational capability has been accompanied by corresponding increases in CPU count, total RAM, and storage capacity. However, a proportional increase in storage bandwidth has lagged behind. In order to improve system reliability and to reduce maintenance effort for modern large-scale systems, system designers have opted to remove node-local storage from the compute nodes. Today's multi-TeraFLOP supercomputers are typically attached to parallel file systems that provide only tens of GBs/s of I/O bandwidth. As a result, such machines have access to much less than 1GB/s of I/O bandwidth per TeraFLOP of compute power, which is below the generally accepted limit required for a well-balanced system. In a many ways, the current I/O bottleneck limits the capabilities of modern supercomputers, specifically in terms of limiting their working sets and restricting fault tolerance techniques, which become critical on systems consisting of tens of thousands of components. This paper resolves the dilemma between high performance and high reliability by presenting an alternative system design which makes use of node-local storage to improve aggregate system I/O bandwidth. In this work, we focus on the checkpointing use-case and present an experimental evaluation of the Scalable Checkpoint/Restart (SCR) library, a new adaptive checkpointing library that uses node-local storage to significantly improve the checkpointing performance of large-scale supercomputers. Experiments show that SCR achieves unprecedented write speeds, reaching a measured 700GB/s of aggregate bandwidth on 8,752 processors and an estimated 1TB/s for a similarly structured machine of 12,500 processors. This corresponds to a speedup of over 70x compared to the bandwidth provided by the 10GB/s parallel file system the cluster uses. Further, SCR can adapt to an environment in which there is wide variation in performance or capacity among the individual node-local storage elements.

99 GENERAL AND MISCELLANEOUS↗

Iso-Energy-Efficiency: An Approach to Power Constrained Parallel Computation

Future large scale high performance supercomputer systems require high energy efficiency to achieve exaflops computational power and beyond. Despite the need to understand energy efficiency in high-performance systems, there are few techniques to evaluate energy efficiency at scale. In this paper, we propose a system level iso-energy-efficiency model to analyze, evaluate and predict energy-performance of data intensive parallel applications with various execution patterns running on large scale power-aware clusters. Our analytical model can help users explore the effects of machine and application dependent characteristics on system energy efficiency and isolate efficient ways to scale system parameters (e.g. processor count, CPU power/frequency, workload size and network bandwidth) to balance energy use and performance. We derive our iso-energy-efficiency model and apply it to the NAS Parallel Benchmarks on two power-aware clusters. Our results indicate that the model accurately predicts total system energy consumption within 5% error on average for parallel applications with various execution and communication patterns. We demonstrate effective use of the model for various application contexts and in scalability decision-making.

Song, Shuaiwen↗

Logically Parallel Communication for Fast MPI+Threads Applications

Supercomputing applications are increasingly adopting the MPI+threads programming model over the traditional "MPI everywhere" approach to better handle the disproportionate increase in the number of cores compared with other on-node resources. In practice, however, most applications observe a slower performance with MPI+threads primarily because of poor communication performance. Recent research efforts on MPI libraries address this bottleneck by mapping logically parallel communication, that is, operations that are not subject to MPI's ordering constraints to the underlying network parallelism. Domain scientists, however, typically do not expose such communication independence information because the existing MPI-3.1 standard's semantics can be limiting. Researchers had initially proposed user-visible endpoints to combat this issue, but such a solution requires intrusive changes to the standard (new APIs). The upcoming MPI-4.0 standard, on the other hand, allows applications to relax unneeded semantics and provides them with many opportunities to express logical communication parallelism. Here in this article, we show how MPI+threads applications can achieve high performance with logically parallel communication. Through application case studies, we compare the capabilities of the new MPI-4.0 standard with those of the existing one and user-visible endpoints (upper bound). Logical communication parallelism can boost the overall performance of an application by over 2x.

97 MATHEMATICS AND COMPUTING↗

A parallel orbital-updating based plane-wave basis method for electronic structure calculations

Highlights: • Propose three parallel orbital-updating based plane-wave basis methods for electronic structure calculations. • These new methods can avoid the generating of large scale eigenvalue problems and then reduce the computational cost. • These new methods allow for two-level parallelization which is particularly interesting for large scale parallelization. • Numerical experiments show that these new methods are reliable and efficient for large scale calculations on modern supercomputers. - Abstract: Motivated by the recently proposed parallel orbital-updating approach in real space method , we propose a parallel orbital-updating based plane-wave basis method for electronic structure calculations, for solving the corresponding eigenvalue problems. In addition, we propose two new modified parallel orbital-updating methods. Compared to the traditional plane-wave methods, our methods allow for two-level parallelization, which is particularly interesting for large scale parallelization. Numerical experiments show that these new methods are more reliable and efficient for large scale calculations on modern supercomputers.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

TOUGH3: A new efficient version of the TOUGH suite of multiphase flow and transport simulators

The TOUGH suite of nonisothermal multiphase flow and transport simulators has been updated by various developers over many years to address a vast range of challenging subsurface problems. The increasing complexity of the simulated processes as well as the growing size of model domains that need to be handled call for an improvement in the simulator's computational robustness and efficiency. Moreover, modifications have been frequently introduced independently, resulting in multiple versions of TOUGH that (1) led to inconsistencies in feature implementation and usage, (2) made code maintenance and development inefficient, and (3) caused confusion to users and developers. TOUGH3—a new base version of TOUGH—addresses these issues. It consolidates both the serial (TOUGH2 V2.1) and parallel (TOUGH2-MP V2.0) implementations, enabling simulations to be performed on desktop computers and supercomputers using a single code. New PETSc parallel linear solvers are added to the existing serial solvers of TOUGH2 and the Aztec solver used in TOUGH2-MP. The PETSc solvers generally perform better than the Aztec solvers in parallel and the internal TOUGH3 linear solver in serial. TOUGH3 also incorporates many new features, addresses bugs, and improves the flexibility of data handling. So due to the improved capabilities and usability, TOUGH3 is more robust and efficient for solving tough and computationally demanding problems in diverse scientific and practical applications related to subsurface flow modeling.

97 MATHEMATICS AND COMPUTING↗

Characterizing output bottlenecks in a supercomputer

Supercomputer I/O loads are often dominated by writes. HPC (High Performance Computing) file systems are designed to absorb these bursty outputs at high bandwidth through massive parallelism. However, the delivered write bandwidth often falls well below the peak. This paper characterizes the data absorption behavior of a center-wide shared Lustre parallel file system on the Jaguar supercomputer. We use a statistical methodology to address the challenges of accurately measuring a shared machine under production load and to obtain the distribution of bandwidth across samples of compute nodes, storage targets, and time intervals. We observe and quantify limitations from competing traffic, contention on storage servers and I/O routers, concurrency limitations in the client compute node operating systems, and the impact of variance (stragglers) on coupled output such as striping. We then examine the implications of our results for application performance and the design of I/O middleware systems on shared supercomputers.

Xie, Bing↗

Characterizing output bottlenecks in a supercomputer

Supercomputer I/O loads are often dominated by writes. HPC (High Performance Computing) file systems are designed to absorb these bursty outputs at high bandwidth through massive parallelism. However, the delivered write bandwidth often falls well below the peak. This paper characterizes the data absorption behavior of a center-wide shared Lustre parallel file system on the Jaguar supercomputer. We use a statistical methodology to address the challenges of accurately measuring a shared machine under production load and to obtain the distribution of bandwidth across samples of compute nodes, storage targets, and time intervals. We observe and quantify limitations from competing traffic, contention on storage servers and I/O routers, concurrency limitations in the client compute node operating systems, and the impact of variance (stragglers) on coupled output such as striping. We then examine the implications of our results for application performance and the design of I/O middleware systems on shared supercomputers.

Xie, Bing↗

Geodesic IO Library

GIO is an application programmer interface (API) or library for high performance IO that is designed for climate models that operate on a geodesic grid. GIO uses a combination of configuration files and API calls to define the mapping of internal model data to output variables in standard climate data formats. The mapping from model data to model output includes method of data slicing, aggregation, and linearization using a morton ordering approach. The data is then persisted through a standard interface that can be implemented using several different high level parallel libraries. Both Parallel NetCDF and NetCDF4 interfaces are provided. GIO is designed to run on supercomputing systems or multi-processor clusters with parallel file systems.

Palmer, Bruce↗

A survey on checkpointing strategies: Should we always checkpoint à la Young/Daly?

The Young/Daly formula provides an approximation of the optimal checkpointing period for a parallel application executing on a supercomputing platform. It was originally designed to handle fail-stop errors for preemptible tightly-coupled applications, but has been extended to other application and resilience frameworks. Here, we provide some background and survey various scenarios to assess the usefulness and limitations of the formula, both for preemptible applications and workflow applications represented as a graph of tasks. We also discuss scenarios with uncertainties, and extend the study to silent errors. We exhibit cases where the optimal period is of a different order than that dictated by the Young/Daly formula, and finally we explain how checkpointing can be further combined with replication.

97 MATHEMATICS AND COMPUTING↗

Accuracy of the explicit energy-conserving particle-in-cell method for under-resolved simulations of capacitively coupled plasma discharges

The traditional explicit electrostatic momentum-conserving particle-in-cell algorithm requires strict resolution of the electron Debye length to deliver numerical stability and accuracy. The explicit electrostatic energy-conserving particle-in-cell algorithm alleviates this constraint with minimal modification to the traditional algorithm, retaining its simplicity, ease of parallelization, and acceleration on modern supercomputing architectures. In this article, we apply the algorithm to model a one-dimensional radio frequency capacitively coupled plasma discharge relevant to industrial applications. The energy-conserving approach closely matches the results from the momentum-conserving algorithm and retains accuracy even for cell sizes up to 8 times the electron Debye length. For even larger cells, the algorithm loses accuracy due to poor resolution of steep gradients within the radio frequency sheath. Accuracy can be recovered by adopting a non-uniform grid, which resolves the sheath and allows for cell sizes up to 32 times the electron Debye length in the quasi-neutral bulk of the discharge. The effect is an up to 8 times reduction in the number of required simulation cells, an improvement that can compound in higher-dimensional simulations. We therefore consider the explicit energy-conserving algorithm as a promising approach to significantly reduce the computational cost of full-scale device simulations and a pathway to delivering kinetic simulation capabilities of use to industry.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗