Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “supercomputer”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 847 records · Page 47

Green Machine

Fast, powerful, built for speed, green innovation, scalable by design. Innovative cooling for significant energy and water savings installation flexibility. The Jaguar X. T. Five not only breaks the petaflop barrier for computing speed, it introduces the most revolutionary and cooling technology and supercomputing ecoflex. Video courtesy Cray, Inc.

CRAY↗

Jaguar/Kraken Time Lapse

Time lapse video of the upgrade to ORNL's Jaguar and the University of Tennessee's Kraken, the world's fastest and third-fastest supercomputers.

97 MATHEMATICS AND COMPUTING↗

Three dimensional simulations of Richtmyer-Meshkov instabilities in gas-curtain shock-tube experiments

It is not feasible to compute high Reynolds-number (Re) turbulent flows by directly resolving all scales of motion and material interfaces; instead, macroscale portions of the unsteady turbulent motion are computed while the rest of the flow physics including molecular diffusion and other micro scale physics (e.g., combustion) remains unresolved. In large eddy simulation (LES), the large energy containing structures are resolved whereas the smaller, presumably more isotropic, structures are filtered out and their unresolved subgrid scale (SGS) effects are modeled. The construction of SGS models for LES is pragmatic and based primarily on empirical information. Adding to the physics based difficulties in developing and validating SGS models, truncation terms due to discretization are comparable to SGS models in typical LES strategies, and LES resolution requirements become prohibitively expensive for practical flows and regimes. Implicit LES (ILES) - and monotone integrated LES (MILES) introduced earlier, effectively address the seemingly insurmountable issues posed to LES by underresolution, by relying on the use of SGS modeling and filtering provided implicitly by physics capturing numerics. Extensive work has demonstrated that predictive unresolved simulations of turbulent velocity fields are possible using any of the class of nonoscillatory finite-volume (NFV) numerical algorithms. Popular NFV methods such as flux-corrected transport (FCT), the piecewise parabolic method (PPM), total variation diminishing (TVD), and hybrid algorithms are being used for ILES. In many applications of interest, turbulence is generated by shock waves via Richtmyer-Meshkov instabilities (RMI). The instability results in vorticity being introduced at material interfaces by the impulsive loading of the shock wave. A critical feature of this impulsive driving is that the turbulence decays as dissipation removes kinetic energy from the system. RMI add the complexity of shock waves and other compressible effects to the basic physics associated with mixing; compressibility further affects the basic nature of material mixing when mass density and material mixing fluctuation effects are not negligible. Because RMI are shock-driven, resolution requirements make direct numerical simulation impossible even on the largest supercomputers. State-of-the-art simulations of RMI instabilities use hybrid methods which require switching between the use of shock capturing schemes and conventional (explicit) LES depending on the local flow conditions. On the other hand. because ILES is based on (locally adaptive) NFV methods it is naturally capable of emulating shock physics. The unique combination of shock and turbulence emulation capabilities supports direct use of ILES as an effective simulation anzatz in RMI research. This possibility is explored in the current paper in the context of a prototypical case study for which the available laboratory data can be used to test and validate ILES modeling based on various NFV methods.

74 ATOMIC AND MOLECULAR PHYSICS↗

Plasticity and Failure in Nanocrystalline BCC Metals via Molecular Dynamics Simulation

Advances in the ability to generate extremely high pressures in dynamic experiments such as at the National Ignition Facility has motivated the need for special materials optimized for those conditions as well as ways to probe the response of these materials as they are deformed. We need to develop a much deeper understanding of the behavior of materials subjected to high pressure, especially the effect of rate at the extremely high rates encountered in those experiments. Here we use large-scale molecular dynamics (MD) simulations of the high-rate deformation of nanocrystalline tantalum at pressures less than 100 GPa to investigate the processes associated with plastic deformation for strains up to 100%. We focus on 3D polycrystalline systems with typical grain sizes of 10-20 nm. We also study a rapidly quenched liquid (amorphous solid) tantalum. We apply a constant volume (isochoric), constant temperature (isothermal) shear deformation over a range of strain rates, and compute the resulting stress-strain curves to large strains for both uniaxial and biaxial compression. We study the rate dependence and identify plastic deformation mechanisms. The identification of the mechanisms is facilitated through a novel technique that computes the local grain orientation, returning it as a quaternion for each atom. This analysis technique is robust and fast, and has been used to compute the orientations on the fly during our parallel MD simulations on supercomputers. We find both dislocation and twinning processes are important, and they interact in the weak strain hardening in these extremely fine-grained microstructures. We also present some results on void growth in nanocrystalline BCC metals under tension.

36 MATERIALS SCIENCE↗

Functional Partitioning to Optimize End-to-End Performance on Many-core Architectures

Scaling computations on emerging massive-core supercomputers is a daunting task, which coupled with the significantly lagging system I/O capabilities exacerbates applications end-to-end performance. The I/O bottleneck often negates potential performance benefits of assigning additional compute cores to an application. In this paper, we address this issue via a novel functional partitioning (FP) runtime environment that allocates cores to specific application tasks - checkpointing, de-duplication, and scientific data format transformation - so that the deluge of cores can be brought to bear on the entire gamut of application activities. The focus is on utilizing the extra cores to support HPC application I/O activities and also leverage solid-state disks in this context. For example, our evaluation shows that dedicating 1 core on an oct-core machine for checkpointing and its assist tasks using FP can improve overall execution time of a FLASH benchmark on 80 and 160 cores by 43.95% and 41.34%, respectively.

97 MATHEMATICS AND COMPUTING↗

Evaluating the performance of parallel subsurface simulators: An illustrative example with PFLOTRAN

To better inform the subsurface scientist on the expected performance of parallel simulators, this work investigates performance of the reactive multiphase flow and multicomponent biogeochemical transport code PFLOTRAN as it is applied to several realistic modeling scenarios run on the Jaguar supercomputer. After a brief introduction to the code's parallel layout and code design, PFLOTRAN's parallel performance (measured through strong and weak scalability analyses) is evaluated in the context of conceptual model layout, software and algorithmic design, and known hardware limitations. PFLOTRAN scales well (with regard to strong scaling) for three realistic problem scenarios: (1) in situ leaching of copper from a mineral ore deposit within a 5-spot flow regime, (2) transient flow and solute transport within a regional doublet, and (3) a real-world problem involving uranium surface complexation within a heterogeneous and extremely dynamic variably saturated flow field. Weak scalability is discussed in detail for the regional doublet problem, and several difficulties with its interpretation are noted.

54 ENVIRONMENTAL SCIENCES↗

A new synoptic scale resolving global climate simulation using the Community Earth System Model

Abstract High‐resolution global climate modeling holds the promise of capturing planetary‐scale climate modes and small‐scale (regional and sometimes extreme) features simultaneously, including their mutual interaction. This paper discusses a new state‐of‐the‐art high‐resolution Community Earth System Model (CESM) simulation that was performed with these goals in mind. The atmospheric component was at 0.25° grid spacing, and ocean component at 0.1°. One hundred years of “present‐day” simulation were completed. Major results were that annual mean sea surface temperature (SST) in the equatorial Pacific and El‐Niño Southern Oscillation variability were well simulated compared to standard resolution models. Tropical and southern Atlantic SST also had much reduced bias compared to previous versions of the model. In addition, the high resolution of the model enabled small‐scale features of the climate system to be represented, such as air‐sea interaction over ocean frontal zones, mesoscale systems generated by the Rockies, and Tropical Cyclones. Associated single component runs and standard resolution coupled runs are used to help attribute the strengths and weaknesses of the fully coupled run. The high‐resolution run employed 23,404 cores, costing 250 thousand processor‐hours per simulated year and made about two simulated years per day on the NCAR‐Wyoming supercomputer “Yellowstone.”

Small, R. Justin↗

Pore-Scale and Multiscale Numerical Simulation of Flow and Transport in a Laboratory-Scale Column

Pore-scale models are useful for studying relationships between fundamental processes and phenomena at larger (i.e., Darcy) scales. However, the size of domains that can be simulated with explicit pore-scale resolution is limited by computational and observational constraints. Direct numerical simulation of pore-scale flow and transport is typically performed on millimeter-scale volumes at which X-ray computed tomography (XCT), often used to characterize pore geometry, can achieve micrometer resolution. In contrast, the scale at which a continuum approximation of a porous medium is valid is usually larger, on the order of centimeters to decimeters. Furthermore, laboratory experiments that measure continuum properties are typically performed on decimeter-scale columns. At this scale, XCT resolution is coarse (tens to hundreds of micrometers) and prohibits characterization of small pores and grains. We performed simulations of pore-scale processes over a decimeter-scale volume of natural porous media with a wide range of grain sizes, and compared to results of column experiments using the same sample. Simulations were conducted using high-performance codes executed on a supercomputer. Two approaches to XCT image segmentation were evaluated, a binary (pores and solids) segmentation and a ternary segmentation that resolved a third category (porous solids with pores smaller than the imaged resolution). We used a mixed Stokes-Darcy simulation method to simulate the combination of Stokes flow in large open pores and Darcy-like flow in porous solid regions. Simulations based on the ternary segmentation provided results that were consistent with experimental observations, demonstrating our ability to successfully model pore-scale flow over a column-scale domain.

CFD↗

Solving Large‐Scale Linear Systems of Equations by a Quantum Hybrid Algorithm

Abstract Today's intermediate‐scale quantum computers, although imperfect, already perform computational tasks that are manifestly beyond the capabilities of modern classical supercomputers. However, so far, quantum‐enabled large‐scale solutions have been realized only for limited set of problems. Here a hybrid algorithm based on phase estimation and classical optimization of the circuit width and depth is employed for solving a specific class of large linear systems of equations ubiquitous to many areas of science and engineering. A classification of linear systems based on the entanglement properties of the associated phase‐estimation unitary operation is introduced, enabling a highly efficient search for solutions that is facilitated by a straightforward matrix‐to‐circuit map. A 2 17 ‐dimensional problem is implemented on several IBM quantum computer superconducting quantum processors, a record‐breaking result for a linear system solved by a quantum computer. Demonstrated realisation sets a clear benchmark in the quest for the future quantum speedup in the linear systems of equations solution.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

Genetic algorithm based task reordering to improve the performance of batch scheduled massively parallel scientific applications

The growth in size of networked high performance computers along with novel accelerator-based node architectures has further emphasized the importance of communication efficiency in high performance computing. The world's largest high performance computers are usually operated as shared user facilities due to the costs of acquisition and operation. Applications are scheduled for execution in a shared environment and are placed on nodes that are not necessarily contiguous on the interconnect. Furthermore, the placement of tasks on the nodes allocated by the scheduler is sub-optimal, leading to performance loss and variability. Here, we investigate the impact of task placement on the performance of two massively parallel application codes on the Titan supercomputer, a turbulent combustion flow solver (S3D) and a molecular dynamics code (LAMMPS). Benchmark studies show a significant deviation from ideal weak scaling and variability in performance. The inter-task communication distance was determined to be one of the significant contributors to the performance degradation and variability. A genetic algorithm-based parallel optimization technique was used to optimize the task ordering. This technique provides an improved placement of the tasks on the nodes, taking into account the application's communication topology and the system interconnect topology. As a result, application benchmarks after task reordering through genetic algorithm show a significant improvement in performance and reduction in variability, therefore enabling the applications to achieve better time to solution and scalability on Titan during production.

97 MATHEMATICS AND COMPUTING↗

Performance analysis of distributed symmetric sparse matrix vector multiplication algorithm for multi-core architectures

In this article, sparse matrix vector multiply (SpMVM) is an important kernel that frequently arises in high performance computing applications. Due to its low arithmetic intensity, several approaches have been proposed in literature to improve its scalability and efficiency in large scale computations. In this paper, our target systems are high end multi-core architectures and we use messaging passing interface + open multiprocessing hybrid programming model for parallelism. We analyze the performance of recently proposed implementation of the distributed symmetric SpMVM, originally developed for large sparse symmetric matrices arising in ab initio nuclear structure calculations. We also study important features of this implementation and compare with previously reported implementations that do not exploit underlying symmetry. Our SpMVM implementations leverage the hybrid paradigm to efficiently overlap expensive communications with computations. Our main comparison criterion is the "CPU core hours" metric, which is the main measure of resource usage on supercomputers. We analyze the effects of topology-aware mapping heuristic using simplified network load model. Furthermore, we have tested the different SpMVM implementations on two large clusters with 3D Torus and Dragonfly topology. Our results show that the distributed SpMVM implementation that exploits matrix symmetry and hides communication yields the best value for the "CPU core hours" metric and significantly reduces data movement overheads.

97 MATHEMATICS AND COMPUTING↗

Preparing NERSC users for Cori, a Cray XC40 system with Intel many integrated cores

The newest NERSC supercomputer Cori is a Cray XC40 system consisting of 2,388 Intel Xeon Haswell nodes and 9,688 Intel Xeon-Phi “Knights Landing” (KNL) nodes. Compared to the Xeon-based clusters NERSC users are familiar with, optimal performance on Cori requires consideration of KNL mode settings; process, thread, and memory affinity; fine-grain parallelization; vectorization; and use of the high-bandwidth MCDRAM memory. This paper describes our efforts preparing NERSC users for KNL through the NERSC Exascale Science Application Program, Web documentation, and user training. We discuss how we configured the Cori system for usability and productivity, addressing programming concerns, batch system configurations, and default KNL cluster and memory modes. Here, system usage data, job completion analysis, programming and running jobs issues, and a few successful user stories on KNL are presented.

97 MATHEMATICS AND COMPUTING↗

TensorFlow at Scale: Performance and productivity analysis of distributed training with Horovod, MLSL, and Cray PE ML

Summary Deep learning has proven to be a successful tool for solving a large variety of problems in various scientific fields and beyond. In recent years, the models as well as the available datasets have grown bigger and more complicated, and thus, an increasing amount of computing resources is required in order to train these models in a reasonable amount of time. Besides being able to use HPC resources, deep learning model developers want flexible frameworks which allow for rapid prototyping. One of the most important of these frameworks is Google TensorFlow, which provides both features, ie, good performance as well as flexibility. In this paper, we discuss different solutions for scaling the TensorFlow Framework to thousands of nodes on contemporary Cray XC supercomputing systems.

Kurth, Thorsten↗

A performance analysis of the first generation of HPC‐optimized Arm processors

Summary In this paper, we present performance results from Isambard, the first production supercomputer to be based on Arm CPUs that have been optimized specifically for HPC. Isambard is the first Cray XC50 “Scout” system, combining Cavium ThunderX2 Arm‐based CPUs with Cray's Aries interconnect. The full Isambard system will be delivered in the summer of 2018, when it will contain over 10 000 Arm cores. In this work, we present node‐level performance results from eight early‐access nodes that were upgraded to B0 beta silicon in March 2018. We present node‐level benchmark results comparing ThunderX2 with mainstream CPUs, including Intel Skylake and Broadwell, as well as Xeon Phi. We focus on a range of applications and mini‐apps important to the UK national HPC service, ARCHER, as well as to the Isambard project partners and the wider HPC community. We also compare performance across three major software toolchains available for Arm: Cray's CCE, Arm's version of Clang/Flang/LLVM, and GNU.

97 MATHEMATICS AND COMPUTING↗

Node failure resiliency for Uintah without checkpointing

The frequency of failures in upcoming exascale supercomputers may well be greater than at present due to many-core architectures if component failure rates remain unchanged. This potential increase in failure frequency coupled with I/O challenges at exascale may prove problematic for current resiliency approaches such as checkpoint restarting, although the use of fast intermediate memory may help. Algorithm-Based Fault Tolerance (ABFT) using Adaptive Mesh Refinement (AMR) is one resiliency approach used to address these challenges. For adaptive mesh codes, a coarse mesh version of the solu- tion may be used to restore the fine mesh solution. This paper addresses the implementation of the ABFT approach within the Uintah software framework: both at a software level within Uintah and in the data reconstruction method used for the recovery of lost data. This method has two problems: inaccuracies introduced during the reconstruction propagate forward in time, and the physical consistency of variables such as positivity or boundedness may be violated during interpolation. These challenges can be addressed by the combination of two techniques: 1. a fault-tolerant MPI implementation to recover from runtime node failures, and 2. high-order interpolation schemes to preserve the physical solution and reconstruct lost data. Here, the approach considered here uses a "Limited Essentially Non-Oscillatory" (LENO) scheme along with AMR to rebuild the lost data without checkpointing using Uintah. Experiments were carried out using a fault-tolerant MPI - ULFM to recover from runtime failure, and LENO to recover data on patches belonging to failed ranks, while the simulation was continued to the end. Results show that this ABFT approach is up to 10x faster than the traditional checkpointing method. The new interpolation approach is more accurate than linear interpolation and not subject to the overshoots found in other interpolation methods.

97 MATHEMATICS AND COMPUTING↗

Hierarchical Roofline analysis for GPUs: Accelerating performance optimization for the NERSC‐9 Perlmutter system

Summary The Roofline performance model provides an intuitive and insightful approach to identifying performance bottlenecks and guiding performance optimization. In preparation for the next‐generation supercomputer Perlmutter at NERSC, this paper presents a methodology to construct a hierarchical Roofline on NVIDIA GPUs and extends it to support reduced precision and Tensor Cores. The hierarchical Roofline incorporates L1, L2, device memory, and system memory bandwidths into one single figure, and it offers more profound insights into performance analysis than the traditional DRAM‐only Roofline. We use our Roofline methodology to analyze three proxy applications: GPP from BerkeleyGW, HPGMG from AMReX, and conv2d from TensorFlow. In doing so, we demonstrate the ability of our methodology to readily understand various aspects of performance and performance bottlenecks on NVIDIA GPUs and motivate code optimizations.

Yang, Charlene↗

Interfacing HDF5 with a scalable object‐centric storage system on hierarchical storage

Summary Object storage technologies that take advantage of multitier storage on HPC systems are emerging. However, to use these technologies at present, applications have to be modified significantly from current I/O libraries. HDF5, a widely used I/O middleware on HPC systems, provides a virtual object layer (VOL) that allows applications to connect to different storage mechanisms transparently without requiring significant code modifications. We recently designed the proactive data containers (PDC) object‐centric storage system that provides the capabilities of transparent, asynchronous, and autonomous data movement taking advantage of multiple storage tiers—a decision that has so far been left upon the user on most current systems. To enable PDC's features through HDF5 without modifying application codes, we have developed an HDF5 VOL connector that interfaces with PDC. We present in this article the connector interface and evaluate its performance on Cori, a Cray XC40 supercomputer located at the National Energy Research Scientific Computing Center (NERSC). Our evaluation demonstrates up to an 8× improvement compared with HDF5 that has the most recent optimizations.

Mu, Jingqing↗

A Co-design Framework for Online Data Analysis and Reduction

Science applications preparing for the exascale era are increasingly exploring in situ computations comprising of simulation-analysis-reduction pipelines coupled in-memory. Efficient composition and execution of such complex pipelines for a target platform is a codesign process that evaluates the impact and tradeoffs of various application- and system-specific parameters. In this article, we describe a toolset for automating performance studies of composed HPC applications that perform online data reduction and analysis. We describe Cheetah, a new framework for composing parametric studies on coupled applications, and Savanna, a runtime engine for orchestrating and executing campaigns of codesign experiments. Furthermore, this toolset facilitates understanding the impact of various factors such as process placement, synchronicity of algorithms, and storage versus compute requirements for online analysis of large data. Ultimately, we aim to create a catalog of performance results that can help scientists understand tradeoffs when designing next-generation simulations that make use of online processing techniques. We illustrate the design of Cheetah and Savanna, and present application examples that use this framework to conduct codesign studies on small clusters as well as leadership class supercomputers.

97 MATHEMATICS AND COMPUTING↗