Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “supercomputers”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 847 records · Page 47

Reducing Application Runtime Variability on Jaguar XT5

Operating system (OS) noise is defined as interference generated by the OS that prevents a compute core from performing ``useful'' work. Compute node kernel daemons, network interfaces, and other OS related services are major sources of such interference. This interference on individual compute cores can vary in duration and frequency, and can cause de-synchronization (jitter) in collective communication tasks and thus results in variable (degraded) overall parallel application performance. This behavior is more observable in large-scale applications using certain types of collective communication primitives, such as MPI\_Allreduce. This paper presents our effort towards reducing the overall effect of OS noise on our large-scale parallel applications. Our tests were performed on the quad-core Jaguar, the Cray XT5 at the Oak Ridge National Laboratory Leadership Computing Facility (OLCF). At the time of these tests, Jaguar was a 1.4 PFLOPS supercomputer with 149,504 compute cores and 8 cores per node. We aggregated OS noise sources onto a single core for each node. The scientific application was then run on six of the remaining cores in each node. Our results show that we were able to improve the MPI_Allreduce performance by two orders of magnitude. We demonstrated up to a 30% boost in the performance of the Parallel Ocean Program (POP) using this technique.

97 MATHEMATICS AND COMPUTING↗

Plasticity and Failure in Nanocrystalline BCC Metals via MD Simulation

Advances in the ability to generate extremely high pressures in dynamic experiments such as at the National Ignition Facility has motivated the need for special materials optimized for those conditions as well as ways to probe the response of these materials as they are deformed. We need to develop a much deeper understanding of the behavior of materials subjected to high pressure, especially the effect of rate at the extremely high rates encountered in those experiments. Here we use large-scale molecular dynamics (MD) simulations of the high-rate deformation of nanocrystalline tantalum at pressures less than 100 GPa to investigate the processes associated with plastic deformation for strains up to 100%. We focus on 3D polycrystalline systems with typical grain sizes of 10-20 nm. We also study a rapidly quenched liquid (amorphous solid) tantalum. We apply a constant volume (isochoric), constant temperature (isothermal) shear deformation over a range of strain rates, and compute the resulting stress-strain curves to large strains for both uniaxial and biaxial compression. We study the rate dependence and identify plastic deformation mechanisms. The identification of the mechanisms is facilitated through a novel technique that computes the local grain orientation, returning it as a quaternion for each atom. This analysis technique is robust and fast, and has been used to compute the orientations on the fly during our parallel MD simulations on supercomputers. We find both dislocation and twinning processes are important, and they interact in the weak strain hardening in these extremely fine-grained microstructures. We also present some results on void growth in nanocrystalline BCC metals under tension.

36 MATERIALS SCIENCE↗

Reconciling Scratch Space Consumption, Exposure, and Volatility to Achieve Timely Staging of Job Input Data

Innovative scientific applications and emerging dense data sources are creating a data deluge for high-end computing systems. Processing such large input data typically involves copying (or staging) onto the supercomputer's specialized high-speed storage, scratch space, for sustained high I/O throughput. The current practice of conservatively staging data as early as possible makes the data vulnerable to storage failures, which may entail re-staging and consequently reduced job throughput. To address this, we present a timely staging framework that uses a combination of job startup time predictions, user-specified intermediate nodes, and decentralized data delivery to coincide input data staging with job start-up. By delaying staging to when it is necessary, the exposure to failures and its effects can be reduced. Evaluation using both PlanetLab and simulations based on three years of Jaguar (No. 1 in Top500) job logs show as much as 85.9% reduction in staging times compared to direct transfers, 75.2% reduction in wait time on scratch, and 2.4% reduction in usage/hour.

99 GENERAL AND MISCELLANEOUS↗

The Automatic Library Tracking Database

A library tracking database has been developed and put into production at the National Institute for Computational Sciences and the Oak Ridge Leadership Computing Facility (both located at Oak Ridge National Laboratory.) The purpose of the library tracking database is to track which libraries are used at link time on Cray XT5 Supercomputers. The database stores the libraries used at link time and also records the executables run in a batch job. With this data, many operationally important questions can be answered such as which libraries are most frequently used and which users are using deprecated libraries or applications. The infrastructure design and reporting mechanisms are presented along with collected production data.

97 MATHEMATICS AND COMPUTING↗

Scalable I/O Tracing and Analysis

As supercomputer performance approached and then surpassed the petaflop level, I/O performance has become a major performance bottleneck for many scientific applications. Several tools exist to collect I/O traces to assist in the analysis of I/O performance problems. However, these tools either produce extremely large trace files that complicate performance analysis, or sacrifice accuracy to collect high-level statistical information. We propose a multi-level trace generator tool, ScalaIOTrace, that collects traces at several levels in the HPC I/O stack. ScalaIOTrace features aggressive trace compression that generates trace files of near constant size for regular I/O patterns and orders of magnitudes smaller for less regular ones. This enables the collection of I/O and communication traces of applications running on thousands of processors. Our contributions also include automated trace analysis to collect selected statistical information of I/O calls by parsing the compressed trace on-the-fly and time-accurate replay of communication events with MPI-IO calls. We evaluated our approach with the Parallel Ocean Program (POP) climate simulation and the FLASH parallel I/O benchmark. POP uses NetCDF as an I/O library while FLASH I/O uses the parallel HDF5 I/O library, which internally maps onto MPI-IO. We collected MPI-IO and low-level POSIX I/O traces to study application I/O behavior. Our results show constant size trace files of only 145KB irrespective of the number of nodes for FLASH I/O benchmark, which exhibits regular I/O and communication pattern. For POP, we observe up to two orders of magnitude reduction in trace file sizes compared to flat traces. Statistical information gathered reveals insight on the number of I/O and communication calls issued in the POP and FLASH I/O. Such concise traces are unprecedented for isolated I/O and combined I/O plus communication tracing.

97 MATHEMATICS AND COMPUTING↗

Engineering PFLOTRAN for Scalable Performance on Cray XT and IBM BlueGene Architectures

We describe PFLOTRAN - a code for simulation of coupled hydro-thermal-chemical processes in variably saturated, non-isothermal, porous media - and the approaches we have employed to obtain scalable performance on some of the largest scale supercomputers in the world. We present detailed analyses of I/O and solver performance on Jaguar, the Cray XT5 at Oak Ridge National Laboratory, and Intrepid, the IBM BlueGene/P at Argonne National Laboratory, that have guided our choice of algorithms.

99 GENERAL AND MISCELLANEOUS↗

Green Machine

Fast, powerful, built for speed, green innovation, scalable by design. Innovative cooling for significant energy and water savings installation flexibility. The Jaguar X. T. Five not only breaks the petaflop barrier for computing speed, it introduces the most revolutionary and cooling technology and supercomputing ecoflex. Video courtesy Cray, Inc.

CRAY↗

Jaguar/Kraken Time Lapse

Time lapse video of the upgrade to ORNL's Jaguar and the University of Tennessee's Kraken, the world's fastest and third-fastest supercomputers.

97 MATHEMATICS AND COMPUTING↗

Three dimensional simulations of Richtmyer-Meshkov instabilities in gas-curtain shock-tube experiments

It is not feasible to compute high Reynolds-number (Re) turbulent flows by directly resolving all scales of motion and material interfaces; instead, macroscale portions of the unsteady turbulent motion are computed while the rest of the flow physics including molecular diffusion and other micro scale physics (e.g., combustion) remains unresolved. In large eddy simulation (LES), the large energy containing structures are resolved whereas the smaller, presumably more isotropic, structures are filtered out and their unresolved subgrid scale (SGS) effects are modeled. The construction of SGS models for LES is pragmatic and based primarily on empirical information. Adding to the physics based difficulties in developing and validating SGS models, truncation terms due to discretization are comparable to SGS models in typical LES strategies, and LES resolution requirements become prohibitively expensive for practical flows and regimes. Implicit LES (ILES) - and monotone integrated LES (MILES) introduced earlier, effectively address the seemingly insurmountable issues posed to LES by underresolution, by relying on the use of SGS modeling and filtering provided implicitly by physics capturing numerics. Extensive work has demonstrated that predictive unresolved simulations of turbulent velocity fields are possible using any of the class of nonoscillatory finite-volume (NFV) numerical algorithms. Popular NFV methods such as flux-corrected transport (FCT), the piecewise parabolic method (PPM), total variation diminishing (TVD), and hybrid algorithms are being used for ILES. In many applications of interest, turbulence is generated by shock waves via Richtmyer-Meshkov instabilities (RMI). The instability results in vorticity being introduced at material interfaces by the impulsive loading of the shock wave. A critical feature of this impulsive driving is that the turbulence decays as dissipation removes kinetic energy from the system. RMI add the complexity of shock waves and other compressible effects to the basic physics associated with mixing; compressibility further affects the basic nature of material mixing when mass density and material mixing fluctuation effects are not negligible. Because RMI are shock-driven, resolution requirements make direct numerical simulation impossible even on the largest supercomputers. State-of-the-art simulations of RMI instabilities use hybrid methods which require switching between the use of shock capturing schemes and conventional (explicit) LES depending on the local flow conditions. On the other hand. because ILES is based on (locally adaptive) NFV methods it is naturally capable of emulating shock physics. The unique combination of shock and turbulence emulation capabilities supports direct use of ILES as an effective simulation anzatz in RMI research. This possibility is explored in the current paper in the context of a prototypical case study for which the available laboratory data can be used to test and validate ILES modeling based on various NFV methods.

74 ATOMIC AND MOLECULAR PHYSICS↗

Plasticity and Failure in Nanocrystalline BCC Metals via Molecular Dynamics Simulation

Advances in the ability to generate extremely high pressures in dynamic experiments such as at the National Ignition Facility has motivated the need for special materials optimized for those conditions as well as ways to probe the response of these materials as they are deformed. We need to develop a much deeper understanding of the behavior of materials subjected to high pressure, especially the effect of rate at the extremely high rates encountered in those experiments. Here we use large-scale molecular dynamics (MD) simulations of the high-rate deformation of nanocrystalline tantalum at pressures less than 100 GPa to investigate the processes associated with plastic deformation for strains up to 100%. We focus on 3D polycrystalline systems with typical grain sizes of 10-20 nm. We also study a rapidly quenched liquid (amorphous solid) tantalum. We apply a constant volume (isochoric), constant temperature (isothermal) shear deformation over a range of strain rates, and compute the resulting stress-strain curves to large strains for both uniaxial and biaxial compression. We study the rate dependence and identify plastic deformation mechanisms. The identification of the mechanisms is facilitated through a novel technique that computes the local grain orientation, returning it as a quaternion for each atom. This analysis technique is robust and fast, and has been used to compute the orientations on the fly during our parallel MD simulations on supercomputers. We find both dislocation and twinning processes are important, and they interact in the weak strain hardening in these extremely fine-grained microstructures. We also present some results on void growth in nanocrystalline BCC metals under tension.

36 MATERIALS SCIENCE↗

Functional Partitioning to Optimize End-to-End Performance on Many-core Architectures

Scaling computations on emerging massive-core supercomputers is a daunting task, which coupled with the significantly lagging system I/O capabilities exacerbates applications end-to-end performance. The I/O bottleneck often negates potential performance benefits of assigning additional compute cores to an application. In this paper, we address this issue via a novel functional partitioning (FP) runtime environment that allocates cores to specific application tasks - checkpointing, de-duplication, and scientific data format transformation - so that the deluge of cores can be brought to bear on the entire gamut of application activities. The focus is on utilizing the extra cores to support HPC application I/O activities and also leverage solid-state disks in this context. For example, our evaluation shows that dedicating 1 core on an oct-core machine for checkpointing and its assist tasks using FP can improve overall execution time of a FLASH benchmark on 80 and 160 cores by 43.95% and 41.34%, respectively.

97 MATHEMATICS AND COMPUTING↗

Evaluating the performance of parallel subsurface simulators: An illustrative example with PFLOTRAN

To better inform the subsurface scientist on the expected performance of parallel simulators, this work investigates performance of the reactive multiphase flow and multicomponent biogeochemical transport code PFLOTRAN as it is applied to several realistic modeling scenarios run on the Jaguar supercomputer. After a brief introduction to the code's parallel layout and code design, PFLOTRAN's parallel performance (measured through strong and weak scalability analyses) is evaluated in the context of conceptual model layout, software and algorithmic design, and known hardware limitations. PFLOTRAN scales well (with regard to strong scaling) for three realistic problem scenarios: (1) in situ leaching of copper from a mineral ore deposit within a 5-spot flow regime, (2) transient flow and solute transport within a regional doublet, and (3) a real-world problem involving uranium surface complexation within a heterogeneous and extremely dynamic variably saturated flow field. Weak scalability is discussed in detail for the regional doublet problem, and several difficulties with its interpretation are noted.

54 ENVIRONMENTAL SCIENCES↗

A new synoptic scale resolving global climate simulation using the Community Earth System Model

Abstract High‐resolution global climate modeling holds the promise of capturing planetary‐scale climate modes and small‐scale (regional and sometimes extreme) features simultaneously, including their mutual interaction. This paper discusses a new state‐of‐the‐art high‐resolution Community Earth System Model (CESM) simulation that was performed with these goals in mind. The atmospheric component was at 0.25° grid spacing, and ocean component at 0.1°. One hundred years of “present‐day” simulation were completed. Major results were that annual mean sea surface temperature (SST) in the equatorial Pacific and El‐Niño Southern Oscillation variability were well simulated compared to standard resolution models. Tropical and southern Atlantic SST also had much reduced bias compared to previous versions of the model. In addition, the high resolution of the model enabled small‐scale features of the climate system to be represented, such as air‐sea interaction over ocean frontal zones, mesoscale systems generated by the Rockies, and Tropical Cyclones. Associated single component runs and standard resolution coupled runs are used to help attribute the strengths and weaknesses of the fully coupled run. The high‐resolution run employed 23,404 cores, costing 250 thousand processor‐hours per simulated year and made about two simulated years per day on the NCAR‐Wyoming supercomputer “Yellowstone.”

Small, R. Justin↗

Pore-Scale and Multiscale Numerical Simulation of Flow and Transport in a Laboratory-Scale Column

Pore-scale models are useful for studying relationships between fundamental processes and phenomena at larger (i.e., Darcy) scales. However, the size of domains that can be simulated with explicit pore-scale resolution is limited by computational and observational constraints. Direct numerical simulation of pore-scale flow and transport is typically performed on millimeter-scale volumes at which X-ray computed tomography (XCT), often used to characterize pore geometry, can achieve micrometer resolution. In contrast, the scale at which a continuum approximation of a porous medium is valid is usually larger, on the order of centimeters to decimeters. Furthermore, laboratory experiments that measure continuum properties are typically performed on decimeter-scale columns. At this scale, XCT resolution is coarse (tens to hundreds of micrometers) and prohibits characterization of small pores and grains. We performed simulations of pore-scale processes over a decimeter-scale volume of natural porous media with a wide range of grain sizes, and compared to results of column experiments using the same sample. Simulations were conducted using high-performance codes executed on a supercomputer. Two approaches to XCT image segmentation were evaluated, a binary (pores and solids) segmentation and a ternary segmentation that resolved a third category (porous solids with pores smaller than the imaged resolution). We used a mixed Stokes-Darcy simulation method to simulate the combination of Stokes flow in large open pores and Darcy-like flow in porous solid regions. Simulations based on the ternary segmentation provided results that were consistent with experimental observations, demonstrating our ability to successfully model pore-scale flow over a column-scale domain.

CFD↗

Solving Large‐Scale Linear Systems of Equations by a Quantum Hybrid Algorithm

Abstract Today's intermediate‐scale quantum computers, although imperfect, already perform computational tasks that are manifestly beyond the capabilities of modern classical supercomputers. However, so far, quantum‐enabled large‐scale solutions have been realized only for limited set of problems. Here a hybrid algorithm based on phase estimation and classical optimization of the circuit width and depth is employed for solving a specific class of large linear systems of equations ubiquitous to many areas of science and engineering. A classification of linear systems based on the entanglement properties of the associated phase‐estimation unitary operation is introduced, enabling a highly efficient search for solutions that is facilitated by a straightforward matrix‐to‐circuit map. A 2 17 ‐dimensional problem is implemented on several IBM quantum computer superconducting quantum processors, a record‐breaking result for a linear system solved by a quantum computer. Demonstrated realisation sets a clear benchmark in the quest for the future quantum speedup in the linear systems of equations solution.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

Genetic algorithm based task reordering to improve the performance of batch scheduled massively parallel scientific applications

The growth in size of networked high performance computers along with novel accelerator-based node architectures has further emphasized the importance of communication efficiency in high performance computing. The world's largest high performance computers are usually operated as shared user facilities due to the costs of acquisition and operation. Applications are scheduled for execution in a shared environment and are placed on nodes that are not necessarily contiguous on the interconnect. Furthermore, the placement of tasks on the nodes allocated by the scheduler is sub-optimal, leading to performance loss and variability. Here, we investigate the impact of task placement on the performance of two massively parallel application codes on the Titan supercomputer, a turbulent combustion flow solver (S3D) and a molecular dynamics code (LAMMPS). Benchmark studies show a significant deviation from ideal weak scaling and variability in performance. The inter-task communication distance was determined to be one of the significant contributors to the performance degradation and variability. A genetic algorithm-based parallel optimization technique was used to optimize the task ordering. This technique provides an improved placement of the tasks on the nodes, taking into account the application's communication topology and the system interconnect topology. As a result, application benchmarks after task reordering through genetic algorithm show a significant improvement in performance and reduction in variability, therefore enabling the applications to achieve better time to solution and scalability on Titan during production.

97 MATHEMATICS AND COMPUTING↗

Performance analysis of distributed symmetric sparse matrix vector multiplication algorithm for multi-core architectures

In this article, sparse matrix vector multiply (SpMVM) is an important kernel that frequently arises in high performance computing applications. Due to its low arithmetic intensity, several approaches have been proposed in literature to improve its scalability and efficiency in large scale computations. In this paper, our target systems are high end multi-core architectures and we use messaging passing interface + open multiprocessing hybrid programming model for parallelism. We analyze the performance of recently proposed implementation of the distributed symmetric SpMVM, originally developed for large sparse symmetric matrices arising in ab initio nuclear structure calculations. We also study important features of this implementation and compare with previously reported implementations that do not exploit underlying symmetry. Our SpMVM implementations leverage the hybrid paradigm to efficiently overlap expensive communications with computations. Our main comparison criterion is the "CPU core hours" metric, which is the main measure of resource usage on supercomputers. We analyze the effects of topology-aware mapping heuristic using simplified network load model. Furthermore, we have tested the different SpMVM implementations on two large clusters with 3D Torus and Dragonfly topology. Our results show that the distributed SpMVM implementation that exploits matrix symmetry and hides communication yields the best value for the "CPU core hours" metric and significantly reduces data movement overheads.

97 MATHEMATICS AND COMPUTING↗

Preparing NERSC users for Cori, a Cray XC40 system with Intel many integrated cores

The newest NERSC supercomputer Cori is a Cray XC40 system consisting of 2,388 Intel Xeon Haswell nodes and 9,688 Intel Xeon-Phi “Knights Landing” (KNL) nodes. Compared to the Xeon-based clusters NERSC users are familiar with, optimal performance on Cori requires consideration of KNL mode settings; process, thread, and memory affinity; fine-grain parallelization; vectorization; and use of the high-bandwidth MCDRAM memory. This paper describes our efforts preparing NERSC users for KNL through the NERSC Exascale Science Application Program, Web documentation, and user training. We discuss how we configured the Cori system for usability and productivity, addressing programming concerns, batch system configurations, and default KNL cluster and memory modes. Here, system usage data, job completion analysis, programming and running jobs issues, and a few successful user stories on KNL are presented.

97 MATHEMATICS AND COMPUTING↗