Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “parallel, scaling”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 955 records · Page 53

Approaching the exa-scale: a real-world evaluation of rendering extremely large data sets

Extremely large scale analysis is becoming increasingly important as supercomputers and their simulations move from petascale to exascale. The lack of dedicated hardware acceleration for rendering on today's supercomputing platforms motivates our detailed evaluation of the possibility of interactive rendering on the supercomputer. In order to facilitate our understanding of rendering on the supercomputing platform, we focus on scalability of rendering algorithms and architecture envisioned for exascale datasets. To understand tradeoffs for dealing with extremely large datasets, we compare three different rendering algorithms for large polygonal data: software based ray tracing, software based rasterization and hardware accelerated rasterization. We present a case study of strong and weak scaling of rendering extremely large data on both GPU and CPU based parallel supercomputers using Para View, a parallel visualization tool. Wc use three different data sets: two synthetic and one from a scientific application. At an extreme scale, algorithmic rendering choices make a difference and should be considered while approaching exascale computing, visualization, and analysis. We find software based ray-tracing offers a viable approach for scalable rendering of the projected future massive data sizes.

97 MATHEMATICS AND COMPUTING↗

The JOREK non-linear extended MHD code and applications to large-scale instabilities and their control in magnetically confined fusion plasmas

JOREK is a massively parallel fully implicit non-linear extended magneto-hydrodynamic (MHD) code for realistic tokamak X-point plasmas. It has become a widely used versatile simulation code for studying large-scale plasma instabilities and their control and is continuously developed in an international community with strong involvements in the European fusion research programme and ITER organization. This article gives a comprehensive overview of the physics models implemented, numerical methods applied for solving the equations and physics studies performed with the code. A dedicated section highlights some of the verification work done for the code. A hierarchy of different physics models is available including a free boundary and resistive wall extension and hybrid kinetic-fluid models. The code allows for flux-surface aligned iso-parametric finite element grids in single and double X-point plasmas which can be extended to the true physical walls and uses a robust fully implicit time stepping. Particular focus is laid on plasma edge and scrape-off layer (SOL) physics as well as disruption related phenomena. Among the key results obtained with JOREK regarding plasma edge and SOL, are deep insights into the dynamics of edge localized modes (ELMs), ELM cycles, and ELM control by resonant magnetic perturbations, pellet injection, as well as by vertical magnetic kicks. Also ELM free regimes, detachment physics, the generation and transport of impurities during an ELM, and electrostatic turbulence in the pedestal region are investigated. Regarding disruptions, the focus is on the dynamics of the thermal quench (TQ) and current quench triggered by massive gas injection and shattered pellet injection, runaway electron (RE) dynamics as well as the RE interaction with MHD modes, and vertical displacement events. Also the seeding and suppression of tearing modes (TMs), the dynamics of naturally occurring TQs triggered by locked modes, and radiative collapses are being studied.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗

Scalable hierarchical PDE sampler for generating spatially correlated random fields using nonmatching meshes: Scalable hierarchical PDE sampler using nonmatching meshes

This work describes a domain embedding technique between two nonmatching meshes used for generating realizations of spatially correlated random fields with applications to large-scale sampling-based uncertainty quantification. The goal is to apply the multilevel Monte Carlo (MLMC) method for the quantification of output uncertainties of PDEs with random input coefficients on general and unstructured computational domains. We propose a highly scalable, hierarchical sampling method to generate realizations of a Gaussian random field on a given unstructured mesh by solving a reaction–diffusion PDE with a stochastic right-hand side. The stochastic PDE is discretized using the mixed finite element method on an embedded domain with a structured mesh, and then, the solution is projected onto the unstructured mesh. This work describes implementation details on how to efficiently transfer data from the structured and unstructured meshes at coarse levels, assuming that this can be done efficiently on the finest level. We investigate the efficiency and parallel scalability of the technique for the scalable generation of Gaussian random fields in three dimensions. An application of the MLMC method is presented for quantifying uncertainties of subsurface flow problems. Here, we demonstrate the scalability of the sampling method with nonmatching mesh embedding, coupled with a parallel forward model problem solver, for large-scale 3D MLMC simulations with up to 1.9·109 unknowns.

97 MATHEMATICS AND COMPUTING↗

Grid-based minimization at scale: Feldman-Cousins corrections for SBN

We present a computational model for the construction of Feldman-Cousins (FC) corrections frequently used in High Energy Physics (HEP) analysis. The program contains a grid-based minimization and is written in C++. Our algorithms exploit vectorization through Eigen3, yielding a single-core speed-up of 350 compared to the original implementation, and achieve MPI data parallelism by using DIY. We demonstrate the application to scale very well at High Performance Computing (HPC) sites. We use HDF5 in conjunction with HighFive to write results of the calculation to file.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗

Forward and inverse cascades in decaying two-dimensional electron magnetohydrodynamic turbulence

Electron magnetohydrodynamic (EMHD) turbulence in two dimensions is studied via high-resolution numerical simulations with a normal diffusivity. The resulting energy spectra asymptotically approach a k{sup -5/2} law with increasing R{sub B}, the ratio of the nonlinear to linear time scales in the governing equation. No evidence is found of a dissipative cutoff, consistent with nonlocal spectral energy transfer. Dissipative cutoffs found in previous studies are explained as artificial effects of hyperdiffusivity. Relatively stationary structures are found to develop in time, rather than the variability found in ordinary or MHD turbulence. Further, EMHD turbulence displays scale-dependent anisotropy with reduced energy transfer in the direction parallel to the uniform background field, consistent with previous studies. Finally, the governing equation is found to yield an inverse cascade, at least partially transferring magnetic energy from small to large scales.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗

BurstMem: A high-performance burst buffer system for scientific applications

The growth of computing power on large-scale systems requires commensurate high-bandwidth I/O systems. Many parallel file systems are designed to provide fast sustainable I/O in response to applications' soaring requirements. To meet this need, a novel system is imperative to temporarily buffer the bursty I/O and gradually flush datasets to long-term parallel file systems. In this paper, we introduce the design of BurstMem, a high-performance burst buffer system. BurstMem provides a storage framework with efficient storage and communication management strategies. Our experiments demonstrate that BurstMem is able to speed up the I/O performance of scientific applications by up to 8.5× on leadership computer systems.

Wang, Teng↗

A performance model for the communication in fast multipole methods on high-performance computing platforms

Exascale systems are predicted to have approximately 1 billion cores, assuming gigahertz cores. Limitations on affordable network topologies for distributed memory systems of such massive scale bring new challenges to the currently dominant parallel programing model. Currently, there are many efforts to evaluate the hardware and software bottlenecks of exascale designs. It is therefore of interest to model application performance and to understand what changes need to be made to ensure extrapolated scalability. The fast multipole method (FMM) was originally developed for accelerating N-body problems in astrophysics and molecular dynamics but has recently been extended to a wider range of problems. Its high arithmetic intensity combined with its linear complexity and asynchronous communication patterns make it a promising algorithm for exascale systems. In this paper, we discuss the challenges for FMM on current parallel computers and future exascale architectures, with a focus on internode communication. We focus on the communication part only; the efficiency of the computational kernels are beyond the scope of the present study. We develop a performance model that considers the communication patterns of the FMM and observe a good match between our model and the actual communication time on four high-performance computing (HPC) systems, when latency, bandwidth, network topology, and multicore penalties are all taken into account. To our knowledge, this is the first formal characterization of internode communication in FMM that validates the model against actual measurements of communication time. The ultimate communication model is predictive in an absolute sense; however, on complex systems, this objective is often out of reach or of a difficulty out of proportion to its benefit when there exists a simpler model that is inexpensive and sufficient to guide coding decisions leading to improved scaling. The current model provides such guidance.

Computer Science↗

An O ( N ) and parallel approach to integral problems by a kernel-independent fast multipole method: Application to polarization and magnetization of interacting particles

Large classes of materials systems in physics and engineering are governed by magnetic and electrostatic interactions. Continuum or mesoscale descriptions of such systems can be cast in terms of integral equations, whose direct computational evaluation requires O ( N 2 ) operations, where N is the number of unknowns. Such a scaling, which arises from the many-body nature of the relevant Green's function, has precluded wide-spread adoption of integral methods for solution of large-scale scientific and engineering problems. In this work, a parallel computational approach is presented that relies on using scalable open source libraries and utilizes a kernel-independent Fast Multipole Method (FMM) to evaluate the integrals in O ( N ) operations, with O ( N ) memory cost, thereby substantially improving the scalability and efficiency of computational integral methods. We demonstrate the accuracy, efficiency, and scalability of our approach in the context of two examples. In the first, we solve a boundary value problem for a ferroelectric/ferromagnetic volume in free space. In the second, we solve an electrostatic problem involving polarizable dielectric bodies in an unbounded dielectric medium. Lastly, the results from these test cases show that our proposed parallel approach, which is built on a kernel-independent FMM, can enable highly efficient and accurate simulations and allow for considerable flexibility in a broad range of applications.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗

Progressive Hedging Decomposition for Solutions of Large-Scale Process Family Design Problems

In previous work, we have introduced a mathematical model for solving a discretized version of the process family design problem. This involves two sets of decision variables. One set selects which unit module designs are included in the process platform out of a candidate set of options; the other set determines which of these unit module designs are assigned to each variant. In this work, we exploit a parallelized Progressive Hedging (PH) algorithm to solve even larger scale design problems. PH is a well-known algorithm traditionally used to solve stochastic programming problems. While our problem is not a two-stage stochastic programming problem, the structure is similar, and it can be directly mapped to the PH approach, which we employ here to solve this deterministic optimization problem. We decompose our problem by process variant. We treat the platform unit module design variables as first-stage and the assignment of unit module designs to variants as second-stage, solving the problem using mpi-sppy. We demonstrate this approach on case studies of CC, water desalination, and refrigeration.

Stinchfield, Georgia↗

AutoSIM

AutoSIM enables several instances of an executable to run in parallel on a directory of input files and consolidate results, at scale on the world's fastest supercomputers. The software accomplishes this by balancing compression, transmission, and computation of input files, executables, and output files to allow scalable parallel runs of any executable. The software breaks-up simulations into a configurable number of "churns" or sequential instances of executable invocations on a configurable number of cores/node. It sends the executable, supporting files, and appropriate number of input files to each node for independent execution, and then transfers the executable results to a shared disk for post-processing analysis. This MPI-based code has been demonstrated using EnergyPlus, DOE's flagship whole-building simulation engine, but was architected to allow scalable runtime of any software for any application (e.g. sensitivity analysis and uncertainty quantification). The code has also been demonstrated to scale on the Titan supercomputer using 131,072 processors to run 524,288 EnergyPlus annual simulations, and write 45TB of data to disk in 68 mins 08 secs.

New, Joshua R↗

GVT Algorithms and Discrete Event Dynamics on 128K+ Processor Cores

Parallel discrete event simulation (PDES) represents a class of codes that are challenging to scale to large number of processors due to tight global timestamp-ordering and fine-grained event execution. One of the critical factors in scaling PDES is the efficiency of the underlying global virtual time (GVT) algorithm needed for correctness of parallel execution and speed of progress. Although many GVT algorithms have been proposed previously, few have been proposed for scalable asynchronous execution and none customized to exploit one-sided communication. Moreover, the detailed performance effects of actual GVT algorithm implementations on large platforms are unknown. Here, three major GVT algorithms intended for scalable execution on high-performance systems are studied: (1) a synchronous GVT algorithm that affords ease of implementation, (2) an asynchronous GVT algorithm that is more complex to implement but can relieve blocking latencies, and (3) a variant of the asynchronous GVT algorithm, proposed and studied for the first time here, to exploit one-sided communication in extant supercomputing platforms. Performance results are presented of implementations of these algorithms on over 64,000 cores of a Cray XT5 system, exercised on a range of parameters: optimistic and conservative synchronization, fine- to medium-grained event computation, synthetic and non-synthetic applications, and different lookahead values. Performance of tens of billions of events executed per second are registered, exceeding the speeds of any known PDES engine, and showing asynchronous GVT algorithms to outperform state-of-the-art synchronous GVT algorithms. Detailed PDES-specific runtime metrics are presented to further the understanding of tightly-coupled discrete event execution dynamics on massively parallel platforms.

99 GENERAL AND MISCELLANEOUS↗

Uranium and strontium fate in waste-weathered sediments: Scaling of molecular processes to predict reactive transport

Objectives of the project: 1. Determine the process coupling that occurs between mineral transformation and contaminant (U and Sr) speciation in acid-uranium waste weathered Hanford sediments. 2. Establish linkages between molecular-scale contaminant speciation and meso-scale contaminant lability, release and reactive transport. 3. Make conjunctive use of molecular- to bench-scale data to constrain the development of a mechanistic, reactive transport model that includes coupling of contaminant sorption-desorption and mineral transformation reactions. Hypotheses tested: - Uranium and strontium speciation in legacy sediments from the U-8 and U-12 Crib sites can be reproduced in bench-scale weathering experiments conducted on unimpacted Hanford sediments from the same formations. - Reactive transport modeling of future uranium and strontium releases from the vadose zone of acid-waste weathered sediments can be effectively constrained by combining molecular-scale information on contaminant bonding environment with grain-scale information on contaminant phase partitioning, and meso-scale kinetic data on contaminant release from the waste-weathered porous media. - Although field contamination and laboratory experiments differ in their diagenetic time scales (decades for field vs. months to years for lab), sediment dissolution, neophase nucleation, and crystal growth reactions that occur during the initial disequilibrium induced by waste-sediment interaction leave a strong imprint that persists over subsequent longer-term equilibration time scales and, therefore, give rise to long-term memory effects. Enabling capabilities developed: Our team developed an iterative measure-model approach that is broadly applicable to elucidate the mechanistic underpinnings of reactive contaminant transport in geomedia subject to active weathering. Experimental design: Hypotheses were tested by comparing (with a similar set of techniques) the geochemical transformations and transport behaviors that occured in bench-scale studies of waste-sediment interaction with parallel model systems studies of homogeneous nucleation and neo-phase dissolution. Initial plans were to compare results with core sample extractions from the acid uranium waste impacted U-8 and U-12 Cribs at Hanford (see original proposal and letter of collaboration from J. Zachara). However, this part of the project was impossible because funding for core extractions were eliminated from the DoE budget. Three distinct crib waste aqueous simulants (whose composition is based on the most up-to-date information from field site investigations) were reacted with Hanford sediments in batch and column systems. Coupling of contaminant uptake to mineral weathering was monitored using a suite of methods both during waste-sediment interaction, and after, when waste-weathered sediments were subjected to infusion with circumneutral background pore water solutions. Our research was designed to adapt as needed to maintain a strong dialogue between laboratory and modeling investigations so that model development was increasingly constrained by emergent data and understanding. Potential impact of the project to DOE: Better prediction of contaminant uranium transport was achieved by employing multi-faceted lines of inquiry to build a strong bridge between molecular- and field-scale information. By focusing multiple lines and scales of observation on a common experimental design, our collaborative team revealed non-linear and emergent behavior in contaminated weathering systems. A goal of the current project was to expand our modeling capabilities, originally focused on hyperalkaline legacy waste streams, to include acidic weathering reactions that, as described above, were expected to result in profoundly different products. We were able to achieve this goal, and showed that these products nonetheless undergo analogous silicate and non-silicate transformation, ripening and aging processes. Our prediction that these weathering reactions would vary with waste stimulant chemistry resulted in data that was incorporated directly into a reactive transport model structure.

54 ENVIRONMENTAL SCIENCES↗

Contingency Analysis Based on Partitioned and Parallel Holomorphic Embedding

In the steady-state contingency analysis, the traditional Newton-Raphson method suffers from non-convergence issues when solving post-outage power flow problems, which hinders the integrity and accuracy of security assessment. In this paper, we propose a novel robust contingency analysis approach based on holomorphic embedding (HE). Here, the HE-based simulator provides theoretical convergence guarantee, which is desirable because it avoids the influence of numerical issues and provides a credible security assessment conclusion. In addition, based on the multi-area characteristics of real-world power systems, a partitioned HE (PHE) method is proposed with an interfacebased partitioning of HE formulation. The PHE method does not undermine the numerical robustness of HE and significantly reduces the computation burden in large-scale contingency analysis. The PHE method is further enhanced by parallel or distributed computation to become parallel PHE (P2HE). Tests on a 458-bus system, a synthetic 419-bus system and a large-scale 21447-bus system demonstrate the advantages of the proposed methods in robustness and efficiency.

42 ENGINEERING↗

Quantum Package 2.0: An Open-Source Determinant-Driven Suite of Programs

Quantum chemistry is a discipline which relies heavily on very expensive numerical computations. The scaling of correlated wave function methods lies, in their standard implementation, between O(N-5) and O(e(N)), where N is proportional to the system size. Therefore, performing accurate calculations on chemically meaningful systems requires (i) approximations that can lower the computational scaling and (ii) efficient implementations that take advantage of modern massively parallel architectures. QUANTUM PACKAGE is an open-source programming environment for quantum chemistry specially designed for wave function methods. Its main goal is the development of determinant-driven selected configuration interaction (sCI) methods and multireference second-order perturbation theory (PT2). The determinant-driven framework allows the programmer to include any arbitrary set of determinants in the reference space, hence providing greater methodological freedom. The sCI method implemented in QUANTUM PACKAGE is based on the CIPSI (Configuration Interaction using a Perturbative Selection made Iteratively) algorithm which complements the variational sCI energy with a PT2 correction. Additional external plugins have been recently added to perform calculations with multireference coupled cluster theory and range-separated density-functional theory. All the programs are developed with the IRPF90 code generator, which simplifies collaborative work and the development of new features. QUANTUM PACKAGE strives to allow easy implementation and experimentation of new methods, while making parallel computation as simple and efficient as possible on modern supercomputer architectures. Currently, the code enables, routinely, to realize runs on roughly 2 000 CPU cores, with tens of millions of determinants in the reference space. Moreover, we have been able to push up to 12 288 cores in order to test its parallel efficiency. In the present manuscript, we also introduce some key new developments: (i) a renormalized second-order perturbative correction for efficient extrapolation to the full CI limit and (ii) a stochastic version of the CIPSI selection performed simultaneously to the PT2 calculation at no extra cost.

Garniron, Yann↗

Manufacturing Supply Chain Development for Modular Solar-Thermochemical Conversion Platform - CRADA 387 (Final Report)

Modular chemical process intensification (MCPI) is an emerging field where chemical processing is performed using small-scale modular equipment instead of conventional large centralized chemical plants. Conventional chemical plants benefit from economies of scale that encourage scale-up to ever larger plants. A goal of MCPI is to develop technology that intensifies processing so that equipment can be dramatically smaller and integrated into modular systems. Scale-up occurs by adding more modules in parallel rather than making the equipment larger. A key concept is that equipment and modules can ultimately be cheaper by leveraging economies of mass production, analogous to the automotive industry, in manufacturing the equipment. This project made significant progress toward this outcome by meeting the RAPID institute metric to reduce equipment cost by 20% for each doubling in manufacturing volume. The MCPI application was thermochemical technology that is being commercialized by STARS Technology Corporation, one of the CRADA partners. The technology converts solar and renewable power to chemical energy to produce renewable hydrogen, fuels, and chemicals. The benefit to the public is reduction in greenhouse gases that are contributing to climate change. The project transitioned the steam methane reforming (SMR) reactor from conventional fabrication methods to additive manufacturing (AM) direct metal laser sintering (DMLS) process. This is projected to reduce the cost of making a reactor by 58% when producing 100 reactors per year. Innovations in the DMLS process produced a patented design that reduces reactor weight by 60%. Reductions in material costs and processing time extend the DMLS advantage to higher production volumes. The new design promises to be 38% cheaper than the conventional processes at 1000 units per year. The resulting 87% reduction in the steam methane reforming (SMR) module cost in scaling from current costs meets the RAPID metric. The project was successful in producing and testing the first ever additively manufactured SMR reactors. A reactor achieved over 82% efficiency in converting electric power to chemical energy, which is a world record for an inductively heated SMR. The project has contributed to the design and assembly of a first demonstration plant that is headed to a hydrogen bus filling station in Thousand Palms, CA.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

Multi-operator Production Cost Modeling

As contiguous power systems are often operated by several regional system operators, optimization models have traditionally been designed to generate optimal system-wide scheduling decisions. This report introduces an approach to reflect the existence of multiple system operators making scheduling decisions within an interconnected power system. The approach is explicitly designed to reflect each operator's limited knowledge about other regions when making scheduling decisions. Detailed analysis using a modified IEEE Reliability Test System reveals that the approach mimics real system operations by providing imperfect information to each operator about external operations and external constraints. The approach scales to a 78,000-bus model of the Eastern Interconnection, where a regression analysis provides quantitative insight into which constraints cause inefficiencies when improperly managed in a multi-operator simulation. Because the approach generates distinct regional scheduling problems, unit commitment and economic dispatch simulations can be applied to individual regions and solved in parallel, enabling production cost simulations at a previously intractable scale.

24 POWER TRANSMISSION AND DISTRIBUTION↗

Distributed Halide

Many image processing tasks are naturally expressed as a pipeline of small computational kernels known as stencils. Halide is a popular domain-specific language and compiler designed to implement image processing algorithms. Halide uses simple language constructs to express what to compute and a separate scheduling co-language for expressing when and where to perform the computation. This approach has demonstrated performance comparable to or better than hand-optimized code. Until now, however, Halide has been restricted to parallel shared memory execution, limiting its performance for memory-bandwidth-bound pipelines or large-scale image processing tasks. We present an extension to Halide to support distributed-memory parallel execution of complex stencil pipelines. These extensions compose with the existing scheduling constructs in Halide, allowing expression of complex computation and communication strategies. Existing Halide applications can be distributed with minimal changes, allowing programmers to explore the tradeoff between recomputation and communication with little effort. Approximately 10 new of lines code are needed even for a 200 line, 99 stage application. On nine image processing benchmarks, our extensions give up to a 1.4× speedup on a single node over regular multithreaded execution with the same number of cores, by mitigating the effects of non-uniform memory access. The distributed benchmarks achieve up to 18× speedup on a 16 node testing machine and up to 57× speedup on 64 nodes of the NERSC Cori supercomputer.

97 MATHEMATICS AND COMPUTING↗

Gyrokinetic simulation of global and local Alfvén eigenmodes driven by energetic particles in a DIII-D discharge

The unstable spectrum of Alfvén eigenmodes (AEs) driven by neutral beam-sourced energetic particles (EPs) in a benchmark DIII-D discharge (142111) is calculated in a fully gyrokinetic model using the GYRO code's massively parallel linear eigenvalue solver. One cycle of the slow (equilibrium scale) frequency sweep of the reverse shear Alfvén eigenmode (RSAE) at toroidal mode number n=3 is mapped. The RSAE second harmonic and an unstable beta-induced Alfvén eigenmode (BAE) are simultaneously tracked alongside the primary RSAE. An observed twist in the eigenmode pattern, caused mostly by shear in the driving EP profile, is shown through artificially varying the E×B rotational velocity shear to depend generally on shear in the local wave phase velocity. Coupling to the BAE and to the toroidal Alfvén eigenmode limit the RSAE frequency sweeps at the lower and upper end, respectively. While the present fully gyrokinetic model (including thermal ions and electrons) constitutes the best treatment of compressibility physics available, the BAE frequency is overpredicted by about 20% against experiment here and is found to be sensitive to energetic beam ion pressure. The RSAE frequency is more accurately matched except when it is limited by the BAE. Simulations suggest that the experiment is very close to marginal AE stability at points of RSAE-BAE coupling. A recipe for comparing the radial profile of quasilinear transport flux from local modes to that from global modes paves the way for the development of a stiff (critical gradient) local AE transport model based on local mode stability thresholds.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗