Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “parallel, scaling”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,027 records · Page 57

The Inner Workings of Magnetic Reconnection: Diffusion Region in the Balance

The question of whether the micro scale controls the macroscale or vice-versa remains one of the most challenging problems in plasmas. A particular topic of interest within this context is collisionless magnetic reconnection, where both points of views are espoused by different groups of researchers. This presentation will focus on this topic. We will begin by analyzing the properties of electron diffusion region dynamics both for guide field and anti-parallel reconnection, and how they can be scaled to different inflow conditions. As a next step, we will study typical temporal variations of the microscopic dynamics with the objective of understanding the potential for secular changes to the macroscopic system. The research will be based on a combination of analytical theory and numerical modeling.

Hesse, Michael↗

PARVMEC: An Efficient, Scalable Implementation of the Variational Moments Equilibrium Code

The ability to sustain magnetically confined plasma in a state of stable equilibrium is crucial for optimal and cost-effective operations of fusion devices like tokamaks and stellarators. The Variational Moments Equilibrium Code (VMEC) is the de-facto serial application used by fusion scientists to compute magnetohydrodynamics (MHD) equilibria and study the physics of three dimensional plasmas in confined configurations. Modern fusion energy experiments have larger system scales with more interactive experimental workflows, both demanding faster analysis turnaround times on computational workloads that are stressing the capabilities of sequential VMEC. In this paper, we present PARVMEC, an efficient, parallel version of its sequential counterpart, capable of scaling to thousands of processors on distributed memory machines. PARVMEC is a non-linear code, with multiple numerical physics modules, each with its own computational complexity. A detailed speedup analysis supported by scaling results on 1,024 cores of a Cray XC30 supercomputer is presented. Depending on the mode of PARVMEC execution, speedup improvements of one to two orders of magnitude are reported. PARVMEC equips fusion scientists for the first time with a state-of-theart capability for rapid, high fidelity analyses of magnetically confined plasmas at unprecedented scales.

Seal, Sudip K↗

Massively parallel phase-field simulations targeting exascale

The interface thickness in the phase-field (PF) method limits its simulation scales. Consequently, large-scale PF simulations become prohibitively expensive for resolving the extremely fine microstructures that typically form during rapid solidification processing. This challenge is significant in predicting microstructure evolution in metal additive manufacturing and has been identified by the United States Department of Energy’s Exascale Computing Project. Here, to address this, we develop a multi-GPU and MPI-based massively parallel simulation code, utilizing state-of-the-art algorithms, software, and libraries, for large-scale three-dimensional (3D) PF simulations. We report the first GPU-parallel PF simulations on Frontier (currently the second TOP500 exascale cluster) and Summit machines, taking dendritic growth as an example problem. We evaluate the parallel performance of our implementation using scaling studies with more than 24 000 GPUs (among the largest known computations to date) and the acceleration performance using large-scale simulations of dendritic growth in 3D. Finally, massively parallel GPUs in these supercomputers enabled the first coupled multiscale simulations of laser melting and subsequent dendritic solidification on the scale of a full melt-pool, demonstrating the feasibility of performing PF simulations with a point total over 2 billion grid points within an acceptable time.

Exascale↗

Space Technology 5 Multi-Point Observations of Temporal Variability of Field-Aligned Currents

Space Technology 5 (ST5) is a three micro-satellite constellation deployed into a 300 x 4500 km, dawn-dusk, sun-synchronous polar orbit from March 22 to June 21, 2006, for technology validations. In this paper, we present a study of the temporal variability of field-aligned currents using multi-point magnetic field measurements from ST5. The data demonstrate that meso-scale current structures are commonly embedded within large-scale field-aligned current sheets. The meso-scale current structures are very dynamic with highly variable current density and/or polarity in time scales of approximately 10 min. They exhibit large temporal variations during both quiet and disturbed times in such time scales. On the other hand, the data also shown that the time scales for the currents to be relatively stable are approximately 1 min for meso-scale currents and approximately 10 min for large scale current sheets. These temporal features are obviously associated with dynamic variations of their particle carriers (mainly electrons) as they respond to the variations of the parallel electric field in auroral acceleration region. The characteristic time scales for the temporal variability of meso-scale field-aligned currents are found to be consistent with those of auroral parallel electric field.

Le, Guan↗

Space Technology 5 Multi-point Observations of Field-aligned Currents: Temporal Variability of Meso-Scale Structures

Space Technology 5 (ST5) is a three micro-satellite constellation deployed into a 300 x 4500 km, dawn-dusk, sun-synchronous polar orbit from March 22 to June 21, 2006, for technology validations. In this paper, we present a study of the temporal variability of field-aligned currents using multi-point magnetic field measurements from ST5. The data demonstrate that meso-scale current structures are commonly embedded within large-scale field-aligned current sheets. The meso-scale current structures are very dynamic with highly variable current density and/or polarity in time scales of - 10 min. They exhibit large temporal variations during both quiet and disturbed times in such time scales. On the other hand, the data also shown that the time scales for the currents to be relatively stable are approx. 1 min for meso-scale currents and approx. 10 min for large scale current sheets. These temporal features are obviously associated with dynamic variations of their particle carriers (mainly electrons) as they respond to the variations of the parallel electric field in auroral acceleration region. The characteristic time scales for the temporal variability of meso-scale field-aligned currents are found to be consistent with those of auroral parallel electric field.

Le, Guan↗

Space Technology 5 (ST-5) Multipoint Observations of Temporal and Spatial Variability of Field-Aligned Currents

Space Technology 5 (ST-5) is a three micro-satellite constellation deployed into a 300 x 4500 km, dawn-dusk, sun-synchronous polar orbit from March 22 to June 21, 2006, for technology validations. In this paper, we present a study of the temporal variability of field-aligned currents using multi-point magnetic field measurements from ST5. The data demonstrate that mesoscale current structures are commonly embedded within large-scale field-aligned current sheets. The meso-scale current structures are very dynamic with highly variable current density and/or polarity in time scales of about 10 min. They exhibit large temporal variations during both quiet and disturbed times in such time scales. On the other hand, the data also shown that the time scales for the currents to be relatively stable are about 1 min for meso-scale currents and about 10 min for large scale current sheets. These temporal features are obviously associated with dynamic variations of their particle carriers (mainly electrons) as they respond to the variations of the parallel electric field in auroral acceleration region. The characteristic time scales for the temporal variability of meso-scale field-aligned currents are found to be consistent with those of auroral parallel electric field.

Le, Guan↗

Space Technology 5 (ST-5) Observations of Field-Aligned Currents: Temporal Variability

Space Technology 5 (ST-5) is a three micro-satellite constellation deployed into a 300 x 4500 km, dawn-dusk, sun-synchronous polar orbit from March 22 to June 21, 2006, for technology validations. In this paper, we present a study of the temporal variability of field-aligned currents using multi-point magnetic field measurements from STS. The data demonstrate that masoscale current structures are commonly embedded within large-scale field-aligned current sheets. The meso-scale current structures are very dynamic with highly variable current density and/or polarity in time scales of about 10 min. They exhibit large temporal variations during both quiet and disturbed times in such time scales. On the other hand, the data also shown that the time scales for the currents to be relatively stable are about I min for meso-scale currents and about 10 min for large scale current sheets. These temporal features are obviously associated with dynamic variations of their particle carriers (mainly electrons) as they respond to the variations of the parallel electric field in auroral acceleration region. The characteristic time scales for the temporal variability of meso-scale field-aligned currents are found to be consistent with those of auroral parallel electric field.

Le, Guan↗

Quantum and Classical Dynamics Simulations of ATP Hydrolysis in Solution

ATP hydrolysis is a key reaction in living cells that drives many cellular processes. The reaction, which involves gamma phosphate cleavage from ATP, converting it to ADP, has been suggested to occur via an associative or dissociative mechanism dependent upon the surrounding environment. Prior quantum chemical studies suffered from short simulation time scales failing to capture free energy contributions due to relaxation of the surrounding aqueous environment. We have developed a highly parallelized QM/MM implementation in the NAMD and OpenAtom simulation packages, using the dual grid, dual length scale method for combined plane-wave and Euler exponential spline-based QM/MM simulations. This approach, using message-driven parallel quantum and classical dynamics, permits sufficient time scale simulations for quantum chemical events such as ATP hydrolysis and is found to accurately and reliably include the free energy contributions of solvent relaxation to hydrolysis. In this paper, we describe the application of the dual grid, dual length plane-wave-based QM/MM method to study both the associative and dissociative mechanisms of ATP hydrolysis, accounting for the free energy contribution from solvent relaxation, as well as for the key role of Mg2+ in the reaction.

Chemistry↗

A family of independent Variable Eddington Factor methods with efficient preconditioned iterative solvers

We present a family of discretizations for the Variable Eddington Factor (VEF) equations that have high-order accuracy on curved meshes and efficient preconditioned iterative solvers. The VEF discretizations are combined with the Discontinuous Galerkin transport discretization from to form effective high-order, linear transport methods. The VEF discretizations are derived by extending the unified analysis of Discontinuous Galerkin methods for elliptic problems presented by Arnold et al. to the VEF equations. This framework is used to define analogs of the interior penalty, second method of Bassi and Rebay, minimal dissipation local Discontinuous Galerkin, and continuous finite element methods. The analysis of subspace correction preconditioners, which use a continuous operator to iteratively precondition the discontinuous discretization, is extended to the case of the non-symmetric VEF system. Numerical results demonstrate that the VEF discretizations have arbitrary-order accuracy on curved meshes, preserve the thick diffusion limit, and are effective on a proxy problem from thermal radiative transfer in both outer transport iterations and inner preconditioned linear solver iterations. We demonstrate that the VEF solution converges to the S N transport solution as the mesh is refined on both problems with smooth and non-smooth behavior in angle. Parallel performance studies show that the interior penalty VEF discretization's linear solve weak scales out to 1024 processors and strong scales well on a single node. Particular attention is paid to the parallel performance of the VEF algorithm when used in combination with a parallel block Jacobi transport sweep.

97 MATHEMATICS AND COMPUTING↗

ATMOSPHERIC DYNAMICS OF BROWN DWARFS AND DIRECTLY IMAGED GIANT PLANETS

A variety of observations provide evidence for vigorous motion in the atmospheres of brown dwarfs and directly imaged giant planets. Motivated by these observations, we examine the dynamical regime of the circulation in the atmospheres and interiors of these objects. Brown dwarfs rotate rapidly, and for plausible wind speeds, the flow at large scales will be rotationally dominated. We present three-dimensional, global, numerical simulations of convection in the interior, which demonstrate that at large scales, the convection aligns in the direction parallel to the rotation axis. Convection occurs more efficiently at high latitudes than low latitudes, leading to systematic equator-to-pole temperature differences that may reach ∼1 K near the top of the convection zone. The interaction of convection with the overlying, stably stratified atmosphere will generate a wealth of atmospheric waves, and we argue that, as in the stratospheres of planets in the solar system, the interaction of these waves with the mean flow will cause a significant atmospheric circulation at regional to global scales. At large scales, this should consist of stratified turbulence (possibly organizing into coherent structures such as vortices and jets) and an accompanying overturning circulation. We present an approximate analytic theory of this circulation, which predicts characteristic horizontal temperature variations of several to ∼50 K, horizontal wind speeds of ∼10-300 m s{sup –1}, and vertical velocities that advect air over a scale height in ∼10{sup 5}-10{sup 6} s. This vertical mixing may help to explain the chemical disequilibrium observed on some brown dwarfs. Moreover, the implied large-scale organization of temperature perturbations and vertical velocities suggests that near the L/T transition, patchy clouds can form near the photosphere, helping to explain recent observations of brown-dwarf variability in the near-IR.

79 ASTRONOMY AND ASTROPHYSICS↗

Visualizing Hierarchical Performance Profiles of Parallel Codes Using CallFlow

Calling context trees (CCTs) couple performance metrics with call paths, helping understand the execution and performance of parallel programs. To identify performance bottlenecks, programmers and performance analysts visually explore CCTs to form and validate hypotheses regarding degraded performance. However, due to the complexity of parallel programs, existing visual representations do not scale to applications running on a large number of processors. In this work, we present CallFlow, an interactive visual analysis tool that provides a high-level overview of CCTs together with semantic refinement operations to progressively explore CCTs. Using a flow-based metaphor, we visualize a CCT by treating execution time as a resource spent during the call chain, and demonstrate the effectiveness of our design with case studies on large-scale, production simulation codes.

97 MATHEMATICS AND COMPUTING↗

Optimizing I/O Forwarding Techniques for Extreme-Scale Event Tracing

Programming development tools are a vital com- ponent for understanding the behavior of parallel applica- tions. Event tracing is a principal ingredient to these tools, but new and serious challenges place event tracing at risk on extreme-scale machines. As the quantity of captured events increases with concurrency, the additional data can over- load the parallel file system and perturb the application be- ing observed. In this work we present a solution for event tracing on extreme-scale machines. We enhance an I/O for- warding software layer to aggregate and reorganize log data prior to writing to the storage system, significantly reduc- ing the burden on the underlying file system. Furthermore, we introduce a sophisticated write buffering capability to limit the impact. To validate the approach, we employ the Vampir tracing toolset using these new capabilities. Our re- sults demonstrate that the approach increases the maximum traced application size by a factor of 5x to more than 200,000 processes.

atomic append↗

Investigation of the effect of resistivity on scrape off layer filaments using three-dimensional simulations

The propagation of filaments in the Scrape Off Layer (SOL) of tokamaks largely determines the plasma profiles in the region. In a conduction limited SOL, parallel temperature gradients are expected, such that the resistance to parallel currents is greater at the target than further upstream. Since the perpendicular motion of an isolated filament is largely determined by balance of currents that flow through it, this may be expected to affect filament transport. 3D simulations have thus been used to study the influence of enhanced parallel resistivity on the dynamics of filaments. Filaments with the smallest perpendicular length scales, which were inertially limited at low resistivity (meaning that polarization rather than parallel currents determines their radial velocities), were unaffected by resistivity. For larger filaments, faster velocities were produced at higher resistivities due to two mechanisms. First parallel currents were reduced and polarization currents were enhanced, meaning that the inertial regime extended to larger filaments, and second, a potential difference formed along the parallel direction so that higher potentials were produced in the region of the filament for the same amount of current to flow into the sheath. These results indicate that broader SOL profiles could be produced at higher resistivities.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗

Dispersion of Radionuclides and Exposure Assessment in Urban Environments: A Joint CEA and LLNL Report

In the interest of promoting the international exchange of technical expertise, the US Department of Energy’s Office of Emergency Operations (NA-40) and the French Commissariat à l'Energie Atomique et aux énergies alternatives (CEA) requested that the National Atmospheric Release Advisory Center (NARAC) of Lawrence Livermore National Laboratory (LLNL) in Livermore, California host a joint table top exercise with experts in emergency management and atmospheric transport modeling. In this table top exercise, LLNL and CEA compared each other’s flow and dispersion models. The goal of the comparison is to facilitate the exchange of knowledge, capabilities, and practices, and to demonstrate the utility of modeling dispersal at different levels of computational fidelity. Two modeling approaches were examined, a regional scale modeling approach, appropriate for simple terrain and/or very large releases, and an urban scale modeling approach, appropriate for small releases in a city environment. This report is a summary of LLNL and CEA modeling efforts from this exercise. Two different types of LLNL and CEA models were employed in the analysis: urban-scale models (Aeolus CFD at LLNL/NARAC and Parallel- Micro-SWIFT-SPRAY, PMSS, at CEA) for analysis of a 5,000 Ci radiological release and Lagrangian Particle Dispersion Models (LODI at LLNL/NARAC and PSPRAY at CEA) for analysis of a much larger (500,000 Ci) regional radiological release. Two densely-populated urban locations were chosen: Chicago with its high-rise skyline and gridded street network and Paris with its more consistent, lower building height and complex unaligned street network. Each location was considered under early summer daytime and nighttime conditions. Different levels of fidelity were chosen for each scale: (1) lower fidelity mass-consistent diagnostic, intermediate fidelity Navier-Stokes RANS models, and higher fidelity Navier-Stokes LES for urban-scale analysis, and (2) lower-fidelity single-profile meteorology versus higher-fidelity three-dimensional gridded weather forecast for regional-scale analysis. Tradeoffs between computation time and the fidelity of the results are discussed for both scales. LES, for example, requires nearly 100 times more processor time than the mass-consistent diagnostic model or the RANS model, and seems better able to capture flow entrainment behind tall buildings. As anticipated, results obtained by LLNL and CEA at regional scale around Chicago and Paris look very similar in terms of both atmospheric dispersion of the radiological release and total effective dose. Both LLNL and CEA used the same meteorological data, Lagrangian particle dispersion models, and the same dose coefficients. LLNL and CEA urban-scale modeling results show consistent phenomenological behavior and predict similar impacted areas even though the detailed 3D flow patterns differ, particularly for the Chicago cases where differences in vertical entrainment behind tall buildings are particularly notable. Although RANS and LES (LLNL) models incorporate more detailed physics than do mass-consistent diagnostic flow models (CEA), it is not possible to reach definite conclusions about the prediction fidelity of the various models as experimental measurements were not available for comparison. Stronger conclusions about the relative performances of the models involved and evaluation of the tradeoffs involved in model simplification could be made with a systematic benchmarking of urban-scale modeling. This could be the purpose of a future US / French collaborative exercise.

54 ENVIRONMENTAL SCIENCES↗

Characteristics of Turbulence-driven Plasma Flow and Origin of Experimental Empirical Scalings of Intrinsic Rotation

Toroidal plasma flow driven by turbulent torque associated with nonlinear residual stress generation is shown to recover the observed key features of intrinsic rotation in experiments. Specifically, the turbulence-driven intrinsic rotation scales close to linearly with plasma gradients and the inverse of the plasma current, qualitatively reproducing empirical scalings obtained from a large experimental data base. The effect of magnetic shear on the symmetry breaking in the parallel wavenumber spectrum is identified. The origin of the current scaling is found to be the enhanced kll symmetry breaking induced by increased radial variation of the safety factor as the current decreases. The physics origin for the linear dependence of intrinsic rotation on the pressure gradient comes from the fact that both turbulence intensity and the zonal flow shear, which are two key ingredients for driving the residual stress, are increased with the strength of the turbulence drives, which are R/LTe and R/Lne for the collisionless trapped electron mode (CTEM). Highlighted results also include robust radial pinches in toroidal flow, heat and particle transport driven by CTEM turbulence, which emerge "in phase", and are shown to play important roles in determining plasma profiles. Also discussed are experimental tests proposed to validate findings from these gyrokinetic simulations.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗

Current parallel I/O limitations to scalable data analysis.

This report describes the limitations to parallel scalability which we have encountered when applying our otherwise optimally scalable parallel statistical analysis tool kit to large data sets distributed across the parallel file system of the current premier DOE computational facility. This report describes our study to evaluate the effect of parallel I/O on the overall scalability of a parallel data analysis pipeline using our scalable parallel statistics tool kit [PTBM11]. In this goal, we tested it using the Jaguar-pf DOE/ORNL peta-scale platform on a large combustion simulation data under a variety of process counts and domain decompositions scenarios. In this report we have recalled the foundations of the parallel statistical analysis tool kit which we have designed and implemented, with the specific double intent of reproducing typical data analysis workflows, and achieving optimal design for scalable parallel implementations. We have briefly reviewed those earlier results and publications which allow us to conclude that we have achieved both goals. However, in this report we have further established that, when used in conjuction with a state-of-the-art parallel I/O system, as can be found on the premier DOE peta-scale platform, the scaling properties of the overall analysis pipeline comprising parallel data access routines degrade rapidly. This finding is problematic and must be addressed if peta-scale data analysis is to be made scalable, or even possible. In order to attempt to address these parallel I/O limitations, we will investigate the use the Adaptable IO System (ADIOS) [LZL+10] to improve I/O performance, while maintaining flexibility for a variety of IO options, such MPI IO, POSIX IO. This system is developed at ORNL and other collaborating institutions, and is being tested extensively on Jaguar-pf. Simulation code being developed on these systems will also use ADIOS to output the data thereby making it easier for other systems, such as ours, to process that data.

99 GENERAL AND MISCELLANEOUS↗

Multiply accelerated ReaxFF molecular dynamics: coupling parallel replica dynamics with collective variable hyper dynamics

To address the time scales required to study complex chemical reactions, methods performing accelerated molecular dynamics are necessary even with the recent advancement in high-performance computing. A number of different acceleration techniques are available. Here we explore potential synergies between two popular acceleration methods – Parallel Replica Dynamics (PRD) and Collective Variable Hyperdynamics (CVHD), by analysing the speedup obtained for the pyrolysis of n-dodecane. We determine that PRD + CVHD provides additional speedup to CVHD by reaching the required time scales for the reaction at an earlier wall-clock time. Although some speedup is obtained with the additional replicas, we found that the effectiveness of the inclusion of PRD is depreciated for systems where there is a dramatic increase in reaction rates induced by CVHD. Similar observations were made in the simulation of ethylene-carbonate/Li system, which is inherently more reactive than pyrolysis, suggest that the speedup obtained via the combination of the two acceleration methods can be generalised to most practical chemical systems.

42 ENGINEERING↗

Iterative methods for large scale static analysis of structures on a scalable multiprocessor supercomputer

A parallel Preconditioned Conjugate Gradient (PCG) iterative solver has been developed and implemented on the iPSC-860 scalable hypercube. This new implementation makes use of the Parallel Automated Runtime Toolkit at ICASE (PARTI) primitives to efficiently program irregular communications patterns that exist in general sparse matrices and in particular in the finite element sparse stiffness matrices. The iterative PCG has been used to solve the finite element equations that result from discretizing large scale aerospace structures. In particular, the static response of the High Speed Civil Transport (HSCT) finite element model is solved on the iPSC-860.

Sobh, Nahil Atef↗