Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “parallel, scaling”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 235 records · Page 13

DIRECT NUMERICAL SIMULATIONS OF REFLECTION-DRIVEN, REDUCED MAGNETOHYDRODYNAMIC TURBULENCE FROM THE SUN TO THE ALFVÉN CRITICAL POINT

We present direct numerical simulations of inhomogeneous reduced magnetohydrodynamic (RMHD) turbulence between the Sun and the Alfvén critical point. These are the first such simulations that take into account the solar-wind outflow velocity and the radial inhomogeneity of the background solar wind without approximating the nonlinear terms in the governing equations. RMHD turbulence is driven by outward-propagating Alfvén waves (z {sup +} fluctuations) launched from the Sun, which undergo partial non-WKB reflection to produce sunward-propagating Alfvén waves (z {sup –} fluctuations). We present 10 simulations with different values of the correlation time τ{sub c{sub sun}{sup +}} and perpendicular correlation length L{sub ☉} of outward-propagating Alfvén waves at the coronal base. We find that between 15% and 33% of the z {sup +} energy launched into the corona dissipates between the coronal base and Alfvén critical point. Between 33% and 40% of this input energy goes into work on the solar-wind outflow, and between 22% and 36% escapes as z {sup +} fluctuations through the simulation boundary at r = r{sub A}. The z {sup ±} power spectra scale like k{sub perpendicular}{sup -α{sup ±}}, where k is the wavenumber in the plane perpendicular to B{sub 0}. In our simulation with the smallest value of τ{sub c{sub sun}{sup +}} (∼2 minutes) and largest value of L{sub ☉} (2 × 10{sup 4} km), we find that α{sup +} decreases approximately linearly with increasing ln (r), reaching a value of 1.3 at r = 11.1 R{sub ☉}. Our simulations with larger values of τ{sub c{sub sun}{sup +}} exhibit alignment between the contours of constant φ{sup +}, φ{sup –}, Ω{sub 0}{sup +}, and Ω{sub 0}{sup -}, where φ{sup ±} are the Elsässer potentials and Ω{sub 0}{sup ±} are the outer-scale parallel Elsässer vorticities.

79 ASTRONOMY AND ASTROPHYSICS↗

MPI-AMRVAC FOR SOLAR AND ASTROPHYSICS

In this paper, we present an update to the open source MPI-AMRVAC simulation toolkit where we focus on solar and non-relativistic astrophysical magnetofluid dynamics. We highlight recent developments in terms of physics modules, such as hydrodynamics with dust coupling and the conservative implementation of Hall magnetohydrodynamics. A simple conservative high-order finite difference scheme that works in combination with all available physics modules is introduced and demonstrated with the example of monotonicity-preserving fifth-order reconstruction. Strong stability-preserving high-order Runge-Kutta time steppers are used to obtain stable evolutions in multi-dimensional applications, realizing up to fourth-order accuracy in space and time. With the new distinction between active and passive grid cells, MPI-AMRVAC is ideally suited to simulate evolutions where parts of the solution are controlled analytically or have a tendency to progress into or out of a stationary state. Typical test problems and representative applications are discussed with an outlook toward follow-up research. Finally, we discuss the parallel scaling of the code and demonstrate excellent weak scaling up to 30, 000 processors, allowing us to exploit modern peta-scale infrastructure.

79 ASTRONOMY AND ASTROPHYSICS↗

Gravity, two times, tractors, Weyl invariance, and six-dimensional quantum mechanics

Fefferman and Graham showed some time ago that four-dimensional conformal geometries could be analyzed in terms of six-dimensional, ambient, Riemannian geometries admitting a closed homothety. Recently, it was shown how conformal geometry provides a description of physics manifestly invariant under local choices of unit systems. Strikingly, Einstein's equations are then equivalent to the existence of a parallel scale tractor (a six-component vector subject to a certain first order covariant constancy condition at every point in four-dimensional spacetime). These results suggest a six-dimensional description of four-dimensional physics, a viewpoint promulgated by the 2 times physics program of Bars. The Fefferman-Graham construction relies on a triplet of operators corresponding, respectively, to a curved six-dimensional light cone, the dilation generator and the Laplacian. These form an sp(2) algebra which Bars employs as a first class algebra of constraints in a six-dimensional gauge theory. In this article four-dimensional gravity is recast in terms of six-dimensional quantum mechanics by melding the 2 times and tractor approaches. This parent formulation of gravity is built from an infinite set of six-dimensional fields. Successively integrating out these fields yields various novel descriptions of gravity including a new four-dimensional one built from a scalar doublet, a tractor-vector multiplet and a conformal class of metrics.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

Accurate and Scalable O(N) Algorithm for First-Principles Molecular-Dynamics Computations on Large Parallel Computers

We present the first truly scalable first-principles molecular dynamics algorithm with O(N) complexity and controllable accuracy, capable of simulating systems with finite band gaps of sizes that were previously impossible with this degree of accuracy. By avoiding global communications, we provide a practical computational scheme capable of extreme scalability. Accuracy is controlled by the mesh spacing of the finite difference discretization, the size of the localization regions in which the electronic wave functions are confined, and a cutoff beyond which the components of the overlap matrix can be omitted when computing selected elements of its inverse. We demonstrate the algorithm's excellent parallel scaling for up to 101 952 atoms on 23 328 processors, with a wall-clock time of the order of 1 min per molecular dynamics time step and numerical error on the forces of less than 7x10 -4 Ha/Bohr.

36 MATERIALS SCIENCE↗

Communication-Avoiding Parallel Sparse-Dense Matrix-Matrix Multiplication

Multiplication of a sparse matrix with a dense matrix is a building block of an increasing number of applications in many areas such as machine learning and graph algorithms. However, most previous work on parallel matrix multiplication considered only both dense or both sparse matrix operands. This paper analyzes the communication lower bounds and compares the communication costs of various classic parallel algorithms in the context of sparse-dense matrix-matrix multiplication. We also present new communication-avoiding algorithms based on a 1D decomposition, called 1.5D, which - while suboptimal in dense-dense and sparse-sparse cases - outperform the 2D and 3D variants both theoretically and in practice for sparse-dense multiplication. Our analysis separates one-time costs from per iteration costs in an iterative machine learning context. Experiments demonstrate speedups up to 100x over a baseline 3D SUMMA implementation and show parallel scaling over 10 thousand cores.

Koanantakool, Penporn↗

Scalable Reactive Molecular Dynamics Simulations for Computational Synthesis

Reactive molecular dynamics (MD) simulation is a powerful research tool for describing chemical reactions. We eliminate the speed-limiting charge iteration in MD with a novel extended-Lagrangian scheme. The extended-Lagrangian reactive MD (XRMD) code drastically improves energy conservation while substantially reducing time-to-solution. Furthermore, we introduce a new polarizable charge equilibration (PQEq) model to accurately predict atomic charges and polarization. The XRMD code based on hybrid message passing+multithreading achieves a weak-scaling parallel efficiency of 0.977 on 786 432 IBM Blue Gene/Q cores for a 67.6 billion-atom system. The performance is portable to the second-generation Intel Xeon Phi, Knights Landing. Blue Gene/Q simulations for the computational synthesis of materials via novel exfoliation mechanisms for synthesizing atomically thin transition metal dichalcogenide layers will dominate nanomaterials science in this century.

Simulation, Modeling, Performance, and Visualizati↗

Diagnosis of Performance Faults in LargeScale MPI Applications via Probabilistic Progress-Dependence Inference

Debugging large-scale parallel applications is challenging. Most existing techniques provide little information about failure root causes. Further, most debuggers significantly slow down program execution, and run sluggishly with massively parallel applications. This paper presents a novel technique that scalably infers the tasks in a parallel program on which a failure occurred, as well as the code in which it originated. Our technique combines scalable runtime analysis with static analysis to determine the least-progressed task(s) and to identify the code lines at which the failure arose. We present a novel algorithm that infers probabilistically progress dependence among MPI tasks using a globally constructed Markov model that represents tasks' control-flow behavior. In comparison to previous work, our algorithm infers more precisely the least-progressed task. Further, we combine this technique with static backward slicing analysis, further isolating the code responsible for the current state. A blind study demonstrates that our technique isolates the root cause of a concurrency bug in a molecular dynamics simulation, which only manifests itself at 7,996 tasks or more. We extensively evaluate fault coverage of our technique via fault injections in 10 HPC benchmarks and show that our analysis takes less than a few seconds on thousands of parallel tasks.

97 MATHEMATICS AND COMPUTING↗

Visualizing an Exascale Data Center Digital Twin: Considerations, Challenges and Opportunities

Digital twins are an excellent tool to model, visualize, and simulate complex systems, to understand and optimize their operation. In this work, we present the technical challenges of real-time visualization of a digital twin of the Frontier supercomputer.We show the initial prototype and current state of the twin and highlight technical design challenges of visualizing such a large High Performance Computing (HPC) system. The goal is to understand the use of augmented reality as a primary way to extract information and collaborate on digital twins of complex systems. This leverages the spatio-temporal aspect of a 3D representation of a digital twin, with the ability to view historical and real-time telemetry, triggering simulations of a system state and viewing the results, which can be augmented via dashboards for details. Finally, we discuss considerations and opportunities for augmented reality of digital twins of large-scale, parallel computers.

Maiterth, Matthias↗

Non-Galerkin Coarse Grids for Algebraic Multigrid

Algebraic multigrid (AMG) is a popular and effective solver for systems of linear equations that arise from discretized partial differential equations. And while AMG has been effectively implemented on large scale parallel machines, challenges remain, especially when moving to exascale. Particularly, stencil sizes (the number of nonzeros in a row) tend to increase further down in the coarse grid hierarchy, and this growth leads to more communication. Therefore, as problem size increases and the number of levels in the hierarchy grows, the overall efficiency of the parallel AMG method decreases, sometimes dramatically. This growth in stencil size is due to the standard Galerkin coarse grid operator, $P^T A P$, where $P$ is the prolongation (i.e., interpolation) operator. For example, the coarse grid stencil size for a simple three-dimensional (3D) seven-point finite differencing approximation to diffusion can increase into the thousands on present day machines, causing an associated increase in communication costs. We therefore consider algebraically truncating coarse grid stencils to obtain a non-Galerkin coarse grid. First, the sparsity pattern of the non-Galerkin coarse grid is determined by employing a heuristic minimal “safe” pattern together with strength-of-connection ideas. Second, the nonzero entries are determined by collapsing the stencils in the Galerkin operator using traditional AMG techniques. The result is a reduction in coarse grid stencil size, overall operator complexity, and parallel AMG solve phase times.

97 MATHEMATICS AND COMPUTING↗

Multigrid Reduction in Time for Nonlinear Parabolic Problems: A Case Study

The need for parallelism in the time dimension is being driven by changes in computer architectures, where performance increases are now provided through greater concurrency, not faster clock speeds. This creates a bottleneck for sequential time marching schemes because they lack parallelism in the time dimension. Multigrid reduction in time (MGRIT) is an iterative procedure that allows for temporal parallelism by utilizing multigrid reduction techniques and a multilevel hierarchy of coarse time grids. MGRIT has been shown to be effective for linear problems, with speedups of up to 50 times. The goal of this work is the efficient solution of nonlinear problems with MGRIT, where efficiency is defined as achieving similar performance when compared to an equivalent linear problem. The benchmark nonlinear problem is the p-Laplacian, where p=4 corresponds to a well-known nonlinear diffusion equation and p=2 corresponds to the standard linear diffusion operator, our benchmark linear problem. The key difficulty encountered is that the nonlinear time-step solver becomes progressively more expensive on coarser time levels as the time-step size increases. To overcome such difficulties, multigrid research has historically targeted an accumulated body of experience regarding how to choose an appropriate solver for a specific problem type. To that end, this paper develops a library of MGRIT optimizations and modifications, most important an alternate initial guess for the nonlinear time-step solver and delayed spatial coarsening, that will allow many nonlinear parabolic problems to be solved with parallel scaling behavior comparable to the corresponding linear problem.

parareal↗

A Parallel Cut-Cell Algorithm for the Free-Boundary Grad--Shafranov Problem

A parallel cut-cell algorithm is described to solve the free-boundary problem of the Grad--Shafranov equation. The algorithm reformulates the free-boundary problem in an irregular bounded domain and its important aspects include a searching algorithm for the magnetic axis and separatrix, a surface integral along the irregular boundary to determine the boundary values, an approach to optimize the coil current based on a targeting plasma shape, Picard iterations with Aitken's acceleration for the resulting nonlinear problem, and a Cartesian grid embedded boundary method to handle the complex geometry. Here the algorithm is implemented in parallel using a standard domain-decomposition approach and a good parallel scaling is observed. Numerical results verify the accuracy and efficiency of the free-boundary Grad--Shafranov solver.

97 MATHEMATICS AND COMPUTING↗

Iterated Gauss-Seidel GMRES

The GMRES algorithm of Saad and Schultz [SIAM J. Sci. Stat. Comput., 7 (1986), pp. 856-869] is an iterative method for approximately solving linear systems Ax = b, with initial guess x0 and residual r0 = b Ax0. The algorithm employs the Arnoldi process to generate the Krylov basis vectors (the columns of Vk ). It is well known that this process can be viewed as a QR factorization of the matrix Bk = [r0, AVk] at each iteration. Despite an O (..epsilon..)..kappa.. (Bk ) loss of orthogonality, for unit roundoff ..epsilon..and condition number ..kappa.. , the modified Gram-Schmidt formulation was shown to be backward stable in the seminal paper by Paige et al. [SIAM J. Matrix Anal.Appl., 28 (2006), pp. 264-284]. We present an iterated Gauss-Seidel formulation of the GMRES algorithm (IGS-GMRES) based on the ideas of Ruhe [Linear Algebra Appl., 52 (1983), pp. 591-601] and Swirydowicz et al. [Numer. Linear Algebra Appl., 28 (2020), pp. 1-20]. IGS-GMRES maintains orthogonality to the level O (..epsilon..)..kappa.. (Bk ) or O (..epsilon..), depending on the choice of one or two iterations; for two Gauss-Seidel iterations, the computed Krylov basis vectors remain orthogonal to working accuracy and the smallest singular value of Vk remains close to one. The resulting GMRES method is thus backward stable. We show that IGS-GMRES can be implemented with only a single synchronization point per iteration, making it relevant to large-scale parallel computing environments. We also demonstrate that, unlike MGS-GMRES, in IGS-GMRES the relative Arnoldi residual corresponding to the computed approximate solution no longer stagnates above machine precision even for highly nonnormal systems.

Arnoldi-QR↗

An LLVM Instrumentation Plug-in for Score-P

Reducing application runtime, scaling parallel applications to higher numbers of processes/threads, and porting applications to new hardware architectures are tasks necessary in the software development process. Therefore, developers have to investigate and understand application runtime behavior. Tools such as monitoring infrastructures that capture performance relevant data during application execution assist in this task. The measured data forms the basis for identifying bottlenecks and optimizing the code.Monitoring infrastructures need mechanisms to record application activities in order to conduct measurements. Automatic instrumentation of the source code is the preferred method in most application scenarios. We introduce a plug-in for the LLVM infrastructure that enables automatic source code instrumentation at compile-time. In contrast to available instrumentation mechanisms in LLVM/Clang, our plug-in can selectively include/exclude individual application functions. This enables developers to fine-tune the measurement to the required level of detail while avoiding large runtime overheads due to excessive instrumentation.

Tschüter, Ronny↗

Improved unconstrained energy functional method for eigensolvers in electronic structure calculations

This paper reports on the performance of a preconditioned conjugate gradient based iterative eigensolver using an unconstrained energy functional minimization scheme. In contrast to standard implementations, this scheme avoids an explicit reorthogonalization of the trial eigenvectors and becomes an attractive alternative for the solution of very large problems. The unconstrained formulation is implemented in the first-principles materials and chemistry CP2K code, which performs electronic structure calculations based on a density functional theory approximation to the solution of the many-body Schrödinger equation. We study the convergence of the unconstrained formulation, as well as its parallel scaling, on a Cray XC40 at the National Energy Research Scientific Computing Center (NERSC). The systems we use in our studies are bulk liquid water, a supramolecular catalyst gold(III)-complex, a bilayer of MoS2-WSe2 and a divacancy point defect in silicon, with the number of atoms ranging from 2,247 to 12,288. We show that the unconstrained formulation with an appropriate preconditioner has good convergence properties and scales well to 230k cores, roughly 38% of the full machine.

Del Ben, M↗

Quantum Dynamics at Scale: Ultrafast Control of Emergent Functional Materials

Confluence of extreme-scale quantum dynamics simulations (i.e. quantum@scale) and cutting-edge x-ray free-electron laser experiments are revolutionizing materials science. An archetypal example is the exciting concept of using picosecond light pulses to control emergent material properties on demand in atomically-thin layered materials. This paper describes efforts to scale our quantum molecular dynamics engine toward the United States' first exaflop/s computer, under an Aurora Early Science Program project named "Metascalable layered material genome". Key algorithmic and computing techniques incorporated are: (1) globally-scalable and locally-fast solvers within a linear-scaling divide-conquer-recombine algorithmic framework; (2) algebraic `BLASification' of computational kernels; and (3) data alignment and loop restructuring, along with register and cache blocking, for enhanced vectorization and efficient memory access. The resulting weak-scaling parallel efficiency was 0.93 on 131,072 Intel Xeon Phi cores for a 56.6 million atom (or 169 million valence-electron) system, whereas the various code transformations achieved 5-fold speedup. The optimized simulation engine allowed us for the first time to establish a significant effect of substrate on the dynamics of layered material upon electronic excitation.

Algebraic BLASification↗

A GPU-Accelerated Population Generation, Sorting, and Mutation Kernel for an Optimization-Based Causal Inference Model

We develop a GPU-accelerated machine learning generative adversarial network model that can be used with observational data for the purpose of constructing causal inferences. The theoretical basis of our machine learning model is novel and is conceptualized to be operable and scalable for high performance computing platforms. Our GPU-accelerated code enables large-scale parallelization of the computation within a common and accessible computing environment. This will expand the reach of our model and empower research in new substantive domains while maintaining the underlying theoretical properties.

Cho, Wendy K. Tam↗

Performance Models for the Spike Banded Linear System Solver

With availability of large-scale parallel platforms comprised of tens-of-thousands of processors and beyond, there is significant impetus for the development of scalable parallel sparse linear system solvers and preconditioners. An integral part of this design process is the development of performance models capable of predicting performance and providing accurate cost models for the solvers and preconditioners. There has been some work in the past on characterizing performance of the iterative solvers themselves. In this paper, we investigate the problem of characterizing performance and scalability of banded preconditioners. Recent work has demonstrated the superior convergence properties and robustness of banded preconditioners, compared to state-of-the-art ILU family of preconditioners as well as algebraic multigrid preconditioners. Furthermore, when used in conjunction with efficient banded solvers, banded preconditioners are capable of significantly faster time-to-solution. Our banded solver, the Truncated Spike algorithm is specifically designed for parallel performance and tolerance to deep memory hierarchies. Its regular structure is also highly amenable to accurate performance characterization. Using these characteristics, we derive the following results in this paper: (i) we develop parallel formulations of the Truncated Spike solver, (ii) we develop a highly accurate pseudo-analytical parallel performance model for our solver, (iii) we show excellent predication capabilities of our model – based on which we argue the high scalability of our solver. Our pseudo-analytical performance model is based on analytical performance characterization of each phase of our solver. These analytical models are then parameterized using actual runtime information on target platforms. An important consequence of our performance models is that they reveal underlying performance bottlenecks in both serial and parallel formulations. All of our results are validated on diverse heterogeneous multiclusters – platforms for which performance prediction is particularly challenging. Finally, we provide predict the scalability of the Spike algorithm using up to 65,536 cores with our model. In this paper we extend the results presented in the Ninth International Symposium on Parallel and Distributed Computing.

Manguoglu, Murat↗