ParaEMT: An Open Source, Parallelizable, and HPC-Compatible EMT Simulator for Large-Scale IBR-Rich Power Grids
Not Available
24 POWER TRANSMISSION AND DISTRIBUTION↗
SEARCH · Engineering Papers
Results for “parallel and distributed computing”
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
Velocity-space synthesis of ISEE-1 measurements of the three dimensional electron distribution function
A computer package which produces contour plots of the three dimensional electron distribution function measured by an electron spectrometer aboard ISEE-1 is described. Examples of the contour plots and an explanation of how to use the program, including the necessary computer code for running the program on the GSFC 360/91 computer is presented. The method by which the discrete measurements of the distribution function, given by points on the four dimensional surface are synthesized into a smooth surface in a three dimensional space which can be contoured is described. The velocity components are parallel and perpendicular to the magnetic field, respectively, in the proper frame of the electrons.
Fitzenreiter, R. J.↗
Improving I/O Performance for Exascale Applications through Online Data Layout Reorganization
The applications being developed within the U.S. Exascale Computing Project (ECP) to run on imminent Exascale computers will generate scientific results with unprecedented fidelity and record turn-around time. Many of these codes are based on particle-mesh methods and use advanced algorithms, especially dynamic load-balancing and mesh-refinement, to achieve high performance on Exascale machines. Yet, as such algorithms improve parallel application efficiency, they raise new challenges for I/O logic due to their irregular and dynamic data distributions. Thus, while the enormous data rates of Exascale simulations already challenge existing file system write strategies, the need for efficient read and processing of generated data introduces additional constraints on the data layout strategies that can be used when writing data to secondary storage. We review these I/O challenges and introduce two online data layout reorganization approaches for achieving good tradeoffs between read and write performance. We demonstrate the benefits of using these two approaches for the ECP particle-in-cell simulation WarpX, which serves as a motif for a large class of important Exascale applications. Here, we show that by understanding application I/O patterns and carefully designing data layouts we can increase read performance by more than 80 percent.
97 MATHEMATICS AND COMPUTING↗
Enabling machine learning-ready HPC ensembles with Merlin
With the growing complexity of computational and experimental facilities, many scientific researchers are turning to machine learning (ML) techniques to analyze large scale ensemble data. With complexities such as multi-component workflows, heterogeneous machine architectures, parallel file systems, and batch scheduling, care must be taken to facilitate this analysis in a high performance computing (HPC) environment. Here, we present Merlin, a workflow framework to enable large ML-friendly ensembles of scientific HPC simulations. By augmenting traditional HPC with distributed compute technologies, Merlin aims to lower the barrier for scientific subject matter experts to incorporate ML into their analysis. As a producer–consumer workflow model, Merlin enables multi-machine, cross-batch job, dynamically allocated yet persistent workflows capable of utilizing surge-compute resources. Key features of Merlin are a flexible HPC-centric interface, low per-task overhead, multi-tiered fault recovery, and a hierarchical sampling algorithm that allows for $\mathscr{O}$(N) task execution and $\mathscr{O}$(N ln N) task queuing to ensembles of millions of tasks. In addition to Merlin’s design, we test the algorithm’s performance in an HPC center and demonstrate the ability to enqueue 40 million simulations in 100 s, with a 30 millisecond per-task overhead that is independent of ensemble size. Finally, we describe some example applications that Merlin has enabled on leadership-class HPC resources, such as the ML-augmented optimization of nuclear fusion experiments and the calibration of infectious disease models to study the progression of and possible mitigation strategies for COVID-19.
97 MATHEMATICS AND COMPUTING↗
Coupling of newborn ions to the solar wind by electromagnetic instabilities and their interaction with the bow shock
The process by which the solar wind assimilates newly ionized atoms is important for understanding the presence of planetary or interstellar helium in the solar wind, the dynamics of the Active Magnetospheric Particle Tracer Explorers (AMPTE) lithium releases in front of the earth's bow shock, and the formation of cometary tails. In this paper is examined how newborn ions can be coupled to the solar wind in the direction parallel to the magnetic field by means of electromagnetic instabilities driven by the distribution of newborn ions. The linear properties of three instabilities are analyzed and compared with numerical solutions of the linear dispersion equation, while their nonlinear behavior is followed by means of computer simulation to obtain the characteristic time for the pickup process. With a primary emphasis on the AMPTE lithiuim releases, various degrees of realism are introduced into the calculations to model the upstream conditions and the intersection of the lithium with the bow shock. It is shown that a time-dependent shock model is needed to correctly reproduce the amount of lithium which is transmitted through the shock and that the resulting lithium ion distribution is still likely to be subject to the same type of instabilities in the magnetosheath. Applications of these results to comets, in particular the artificial comet expected to be generated by the AMPTE barium release in the magnetosheath, is also briefly discussed.
Winske, D.↗
A solar chromosphere and spicule model based on far-infrared limb observations
Techniques developed for LTE radiative transfer problems in a rough atmosphere were used to compute a model chromosphere containing spicules consistent with high-resolution solar limb observations from 100 microns to 2.6 mm. The model consists of a smooth, plane-parallel temperature minimum region extending from the photosphere to a height of 1000 km and randomly distributed cylindrical spicules above this height. It is found that the observed limb brightness profiles are well fitted by spicules with electron temperatures on the order of 7000 K.
Braun, D.↗
Parallel Computation of Unsteady Flows on a Network of Workstations
Parallel computation of unsteady flows requires significant computational resources. The utilization of a network of workstations seems an efficient solution to the problem where large problems can be treated at a reasonable cost. This approach requires the solution of several problems: 1) the partitioning and distribution of the problem over a network of workstation, 2) efficient communication tools, 3) managing the system efficiently for a given problem. Of course, there is the question of the efficiency of any given numerical algorithm to such a computing system. NPARC code was chosen as a sample for the application. For the explicit version of the NPARC code both two- and three-dimensional problems were studied. Again both steady and unsteady problems were investigated. The issues studied as a part of the research program were: 1) how to distribute the data between the workstations, 2) how to compute and how to communicate at each node efficiently, 3) how to balance the load distribution. In the following, a summary of these activities is presented. Details of the work have been presented and published as referenced.
Source record↗
ARENA: Asynchronous Reconfigurable Accelerator Ring to Enable Data-Centric Parallel Computing
The next generation HPC and data centers are likely to be reconfigurable and data-centric due to the trend of hardware specialization and the emergence of data-driven applications. In this work, we propose ARENA – an asynchronous reconfigurable accelerator ring architecture as a potential scenario on how the future HPC and data centers will be like. Despite using the coarse-grained reconfigurable arrays (CGRAs) as the substrate platform, our key contribution is not only the CGRA-cluster design itself, but also the ensemble of a new architecture and programming model that enables asynchronous tasking across a cluster of reconfigurable nodes, so as to bring specialized computation to the data rather than the reverse. We presume distributed data storage without asserting any prior knowledge on the data distribution. Hardware specialization occurs at runtime when a task finds the majority of data it requires are available at the present node. In other words, we dynamically generate specialized CGRA accelerators where the data reside. The asynchronous tasking for bringing computation to data is achieved by circulating the task token, which describes the dataflow graphs to be executed for a task, among the CGRA cluster connected by a fast ring network. Evaluations on a set of HPC and data-driven applications across different domains show that ARENA can provide better parallel scalability with reduced data movement (53.9 percent). Compared with contemporary compute-centric parallel models, ARENA can bring on average 4.37× speedup. The synthesized CGRAs and their task-dispatchers only occupy 2.93mm 2 chip area under 45nm process technology and can run at 800MHz with on average 759.8mW power consumption. ARENA also supports the concurrent execution of multi-applications, offering ideal architectural support for future high-performance parallel computing and data analytics systems.
97 MATHEMATICS AND COMPUTING↗
Particlization in fluid dynamical simulations of heavy-ion collisions: The IS3D module
The IS3D particlization module simulates the emission of hadrons from heavy-ion collisions via Monte-Carlo sampling of the Cooper–Frye formula which converts fluid dynamical information into local phase-space distributions for hadrons. The code package includes multiple choices for the non-equilibrium correction to these distribution functions: the 14-moment approximation, first-order Chapman–Enskog expansion, and two types of modified equilibrium distributions. This makes it possible to explore to what extent heavy-ion experimental data are sensitive to different choices for δf n , presently the main source of theoretical uncertainty in the particlization stage. Here, we validate our particle sampler with a high degree of precision by generating several million hadron emission events from a longitudinally boost-invariant hypersurface and comparing the event-averaged particle spectra and space–time distributions to the Cooper–Frye formula.
73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗
The impact of non-local parallel electron transport on plasma-impurity reaction rates in tokamak scrape-off layer plasmas
Abstract Plasma-impurity reaction rates are a crucial part of modelling tokamak scrape-off layer (SOL) plasmas. To avoid calculating the full set of rates for the large number of important processes involved, a set of effective rates are typically derived which assume Maxwellian electrons. However, non-local parallel electron transport may result in non-Maxwellian electrons, particularly close to divertor targets. Here, the validity of using Maxwellian-averaged rates in this context is investigated by computing the full set of rate equations for a fixed plasma background from kinetic and fluid SOL simulations. We consider the effect of the electron distribution as well as the impact of the electron transport model on plasma profiles. Results are presented for lithium, beryllium, carbon, nitrogen, neon and argon. It is found that electron distributions with enhanced high-energy tails can result in significant modifications to the ionisation balance and radiative power loss rates from excitation, on the order of 50%–75% for the latter. Fluid electron models with Spitzer-Härm or flux-limited Spitzer-Härm thermal conductivity, combined with Maxwellian electrons for rate calculations, can increase or decrease this error, depending on the impurity species and plasma conditions. Based on these results, we also discuss some approaches to experimentally observing non-local electron transport in SOL plasmas.
70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗
State-of-the-art Space Telescope Digicon performance data
The Digicon has been chosen as the detector for the High Resolution Spectrograph and the Faint Object Spectrograph of the Space Telescope. Both tubes are 512 channel, parallel-output devices and feature CsTe photocathodes on MgF2 faceplates. Using a computer-assisted test facility, the tubes have been characterized with respect to diode array performance, photocathode response (1100-9000 A), and imaging capability. Data are presented on diode dark current and capacitance distributions, pulse height resolution, photocathode quantum efficiency, uniformity and blemishes, dark count rate, distortion, resolution, and crosstalk.
Ginaven, R. O.↗
Asynchronous interactive control systems
A class of interactive control systems is derived by generalizing interactive manipulator control systems. The general structural properties of such systems are discussed and an appropriate general software implementation is proposed. This is based on the fact that tasks of interactive control systems can be represented as a network of a finite set of actions which have specific operational characteristics and specific resource requirements, and which are of limited duration. This has enabled the decomposition of the overall control algorithm into a set of subalgorithms, called subcontrollers, which can operate simultaneously and asynchronously. Coordinate transformations of sensor feedback data and actuator set-points have enabled the further simplification of the subcontrollers and have reduced their conflicting resource requirements. The modules of the decomposed control system are implemented as parallel processes with disjoint memory space communicating only by I/O. The synchronization mechanisms for dynamic resource allocation among subcontrollers and other synchronization mechanisms are also discussed in this paper. Such a software organization is suitable for the general form of multiprocessing using computer networks with distributed storage.
Vuskovic, M. I.↗
Ponderomotive effects on distributions of O(+) ions in the auroral zone
Test particle calculations are used to compute the effects of gravity and ponderomotive acceleration by shear Alfven wave oscillations on the distribution function of O(+) ions along auroral field lines, assuming an ionospheric Maxwellian source of the ions at 2000 km altitude with approximately 0.5 eV of thermal energy in the parallel component of velocity. The electric field model corresponds to a standing wave oscillation with a frequency approximately 1 Hz in the azimuthal direction superimposed on the background dipole field, in which the wave amplitude is either increasing or decreasing in time. The electric field is taken to be primarily in the perpendicular direction. The time varying wave produces broad distributions with widths of 2 to 10 times the initial 0.5-eV thermal energy of the Maxwellian source, and the density and flux of upward going O(+) ions at one Earth radius are both enhanced in this model. The oxygen ion distribution functions at 1 R(sub E) altitude resulting from interaction with waves whose amplitudes are increasing in time have a more gradual lower energy cutoff than do the distribution functions resulting from decaying waves. The high-energy part of the distribution functions in growing waves reflects the temperature of the Maxwellian source, while the high-energy part of the distributions resulting from decaying waves steepens with time, independent of the source temperature.
Witt, E.↗
GSRP/David Marshall: Fully Automated Cartesian Grid CFD Application for MDO in High Speed Flows
With the renewed interest in Cartesian gridding methodologies for the ease and speed of gridding complex geometries in addition to the simplicity of the control volumes used in the computations, it has become important to investigate ways of extending the existing Cartesian grid solver functionalities. This includes developing methods of modeling the viscous effects in order to utilize Cartesian grids solvers for accurate drag predictions and addressing the issues related to the distributed memory parallelization of Cartesian solvers. This research presents advances in two areas of interest in Cartesian grid solvers, viscous effects modeling and MPI parallelization. The development of viscous effects modeling using solely Cartesian grids has been hampered by the widely varying control volume sizes associated with the mesh refinement and the cut cells associated with the solid surface. This problem is being addressed by using physically based modeling techniques to update the state vectors of the cut cells and removing them from the finite volume integration scheme. This work is performed on a new Cartesian grid solver, NASCART-GT, with modifications to its cut cell functionality. The development of MPI parallelization addresses issues associated with utilizing Cartesian solvers on distributed memory parallel environments. This work is performed on an existing Cartesian grid solver, CART3D, with modifications to its parallelization methodology.
Source record↗
Supercomputing Aspects for Simulating Incompressible Flow
The primary objective of this research is to support the design of liquid rocket systems for the Advanced Space Transportation System. Since the space launch systems in the near future are likely to rely on liquid rocket engines, increasing the efficiency and reliability of the engine components is an important task. One of the major problems in the liquid rocket engine is to understand fluid dynamics of fuel and oxidizer flows from the fuel tank to plume. Understanding the flow through the entire turbo-pump geometry through numerical simulation will be of significant value toward design. One of the milestones of this effort is to develop, apply and demonstrate the capability and accuracy of 3D CFD methods as efficient design analysis tools on high performance computer platforms. The development of the Message Passage Interface (MPI) and Multi Level Parallel (MLP) versions of the INS3D code is currently underway. The serial version of INS3D code is a multidimensional incompressible Navier-Stokes solver based on overset grid technology, INS3D-MPI is based on the explicit massage-passing interface across processors and is primarily suited for distributed memory systems. INS3D-MLP is based on multi-level parallel method and is suitable for distributed-shared memory systems. For the entire turbo-pump simulations, moving boundary capability and efficient time-accurate integration methods are built in the flow solver, To handle the geometric complexity and moving boundary problems, an overset grid scheme is incorporated with the solver so that new connectivity data will be obtained at each time step. The Chimera overlapped grid scheme allows subdomains move relative to each other, and provides a great flexibility when the boundary movement creates large displacements. Two numerical procedures, one based on artificial compressibility method and the other pressure projection method, are outlined for obtaining time-accurate solutions of the incompressible Navier-Stokes equations. The performance of the two methods is compared by obtaining unsteady solutions for the evolution of twin vortices behind a flat plate. Calculated results are compared with experimental and other numerical results. For an unsteady flow, which requires small physical time step, the pressure projection method was found to be computationally efficient since it does not require any subiteration procedure. It was observed that the artificial compressibility method requires a fast convergence scheme at each physical time step in order to satisfy the incompressibility condition. This was obtained by using a GMRES-ILU(0) solver in present computations. When a line-relaxation scheme was used, the time accuracy was degraded and time-accurate computations became very expensive.
Kwak, Dochan↗
Globalized Newton-Krylov-Schwarz Algorithms and Software for Parallel Implicit CFD
Implicit solution methods are important in applications modeled by PDEs with disparate temporal and spatial scales. Because such applications require high resolution with reasonable turnaround, "routine" parallelization is essential. The pseudo-transient matrix-free Newton-Krylov-Schwarz (Psi-NKS) algorithmic framework is presented as an answer. We show that, for the classical problem of three-dimensional transonic Euler flow about an M6 wing, Psi-NKS can simultaneously deliver: globalized, asymptotically rapid convergence through adaptive pseudo- transient continuation and Newton's method-, reasonable parallelizability for an implicit method through deferred synchronization and favorable communication-to-computation scaling in the Krylov linear solver; and high per- processor performance through attention to distributed memory and cache locality, especially through the Schwarz preconditioner. Two discouraging features of Psi-NKS methods are their sensitivity to the coding of the underlying PDE discretization and the large number of parameters that must be selected to govern convergence. We therefore distill several recommendations from our experience and from our reading of the literature on various algorithmic components of Psi-NKS, and we describe a freely available, MPI-based portable parallel software implementation of the solver employed here.
Gropp, W. D.↗
Model-parallel Fourier neural operators as learned surrogates for large-scale parametric PDEs
Fourier neural operators (FNOs) are a recently introduced neural network architecture for learning solution operators of partial differential equations (PDEs), which have been shown to perform significantly better than comparable deep learning approaches. Once trained, FNOs can achieve speed-ups of multiple orders of magnitude over conventional numerical PDE solvers. However, due to the high dimensionality of their input data and network weights, FNOs have so far only been applied to two-dimensional or small three-dimensional problems. To remove this limited problem-size barrier, we propose a model-parallel version of FNOs based on domain-decomposition of both the input data and network weights. Here, we demonstrate that our model-parallel FNO is able to predict time-varying PDE solutions of over 2.6 billion variables on Perlmutter using up to 512 A100 GPUs and show an example of training a distributed FNO on the Azure cloud for simulating multiphase CO 2 dynamics in the Earth’s subsurface.
58 GEOSCIENCES↗
Current possibilities and future opportunities for erasure coded computations
The key capability established through the research funded by this award are erasure coded computations for linear systems, in serial and in parallel. This capability enables powerful efficient and scalable alternatives to existing linear system solvers in fault-prone computational systems.
97 MATHEMATICS AND COMPUTING↗