Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Message Passing”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 235 records · Page 13
All sourcesNASA NTRSDOE OSTI
DOE OSTI2022

GenASiS Basics: Object-oriented utilitarian functionality for large-scale physics simulations (Version 4)

GenASiS Basics provides modern Fortran classes furnishing extensible object-oriented utilitarian functionality for large-scale physics simulations on distributed memory supercomputers. This functionality includes physical units and constants; display to the screen or standard output device; message passing; I/O to disk; and runtime parameter management and usage statistics. Herein, this revision—Version 4 of Basics—includes a name change and additions to functionality, including the facilitation of direct communication between GPUs.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
DOE OSTI2024

A parallel and performance portable implementation of a full-field crystal plasticity model

We have developed a parallel implementation of an Elasto-Viscoplastic Fast Fourier Transform-based (EVPFFT) micromechanical solver to enable computationally efficient crystal plasticity modeling for polycrystalline materials. Our primary focus lies in achieving performance portability, allowing a single EVPFFT implementation to run optimally on various homogeneous architectures, including multi-core Central Processing Units (CPUs), as well as on heterogeneous computer architectures comprising multi-core CPUs and Graphics Processing Units (GPUs) from different vendors. To accomplish this goal, we have leveraged MATAR, a C++ software library that simplifies the creation and utilization of multidimensional dense or sparse matrix and array data structures. These data structures are designed to be portable across diverse architectures through the use of Kokkos, a performance-portable library. Additionally, we have employed the Message Passing Interface (MPI) to efficiently distribute the computational workload among processors. The heFFTe (Highly Efficient FFT for Exascale) library is used to facilitate the performance portability of the fast Fourier transforms (FFTs) computation. The computational performance of EVPFFT is evaluated and presented in terms of parallel scalability and simulation runtime on different high-performance computing (HPC) architectures. As a result, the utility of the developed framework to efficiently simulate the micro-mechanical fields in polycrystalline microstructures in engineering applications is discussed.

36 MATERIALS SCIENCE↗
DOE OSTI2024

Massively parallel axisymmetric fluid model for streamer discharges

A highly parallelizable fluid plasma simulation tool based upon the first-order drift-diffusion equations is discussed. Atmospheric pressure plasmas have densities and gradients that require small element sizes in order to accurately simulate the plasm resulting in computational meshes on the order of millions to tens of millions of elements for realistic size plasma reactors. To enable simulations of this nature, parallel computing is required and must be optimized for the particular problem. Here, a finite-volume, electrostatic drift-diffusion implementation for low-temperature plasma is discussed. The implementation is built upon the Message Passing Interface (MPI) library in C++ using Object Oriented Programming. The underlying numerical method is outlined in detail and benchmarked against simple streamer formation from other streamer codes. Electron densities, electric field, and propagation speeds are compared with the reference case and show good agreement. Convergence studies are also performed showing a minimal space step of approximately 4 μm required to reduce relative error to below 1% during early streamer simulation times and even finer space steps are required for longer times. Additionally, strong and weak scaling of the implementation are studied and demonstrate the excellent performance behavior of the implementation up to 100 million elements on 1024 processors. Lastly, different advection schemes are compared for the simple streamer problem to analyze the influence of numerical diffusion on the resulting quantities of interest.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗
DOE OSTI2024

QRCODE: Massively parallelized real-time time-dependent density functional theory for periodic systems

We present a new software module, QRCODE (Quantum Research for Calculating Optically Driven Excitations), for massively parallelized real-time time-dependent density functional theory (RT-TDDFT) calculations of periodic systems in the open-source Qbox software package. Our approach utilizes a custom implementation of a fast Fourier transformation scheme that significantly reduces inter-node message passing interface (MPI) communication of the major computational kernel and shows impressive scaling up to 16,344 CPU cores. In addition to improving computational performance, QRCODE contains a suite of various time propagators for accurate RT-TDDFT calculations. As benchmark applications of QRCODE, we calculate the current density and optical absorption spectra of hexagonal boron nitride (h-BN) and photo-driven reaction dynamics of the ozone-oxygen reaction. We also calculate the second and higher harmonic generation of monolayer and multi-layer boron nitride structures as examples of large material systems. Our optimized implementation of RT-TDDFT in QRCODE enables large-scale calculations of real-time electron dynamics of chemical and material systems with enhanced computational performance and impressive scaling across several thousand CPU cores.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2026

Enhancing photoionization rate calculations in low-temperature plasmas using spectral methods

Photoionization plays a central role in the development of streamer discharges and other non-equilibrium plasma phenomena. It creates seed electrons, which are essential for positive streamer propagation, allowing the ionization front to move forward. Because of this, accurate modeling of photoionization is very important for predicting streamer behavior and plasma evolution. The photoionization process in air (N 2 – O 2 mixture) is often described by the Zheleznyak model (1982). This model is usually solved through Helmholtz-type equations that approximate the Zheleznyak photoionization model (Zheleznyak et al. 1982) as Partial Differential Equations (PDEs). Conventional numerical methods, such as the Finite Difference Method (FDM) or Finite Volume Method (FVM), are widely used to solve these equations. Although they are prevalent, the computational cost of these methods due to their need for matrix operations and iterative solver is demanding. To address this challenge, this work develops a spectral solver based on the Fast Fourier Transform (FFT) combined with Discrete Cosine Transform (DCT) and Discrete Sine Transform (DST) to calculate the photoionization rate efficiently in an axisymmetric cylindrical domain. This method naturally satisfies the boundary conditions used in the model and converts the PDE into algebraic ones in spectral space. Thus, avoids the need for iterative matrix solvers. When compared with FDM results, it is demonstrated that the new solver not only maintains accuracy, but also reduces the computational cost, showing a performance increase of approximately 100 compared to FDM over a wide range of problem sizes. The method is parallelized using Message Passing Interface (MPI) and has been integrated into a fluid plasma model for streamer simulation. Here, this FFT-based approach provides a fast and reliable alternative for calculating photoionization in fluid models, helping large-scale plasma simulations run faster and efficiently, and allows higher-resolution simulation without extra computational cost.

Axisymmetric system↗
DOE OSTI2019

Cross-scale modeling of storm surge, tide, and inundation in Mid-Atlantic Bight and New York City during Hurricane Sandy, 2012

Driven by high-resolution NAM (North America Mesoscale) and ECMWF (European Centre for Medium-Range Weather Forecasts) atmospheric fields, a 3-D unstructured grid SCHISM (Semi-implicit Cross-scale Hydroscience Integrated System Model) was applied to simulate total water level in the Mid-Atlantic Bight during Hurricane Sandy in 2012. The simulated storm surge, tide, significant wave height, and wave peak period results compared favorably with NOAA observations along US East Coast, Long Island Sound, and New York Harbor. The maximum total water level at The Battery in the New York Harbor was accurately simulated with an absolute error of less than 0.08 m (out of 2.9 m peak surge) and a timing difference less than 10 min, which includes the effects on the order of 0.1–0.3 m (5–10%) from the coastal setup induced by the surface waves. The scenario comparisons of (1) “NAM” versus “ECMWF”, (2) “2-D” versus “3-D”, and (3) “with” versus “without” wind wave model were examined. The 3-D barotropic model forced by ECMWF including the effects of wind waves performs the best, attributed to wave-induced radiation stress and reduction of bottom stress when the 3-D version is used. Simultaneously, the ELCIRC-Sub model, using a 5-m topography/bathymetry sub-grid and a regular 200-m resolution finite volume computational model grid, was developed to simulate the street-level inundation in New York City. The momentum and mass fluxes calculated by the coarser base grid model were effectively coupled with the sub-grid so that running a full-blown high-resolution base model is not required. The ELCIRC-Sub model uses MPI (Message Passing Interface) parallel computing to enlarge the coverage of the land surface and an efficient non-linear solver to improve the accuracy of the wetting-and-drying processes. The temporal comparisons of modeled water level with NOAA's tidal stations and USGS' rapid-deployed gauges showed overall performance with an average error of 0.1 m. Particularly, ELCIRC-Sub captured the profile of the highest peak surge (3.9 m), which rose 2 m within 3 hour at Kings Point, NY. The spatial comparisons of the modeled peak water level at 80 surveyed locations showed an average error less than 0.13 m. The modeled maximum inundation extent also matched well with an 80% hit rate against FEMA's Hurricane Sandy maximum flooding extent.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2011

Extraction of Hydrological Proximity Measures from DEMs using Parallel Processing

Land surface topography is one of the most important terrain properties which impact hydrological, geomorphological, and ecological processes active on a landscape. In our previous efforts to develop a soil depth model based upon topographic and land cover variables, we extracted a set of hydrological proximity measures (HPMs) from a Digital Elevation Model (DEM) as potential explanatory variables for soil depth. These HPMs may also have other, more general modeling applicability in hydrology, geomorphology and ecology, and so are described here from a general perspective. The HPMs we derived are variations of the distance up to ridge points (cells with no incoming flow) and variations of the distance down to stream points (cells with a contributing area greater than a threshold), following the flow path. These HPMs were computed using the D-infinity flow model that apportions flow between adjacent neighbors based on the direction of steepest downward slope on the eight triangular facets constructed in a 3 x 3 grid cell window using the center cell and each pair of adjacent neighboring grid cells in turn. The D-infinity model typically results in multiple flow paths between 2 points on the topography, with the result that distances may be computed as the minimum, maximum or average of the individual flow paths. In addition, each of the HPMs, are calculated vertically, horizontally, and along the land surface. Previously, these HPMs were calculated using recursive serial algorithms which suffered from stack overflow problems when used to process large datasets, limiting the size of DEMs that could be analyzed using that method to approximately 7000 x 7000 cells. To overcome this limitation, we developed a message passing interface (MPI) parallel approach for calculating these HPMs. The parallel algorithms of the HPMs spatially partition the input grid into stripes which are each assigned to separate processes for computation. Each of those processes then uses a queue data structure to order the processing of cells so that each cell is visited only once and the cross-process communications that are a standard part of MPI are handled in an efficient manner. This parallel approach allows analysis of much larger DEMs as compared to the serial recursive algorithms. In this paper, we present the definitions of the HPMs, the serial and parallel algorithms used in their extraction and their potential applications in hydrology, geomorphology and ecology.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2024

Toward a high-fidelity tritium transport modeling for retention and permeation experiments

We report Tritium Migration Analysis Program version 8 (TMAP8), the latest version of TMAP, was developed within the framework of the Multiphysics Object-Oriented Simulation Environment (MOOSE). Created at Idaho National Laboratory (INL), MOOSE is an open-source, dimension-agnostic, fully coupled, and fully implicit multiphysics platform featuring massively parallel computation capabilities. Using TMAP8, tritium transport in a divertor monoblock was analyzed to elucidate the effects of pulsed operation (up to fifty 1,600 s plasma discharge and cool-down cycles) on the tritium in-vessel inventory source term and ex-vessel release term (i.e., tritium retention and permeation) for safety analysis. With its built-in Message Passing Interface capability, TMAP8 can, in under 2 h, simulate tritium transport in three different layered materials (i.e., tungsten, copper, and copper-chromium-zirconium alloy) in 2D geometry, using a single device/computer with 10 cores. The MOOSE-based TMAP8 code can leverage other MOOSE tools developed under the Nuclear Energy Advanced Modeling and Simulation program to perform tritium and thermal transport in complex geometries and multiphysics environments. And via its massively parallel computation, MOOSE will enable the fusion pilot plant designers to conduct high-fidelity multiphysics modeling for the design of the divertor and blanket systems as well as for the safety analysis.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗
DOE OSTI2014

Scaling To A Million Cores And Beyond: Using Light-Weight Simulation to Understand The Challenges Ahead On The Road To Exascale

As supercomputers scale to 1,000 PFlop/s over the next decade, investigating the performance of parallel applications at scale on future architectures and the performance impact of different architecture choices for high-performance computing (HPC) hardware/software co-design is crucial. This paper summarizes recent efforts in designing and implementing a novel HPC hardware/software co-design toolkit. The presented Extreme-scale Simulator (xSim) permits running an HPC application in a controlled environment with millions of concurrent execution threads while observing its performance in a simulated extreme-scale HPC system using architectural models and virtual timing. This paper demonstrates the capabilities and usefulness of the xSim performance investigation toolkit, such as its scalability to 2^27 simulated Message Passing Interface (MPI) ranks on 960 real processor cores, the capability to evaluate the performance of different MPI collective communication algorithms, and the ability to evaluate the performance of a basic Monte Carlo application with different architectural parameters.

Engelmann, Christian↗
DOE OSTI2015

Implementation of Parallel Dynamic Simulation on Shared-Memory vs. Distributed-Memory Environments

Power system dynamic simulation computes the system response to a sequence of large disturbance, such as sudden changes in generation or load, or a network short circuit followed by protective branch switching operation. It consists of a large set of differential and algebraic equations, which is computational intensive and challenging to solve using single-processor based dynamic simulation solution. High-performance computing (HPC) based parallel computing is a very promising technology to speed up the computation and facilitate the simulation process. This paper presents two different parallel implementations of power grid dynamic simulation using Open Multi-processing (OpenMP) on shared-memory platform, and Message Passing Interface (MPI) on distributed-memory clusters, respectively. The difference of the parallel simulation algorithms and architectures of the two HPC technologies are illustrated, and their performances for running parallel dynamic simulation are compared and demonstrated.

dynamic simulation↗
DOE OSTI2020

Development of a CPU/GPU portable software library for Lagrangian–Eulerian simulations of liquid sprays

The Lagrangian–Eulerian method is widely used for simulations of fuel sprays in turbulent combustion because of the advantage of treating the spray droplets as discrete points. One challenge of the Lagrangian–Eulerian method is the intense computational requirement when tracking the large number of Lagrangian particles needed for high fidelity. We have developed a performance-portable library, Grit, to track the Lagrangian particles in parallel on central processing unit (CPU) and graphics processing unit (GPU) accelerated high performance computing (HPC) architectures. Grit is a C++ library which employs Message Passing Interface (MPI) for distributed memory parallelism and Kokkos programming model for on-node shared memory parallelism with performance portability across different architectures of GPUs and multi-core/manycore CPUs. The parallel algorithms, key parallel kernels, and their performances on the pre-exascale supercomputer, Summit, are presented. Grit is coupled with a direct numerical simulation (DNS) solver, S3D, for multiphase simulations. A conservative formulation has been developed and implemented in Grit for phase coupling with thermodynamic consistency. The formulation separates the conservation of mass, momentum and energy from the physical models to prevent accidental violation of conservation laws due to inconsistent models. The formulation also enforces consistent definitions of enthalpies of the fuel for both phases and the latent heat of evaporation. Finally, simulations of turbulent particle-laden flow and the evaporation of dilute turbulent spray jet are performed to verify the software implementation, and to demonstrate the scalability of Grit for large-scale multiphase simulations.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2010

On the use of flux limiters in the discrete ordinates method for 3D radiation calculations in absorbing and scattering media

The application of flux limiters to the discrete ordinates method (DOM), S{sub N}, for radiative transfer calculations is discussed and analyzed for 3D enclosures for cases in which the intensities are strongly coupled to each other such as: radiative equilibrium and scattering media. A Newton-Krylov iterative method (GMRES) solves the final systems of linear equations along with a domain decomposition strategy for parallel computation using message passing libraries in a distributed memory system. Ray effects due to angular discretization and errors due to domain decomposition are minimized until small variations are introduced by these effects in order to focus on the influence of flux limiters on errors due to spatial discretization, known as numerical diffusion, smearing or false scattering. Results are presented for the DOM-integrated quantities such as heat flux, irradiation and emission. A variety of flux limiters are compared to 'exact' solutions available in the literature, such as the integral solution of the RTE for pure absorbing-emitting media and isotropic scattering cases and a Monte Carlo solution for a forward scattering case. Additionally, a non-homogeneous 3D enclosure is included to extend the use of flux limiters to more practical cases. The overall balance of convergence, accuracy, speed and stability using flux limiters is shown to be superior compared to step schemes for any test case.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
DOE OSTI2012

Solution to PDEs using radial basis function finite-differences (RBF-FD) on multiple GPUs

This paper presents parallelization strategies for the radial basis function-finite difference (RBF-FD) method. As a generalized finite differencing scheme, the RBF-FD method functions without the need for underlying meshes to structure nodes. It offers high-order accuracy approximation and scales as O(N) per time step, with N being with the total number of nodes. To our knowledge, this is the first implementation of the RBF-FD method to leverage GPU accelerators for the solution of PDEs. Additionally, this implementation is the first to span both multiple CPUs and multiple GPUs. OpenCL kernels target the GPUs and inter-processor communication and synchronization is managed by the Message Passing Interface (MPI). We verify our implementation of the RBF-FD method with two hyperbolic PDEs on the sphere, and demonstrate up to 9x speedup on a commodity GPU with unoptimized kernel implementations. On a high performance cluster, the method achieves up to 7x speedup for the maximum problem size of 27,556 nodes.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
DOE OSTI2013

Parallel multiscale simulations of a brain aneurysm

Cardiovascular pathologies, such as a brain aneurysm, are affected by the global blood circulation as well as by the local microrheology. Hence, developing computational models for such cases requires the coupling of disparate spatial and temporal scales often governed by diverse mathematical descriptions, e.g., by partial differential equations (continuum) and ordinary differential equations for discrete particles (atomistic). However, interfacing atomistic-based with continuum-based domain discretizations is a challenging problem that requires both mathematical and computational advances. We present here a hybrid methodology that enabled us to perform the first multiscale simulations of platelet depositions on the wall of a brain aneurysm. The large scale flow features in the intracranial network are accurately resolved by using the high-order spectral element Navier–Stokes solver NεκTαr. The blood rheology inside the aneurysm is modeled using a coarse-grained stochastic molecular dynamics approach (the dissipative particle dynamics method) implemented in the parallel code LAMMPS. The continuum and atomistic domains overlap with interface conditions provided by effective forces computed adaptively to ensure continuity of states across the interface boundary. A two-way interaction is allowed with the time-evolving boundary of the (deposited) platelet clusters tracked by an immersed boundary method. The corresponding heterogeneous solvers (NεκTαr and LAMMPS) are linked together by a computational multilevel message passing interface that facilitates modularity and high parallel efficiency. Results of multiscale simulations of clot formation inside the aneurysm in a patient-specific arterial tree are presented. We also discuss the computational challenges involved and present scalability results of our coupled solver on up to 300 K computer processors. Validation of such coupled atomistic-continuum models is a main open issue that has to be addressed in future work.

60 APPLIED LIFE SCIENCES↗
DOE OSTI2013

Extending the length and time scales of Gram–Schmidt Lyapunov vector computations

Lyapunov vectors have found growing interest recently due to their ability to characterize systems out of thermodynamic equilibrium. The computation of orthogonal Gram–Schmidt vectors requires multiplication and QR decomposition of large matrices, which grow as N{sup 2} (with the particle count). This expense has limited such calculations to relatively small systems and short time scales. Here, we detail two implementations of an algorithm for computing Gram–Schmidt vectors. The first is a distributed-memory message-passing method using Scalapack. The second uses the newly-released MAGMA library for GPUs. We compare the performance of both codes for Lennard–Jones fluids from N=100 to 1300 between Intel Nahalem/Infiniband DDR and NVIDIA C2050 architectures. To our best knowledge, these are the largest systems for which the Gram–Schmidt Lyapunov vectors have been computed, and the first time their calculation has been GPU-accelerated. We conclude that Lyapunov vector calculations can be significantly extended in length and time by leveraging the power of GPU-accelerated linear algebra.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2013

An improved iteration loop for the three dimensional quasi-static particle-in-cell algorithm: QuickPIC

We present improvements to the three-dimensional (3D) quasi-static particle-in-cell (PIC) algorithm, which is used to efficiently model short-pulse laser and particle beam–plasma interactions. In this algorithm the fields including the index of refraction created by a static particle/laser beam are calculated. These fields are then used to advance the particle/laser beam forward in time (distance). For a 3D quasi-static code, calculating the wake fields is done using a two-dimensional (2D) PIC code where the time variable is ξ=ct-z and z is the propagation direction of the particle/laser beam. When calculating the wake, the fields, particle positions and momenta are not naturally time centered so an iterative predictor corrector loop is required. In the previous iterative loop in QuickPIC (currently the only 3D quasi-static PIC code), the field equations are derived using the Lorentz gauge. Here we describe a new algorithm which uses gauge independent field equations. It is found that with this new algorithm, the results converge to the results from fully explicitly PIC codes with far fewer iterations (typically 1 iteration as compared to 2–8) for a wide range of problems. In addition, we describe a new deposition scheme for directly depositing the time derivative of the current that is needed in one of the field equations. The new deposition scheme does not require message passing for the particles inside the iteration loop, which greatly improves the speed for parallelized calculations. Comparisons of results from the new and old algorithms and to fully explicit PIC codes are also presented.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗
DOE OSTI2014

An efficient parallel implementation of explicit multirate Runge–Kutta schemes for discontinuous Galerkin computations

Although explicit time integration schemes require small computational efforts per time step, their efficiency is severely restricted by their stability limits. Indeed, the multi-scale nature of some physical processes combined with highly unstructured meshes can lead some elements to impose a severely small stable time step for a global problem. Multirate methods offer a way to increase the global efficiency by gathering grid cells in appropriate groups under local stability conditions. These methods are well suited to the discontinuous Galerkin framework. The parallelization of the multirate strategy is challenging because grid cells have different workloads. The computational cost is different for each sub-time step depending on the elements involved and a classical partitioning strategy is not adequate any more. In this paper, we propose a solution that makes use of multi-constraint mesh partitioning. It tends to minimize the inter-processor communications, while ensuring that the workload is almost equally shared by every computer core at every stage of the algorithm. Particular attention is given to the simplicity of the parallel multirate algorithm while minimizing computational and communication overheads. Our implementation makes use of the MeTiS library for mesh partitioning and the Message Passing Interface for inter-processor communication. Performance analyses for two and three-dimensional practical applications confirm that multirate methods preserve important computational advantages of explicit methods up to a significant number of processors.

97 MATHEMATICS AND COMPUTING↗