Parallel Large Eddy Simulation in a Finite Element Projection Method for all Flow Regimes
Explore the source record for details and available documents.
SEARCH · Engineering Papers
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
Explore the source record for details and available documents.
The Resolution of the Identity second-order Møller-Plesset perturbation theory (RI-MP2) method is implemented within the linear-scaling Divide-Expand-Consolidate (DEC) framework. In a DEC calculation, the full molecular correlated calculation is replaced by a set of independent fragment calculations each using a subset of the total orbital space. The number of independent fragment calculations scales linearly with the system size, rendering the method linear-scaling and massively parallel. The DEC-RI-MP2 method can be viewed as an approximation to the DEC-MP2 method where the RI approximation is utilized in each fragment calculation. The individual fragment calculations scale with the fifth power of the fragment size for both methods. However, the DEC-RI-MP2 method has a reduced prefactor compared to DEC-MP2 and is well-suited for implementation on massively parallel supercomputers, as demonstrated by test calculations on a set of medium-sized molecules. The DEC error control ensures that the standard RI-MP2 energy can be obtained to the predefined precision. The errors associated with the RI and DEC approximations are compared, and it is shown that the DEC-RI-MP2 method can be applied to systems far beyond the ones that can be treated with a conventional RI-MP2 implementation.
The present conference on computational methods for aeronautics applications discusses topics in the fields of parallel computing, multidisciplinary computational methods, grid generation, visualization methods for CFD, probabilistic modeling, numerical simulations and methodologies for different flow regimes, computational strategies and adaptive methods in CFD, and computational strategies and dynamics and control. Attention is given to the MACH system-software kernel, a multidisciplinary approach to aeroelastic analysis, interactive grid generation with control points, interactive flow visualization using stream surfaces, numerical simulations of dynamic/aerodynamic interactions, implicit mathods for the Navier-Stokes equations, and the automatic phase-space analysis of dynamical systems.
Purpose: Integrating a linac with an MRI system would allow for real time tumour tracking however the patient will be irradiated in the presence of a magnetic field. The present study experimentally investigates the magnetic field effects on entrance, exit, and interface dose for both transverse and parallel magnetic fields. Methods: Polystyrene was used to construct a set of phantoms for Gafchromic film measurements. One phantom had an adjustable air gap and four other phantoms had one surface at various angles. The linac-MR prototype consisting of a biplanar permanent magnet coupled to a linac was used for the transverse magnetic field measurements. A couple of solenoid electromagnets, stacked on top of each other and irradiated along their bore, were used for the parallel field measurements. Results: All doses are relative to no magnetic field. The transverse magnetic field reduced the entrance dose for all surface angles by strongly deflecting the contaminant electrons. The exit dose in a transverse magnetic field was found to be significantly higher. The entrance dose with a parallel magnetic field present is higher due to the contaminant electrons being concentrated within the beam area. The air gap phantom measurements, done in a transverse magnetic field, show a significant increase of the dose at the proximal side of the air gap and a decrease at the distal side. The measurements, done in the parallel magnetic field, show the concentration of secondary electrons in the air gap. Conclusion: The radiation dose measurements of a 6MV beam in a parallel and transverse magnetic field presented here are currently being replicated using Monte Carlo simulations. This verified Monte Carlo system could provide the dose calculation basis for future linac-MR systems.
A scalable stochastic algorithm is presented that can evaluate explicitly correlated (F12) second-order many-body perturbation (MP2) energies of weak, noncovalent, intermolecular interactions. It first transforms the formulas of the MP2 and F12 energy differences into a short sum of high-dimensional integrals of Green’s functions in real space and imaginary time. Furthermore, these integrals are then evaluated by the Monte Carlo method augmented by parallel execution, redundant-walker convergence acceleration, direct-sampling autocorrelation elimination, and control-variate error reduction. By sharing electron-pair walkers across the supermolecule and its subsystems spanned by the joint basis set, the statistical uncertainty is reduced by one to 2 orders of magnitude in the MP2 binding energy corrected for the basis-set incompleteness and superposition errors. The method predicts the MP2-F12/aug-cc-pVDZ binding energy of 19.1 ± 4.0 kcal mol –1 for the C 60 dimer at the center distance of 9.748 Å.
The work deals with the Particle in Cell/Monte Carlo Collision (PIC/MCC) analysis of the problem of detection and identification of impurities in the nonlocal plasma of gas discharge using the Plasma Electron Spectroscopy (PLES) method. For this purpose, 1d3v PIC/MCC code for numerical simulation of glow discharge with nonlocal electron energy distribution function is developed. The elastic, excitation, and ionization collisions between electron-neutral pairs and isotropic scattering and charge exchange collisions between ion-neutral pairs and Penning ionizations are taken into account. Applicability of the numerical code is verified under the Radio-Frequency capacitively coupled discharge conditions. The efficiency of the code is increased by its parallelization using Open Message Passing Interface. As a demonstration of the PLES method, parallel PIC/MCC code is applied to the direct current glow discharge in helium doped with a small amount of argon. Numerical results are consistent with the theoretical analysis of formation of nonlocal EEDF and existing experimental data.
Primal and dual block coordinate descent methods are iterative methods for solving regularized and unregularized optimization problems. Distributed-memory parallel implementations of these methods have become popular in analyzing large machine learning datasets. However, existing implementations communicate at every iteration, which, on modern data center and supercomputing architectures, often dominates the cost of floating-point computation. Recent results on communication-avoiding Krylov subspace methods suggest that large speedups are possible by re-organizing iterative algorithms to avoid communication. We show how applying similar algorithmic transformations can lead to primal and dual block coordinate descent methods that only communicate every $s$ iterations---where $s$ is a tuning parameter---instead of every iteration for the regularized least-squares problem. We show that the communication-avoiding variants reduce the number of synchronizations by a factor of $s$ on distributed-memory parallel machines without altering the convergence rate and attain strong scaling speedups of up to $6.1\times$ over the “standard algorithm" on a Cray XC30 supercomputer.
By partitioning the electron density into subsystem contributions, the Frozen Density Embedding (FDE) formulation of subsystem Density Functional Theory (DFT) has recently emerged as a powerful tool for reducing the computational scaling of Kohn–Sham DFT. To date, however, FDE has been employed to molecular systems only. Periodic systems, such as metals, semiconductors, and other crystalline solids have been outside the applicability of FDE, mostly because of the lack of a periodic FDE implementation. To fill this gap, in this work we aim at extending FDE to treat subsystems of molecular and periodic character. This goal is achieved by a dual approach. On one side, the development of a theoretical framework for periodic subsystem DFT. On the other, the realization of the method into a parallel computer code. We find that periodic FDE is capable of reproducing total electron densities and (to a lesser extent) also interaction energies of molecular systems weakly interacting with metallic surfaces. In the pilot calculations considered, we find that FDE fails in those cases where there is appreciable density overlap between the subsystems. Conversely, we find FDE to be in semiquantitative agreement with Kohn–Sham DFT when the inter-subsystem density overlap is low. We also conclude that to make FDE a suitable method for describing molecular adsorption at surfaces, kinetic energy density functionals that go beyond the GGA level must be employed.
Collective buffering and data pattern solutions are provided for storage, retrieval, and/or analysis of data in a collective parallel processing environment. For example, a method can be provided for data storage in a collective parallel processing environment. The method comprises receiving data to be written for a plurality of collective processes within a collective parallel processing environment, extracting a data pattern for the data to be written for the plurality of collective processes, generating a representation describing the data pattern, and saving the data and the representation.
Problems of temperature behavior of specific heat are solved by the entropy simulation method for Ising models on a simple square lattice and a square spin ice (SSI) lattice with nearest neighbor interaction, models of hexagonal lattices with short-range (SR) dipole interaction, as well as with long-range (LR) dipole interaction and free boundary conditions, and models of spin quasilattices with finite interaction radius. It is established that systems of a finite number of Ising spins with LR dipole interaction can have unusual thermodynamic properties characterized by several specific-heat peaks in the absence of an external magnetic field. For a parallel multicanonical sampling method, optimal schemes are found empirically for partitioning the space of states into energy bands for Ising and SSI models, methods of concatenation and renormalization of histograms are discussed, and a flatness criterion of histograms is proposed. It is established that there is no phase transition in a model with nearest neighbor interaction on a hexagonal lattice, while the temperature behavior of specific heat exhibits singularity in the same model, in case of LR interaction. A spin quasilattice is found that exhibits a nonzero value of residual entropy.
We present a new method for mapping applications' MPI tasks to cores of a parallel computer such that applications' communication time is reduced. We address the case of sparse node allocation, where the nodes assigned to a job are not necessarily located in a contiguous block nor within close proximity to each other in the network, although our methods generalize to contiguous allocations as well. The goal is to assign tasks to cores so that interdependent tasks are performed by "nearby' cores, thus lowering the distance messages must travel, the amount of congestion in the network, and the overall cost of communication. Our new method applies a geometric partitioning algorithm to both the tasks and the processors, and assigns task parts to the corresponding processor parts. We also present a number of algorithmic optimizations that exploit specific features of the network or application. We show that, for the structured finite difference mini-application MiniGhost, our mapping methods reduced communication time up to 75% relative to MiniGhost's default mapping on 128K cores of a Cray XK7 with sparse allocation. For the atmospheric modeling code E3SM/HOMME, our methods reduced communication time up to 31% on 32K cores of an IBM BlueGene/Q with contiguous allocation.
The art and science of writing parallelized software, using methods such as Open Multi-Processing (OpenMP) and Open Accelerators (OpenACC), is dominated by computer scientists. Engineers and non-computer scientists looking to apply these techniques to their project applications face a steep learning curve, especially when looking to adapt their original single threaded software to run multi-threaded on graphics processing units (GPUs). There are significant changes in mindset that must occur; such as how to manage memory, the organization of instructions, and the use of if statements (also known as branching). The purpose of this work is twofold: 1) to demonstrate the applicability of parallelized coding methodologies, OpenMP and OpenACC, to tasks outside of the typical large scale matrix mathematics; and 2) to discuss, from an engineer’s perspective, the lessons learned from parallelizing software using these computer science techniques. This work applies OpenMP, on both multi-core central processing units (CPUs) and Intel® Xeon Phi™ 7210, and OpenACC on GPUs. These parallelization techniques are used to tackle the simulation of thousands of entry vehicle trajectories through the integration of six degree of freedom (DoF) equations of motion (EoM). The forces and moments acting on the entry vehicle, and used by the EoM, are estimated using multiple models of varying levels of complexity. Several benchmark comparisons are made on the execution of six DoF trajectory simulation: single thread Intel® Xeon® E5-2670 CPU, multi-thread CPU using OpenMP, multi-thread Xeon Phi™ 7210 using OpenMP, and multi-thread NVIDIA® Tesla® K40 GPU using OpenACC. These benchmarks are run on the Pleiades Supercomputer Cluster at the National Aeronautics and Space Administration (NASA) Ames Research Center (ARC), and a Xeon Phi™ 7210 node at NASA Langley Research Center (LaRC).
A computational aeroacoustics code based on the discontinuous Galerkin method is ported to several parallel platforms using MPI. The discontinuous Galerkin method is a compact high-order method that retains its accuracy and robustness on non-smooth unstructured meshes. In its semi-discrete form, the discontinuous Galerkin method can be combined with explicit time marching methods making it well suited to time accurate computations. The compact nature of the discontinuous Galerkin method also makes it well suited for distributed memory parallel platforms. The original serial code was written using an object-oriented approach and was previously optimized for cache-based machines. The port to parallel platforms was achieved simply by treating partition boundaries as a type of boundary condition. Code modifications were minimal because boundary conditions were abstractions in the original program. Scalability results are presented for the SCI Origin, IBM SP2, and clusters of SGI and Sun workstations. Slightly superlinear speedup is achieved on a fixed-size problem on the Origin, due to cache effects.
Methods for analyzing and improving a target computer application and corresponding systems and computer-readable mediums. A method includes receiving the target application. The method includes generating a parallel control flow graph (ParCFG) corresponding to the target application. The method includes analyzing the ParCFG by the computer system. The method includes generating and storing the modified ParCFG for the target application.
A scalable, stochastic algorithm evaluating the fourth-order many-body perturbation (MP4) correction to energy is proposed. Three hundred Goldstone diagrams representing the MP4 correction are computer generated and then converted into algebraic formulas expressed in terms of Green’s functions in real space and imaginary time. They are evaluated by the direct (i.e., non-Markov, non-Metropolis) Monte Carlo (MC) integration accelerated by the redundant-walker and control-variate algorithms. The resulting MC-MP4 method is efficiently parallelized and is shown to display O(n 5.3 ) size-dependence of cost, which is nearly two ranks lower than the O(n 7 ) dependence of the deterministic MP4 algorithm. Furthermore, it evaluates the MP4/aug-cc-pVDZ energy for benzene, naphthalene, phenanthrene, and corannulene with the statistical uncertainty of 10 mE h (1.1% of the total basis-set correlation energy), 38 mE h (2.6%), 110 mE h (5.5%), and 280 mE h (9.0%), respectively, after about 10 9 MC steps.
We consider the parallel-in-time solution of both linear and nonlinear hyperbolic partial differential equation (PDE) systems in one spatial dimension. In the nonlinear setting, the discretized equations are solved with a preconditioned residual iteration based on a global linearization. The linear(ized) equation systems are approximately solved parallel-in-time using a block preconditioner applied in the characteristic variables of the underlying linear(ized) hyperbolic PDE. This change of variables is motivated by the observation that intervariable coupling between characteristic variables is weak, at least locally where spatio-temporal variations in the eigenvectors of the associated flux Jacobian are sufficiently small, while that between the original variables is not. For an ℓ-dimensional system of PDEs, applying the preconditioner consists of solving a sequence of ℓ scalar linear(ized)-advection-like problems, each associated with a different characteristic wave-speed in the underlying linear(ized) PDE. Furthermore, we approximately solve these linear advection problems using multigrid reduction-in-time (MGRIT); however, any other suitable parallel-in-time method could be used. Numerical examples are shown for the (linear) acoustics equations in heterogeneous media and for the (nonlinear) shallow water equations and Euler equations of gas dynamics with shocks and rarefactions. For many test problems, the solver converges in just a handful of iterations and with mesh-independent convergence rates.
Large-scale multiphysics simulations are computationally challenging due to the coupling of multiple processes with widely disparate time scales. The advent of exascale computing systems exacerbates these challenges since these systems enable ever-increasing size and complexity. In recent years, there has been renewed interest in developing multirate methods as a means to handle the large range of time scales, as these methods may afford greater accuracy and efficiency than more traditional approaches of using implicit-explicit (IMEX) and low-order operator splitting schemes. However, to date there have been few performance studies that compare different classes of multirate integrators on complex application problems. In this work, we study the performance of several newly developed multirate infinitesimal (MRI) methods, implemented in the SUNDIALS solver package, on two reacting flow model problems built on structured mesh frameworks. The first model revisits prior work on a compressible reacting flow problem with complex chemistry that is implemented using BoxLib but where we now include comparisons between a new explicit MRI scheme with the multirate spectral deferred correction (SDC) methods in the original paper. The second problem uses the same complex chemistry as the first problem, combined with a simplified flow model, but runs at a large spatial scale where explicit methods become infeasible due to stability constraints. Two recently developed IMEX MRI multirate methods are tested. These methods rely on advanced features of the AMReX framework on which the model is built, such as multilevel grids and multilevel preconditioners. The results from these two problems show that MRI multirate methods can offer significant performance benefits on complex multiphysics application problems and that these methods may be combined with advanced spatial discretization to compound the advantages of both.
In this study, we reported the development of a new quantum mechanics/molecular mechanics (QM/MM)-type framework to describe chemical processes in solution by combining standard molecular-orbital calculations with a three-dimensional formalism of integral equation theory for molecular liquids (multi-center molecular Ornstein–Zernike (MC-MOZ) method). The theoretical procedure is very similar to the 3D-reference interaction site model self-consistent field (RISM-SCF) approach. Since the MC-MOZ method is highly parallelized for computation, the present approach has the potential to be one of the most efficient procedures to treat chemical processes in solution. Benchmark tests to check the validity of this approach were performed for two solute (solute water and formaldehyde) systems and a simple S{sub N}2 reaction (Cl{sup −} + CH{sub 3}Cl → ClCH{sub 3} + Cl{sup −}) in aqueous solution. The results for solute molecular properties and solvation structures obtained by the present approach were in reasonable agreement with those obtained by other hybrid frameworks and experiments. In particular, the results of the proposed approach are in excellent agreements with those of 3D-RISM-SCF.