A direct-compatible formulation of the coupled perturbed complete active space self-consistent field equations on graphical processing units
Explore the source record for details and available documents.
SEARCH · Engineering Papers
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
Explore the source record for details and available documents.
Not provided.
Block-structured adaptive mesh refinement (AMR) is a technique that can be used when solving partial differential equations to reduce the number of cells necessary to achieve the required accuracy in areas of interest. These areas (shock fronts, material interfaces, etc.) are recursively covered with finer mesh patches that are grouped into a hierarchy of refinement levels. Despite the potential for large savings in computational requirements and memory usage without a corresponding reduction in accuracy, AMR adds overhead in managing the mesh hierarchy, adding complex communication and data movement requirements to a simulation. In this paper, we describe the design and implementation of a resident GPU-based AMR library, including: the classes used to manage data on a mesh patch, the routines used for transferring data between GPUs on different nodes, and the data-parallel operators developed to coarsen and refine mesh data. We validate the performance and accuracy of our implementation using three test problems and two architectures: an 8 node cluster, and 4,196 nodes of Oak Ridge National Laboratory's Titan supercomputer. Our GPU-based AMR hydrodynamics code performs up to 4.87× faster than the CPU-based implementation, and is scalable on 4,196 K20x GPUs using a combination of MPI and CUDA.
This is a general purpose math library implementing basic tensor algebra operations on NVidia GPU accelerators. This software is a tensor algebra library that can perform basic tensor algebra operations, including tensor contractions, tensor products, tensor additions, etc., on NVidia GPU accelerators, asynchronously with respect to the CPU host. It supports a simultaneous use of multiple NVidia GPUs. Each asynchronous API function returns a handle which can later be used for querying the completion of the corresponding tensor algebra operation on a specific GPU. The tensors participating in a particular tensor operation are assumed to be stored in local RAM of a node or GPU RAM. The main research area where this library can be utilized is the quantum many-body theory (e.g., in electronic structure theory).
We present the GPU implementation efforts and challenges of the sparse solver package STRUMPACK. The code is made publicly available on github with a permissive BSD license. STRUMPACK implements an approximate multifrontal solver, a sparse LU factorization which makes use of compression methods to accelerate time to solution and reduce memory usage. Multiple compression schemes based on rank-structured and hierarchical matrix approximations are supported, including hierarchically semi-separable, hierarchically off-diagonal butterfly, and block low rank. Here, in this paper, we present the GPU implementation of the block low rank (BLR) compression method within a multifrontal solver. Our GPU implementation relies on highly optimized vendor libraries such as cuBLAS and cuSOLVER for NVIDIA GPUs, rocBLAS and rocSOLVER for AMD GPUs and the Intel oneAPI Math Kernel Library (oneMKL) for Intel GPUs. Additionally, we rely on external open source libraries such as SLATE (Software for Linear Algebra Targeting Exascale), MAGMA (Matrix Algebra on GPU and Multi-core Architectures), and KBLAS (KAUST BLAS). SLATE is used as a GPU-capable ScaLAPACK replacement. From MAGMA we use variable sized batched dense linear algebra operations such as GEMM, TRSM and LU with partial pivoting. KBLAS provides efficient (batched) low rank matrix compression for NVIDIA GPUs using an adaptive randomized sampling scheme. The resulting sparse solver and preconditioner runs on NVIDIA, AMD and Intel GPUs. Interfaces are available from PETSc, Trilinos and MFEM, or the solver can be used directly in user code. We report results for a range of benchmark applications, using the Perlmutter system from NERSC, Frontier from ORNL, and Aurora from ALCF. For a high frequency wave equation on a regular mesh, using 32 Perlmutter compute nodes, the factorization phase of the exact GPU solver is about 6.5× faster compared to the CPU-only solver. The BLR-enabled GPU solver is about 13.8× faster than the CPU exact solver. For a collection of SuiteSparse matrices, the STRUMPACK exact factorization on a single GPU is on average 1.9× faster than NVIDIA’s cuDSS solver.
Block-structured adaptive mesh refinement is a technique that can be used when solving partial differential equations to reduce the number of zones necessary to achieve the required accuracy in areas of interest. These areas (shock fronts, material interfaces, etc.) are recursively covered with finer mesh patches that are grouped into a hierarchy of refinement levels. Despite the potential for large savings in computational requirements and memory usage without a corresponding reduction in accuracy, AMR adds overhead in managing the mesh hierarchy, adding complex communication and data movement requirements to a simulation. In this paper, we describe the design and implementation of a native GPU-based AMR library, including: the classes used to manage data on a mesh patch, the routines used for transferring data between GPUs on different nodes, and the data-parallel operators developed to coarsen and refine mesh data. We validate the performance and accuracy of our implementation using three test problems and two architectures: an eight-node cluster, and over four thousand nodes of Oak Ridge National Laboratory’s Titan supercomputer. Our GPU-based AMR hydrodynamics code performs up to 4.87× faster than the CPU-based implementation, and has been scaled to over four thousand GPUs using a combination of MPI and CUDA.
Earth system models like the Goddard Earth Observing System model (GEOS-5) have been pushing the limits of large clusters of multi-core microprocessors, producing breath-taking fidelity in resolving cloud systems at a global scale. GPU computing presents an opportunity for improving the efficiency of these leading edge models. A GPU implementation of GEOS-5 will facilitate the use of cloud-system resolving resolutions in data assimilation and weather prediction, at resolutions near 3.5 km, improving our ability to extract detailed information from high-resolution satellite observations and ultimately produce better weather and climate predictions
Abstract not provided
A system of interconnected chips comprising a multi-chip module (MCM) includes a processor chip, a system functions chip, and an MCM package configured to include the processor chip, the system functions chip, and an interconnect circuit. The processor chip is configured to include a first ground-referenced single-ended signaling interface circuit. A first set of electrical traces manufactured within the MCM package and configured to couple the first single-ended signaling interface circuit to the interconnect circuit. The system functions chip is configured to include a second single-ended signaling interface circuit and a host interface. A second set of electrical traces manufactured within the MCM package and configured to couple the host interface to at least one external pin of the MCM package. In one embodiment, each single-ended signaling interface advantageously implements ground-referenced single-ended signaling.
A compute unit configured to execute multiple threads in parallel is presented. The compute unit includes one or more single instruction multiple data (SIMD) units and a fetch and decode logic. The SIMD units have differing numbers of arithmetic logic units (ALUs), such that each SIMD unit can execute a different number of threads. The fetch and decode logic is in communication with each of the SIMD units, and is configured to assign the threads to the SIMD units for execution based on such differing numbers of ALUs.
A processor core is configured to execute a parent task that is described by a data structure stored in a memory. A coprocessor is configured to dispatch a child task to the at least one processor core in response to the coprocessor receiving a request from the parent task concurrently with the parent task executing on the at least one processor core. In some cases, the parent task registers the child task in a task pool and the child task is a future task that is configured to monitor a completion object and enqueue another task associated with the future task in response to detecting the completion object. The future task is configured to self-enqueue by adding a continuation future task to a continuation queue for subsequent execution in response to the future task failing to detect the completion object.
A processor core is configured to execute a parent task that is described by a data structure stored in a memory. A coprocessor is configured to dispatch a child task to the at least one processor core in response to the coprocessor receiving a request from the parent task concurrently with the parent task executing on the at least one processor core. In some cases, the parent task registers the child task in a task pool and the child task is a future task that is configured to monitor a completion object and enqueue another task associated with the future task in response to detecting the completion object. The future task is configured to self-enqueue by adding a continuation future task to a continuation queue for subsequent execution in response to the future task failing to detect the completion object.
New heavy vector bosons, also known as ``heavy photons'', ``dark photons'', or ``hidden sector photons'', are expected on very general theoretical grounds. Recent astrophysical evidence motivates the direct search for a heavy photon $A'$ in the mass range $m_{A'} \sim 20$ to 1000 MeV/c$^{2}$, with coupling to ordinary photons {\it via} a kinetic mixing term $\propto\epsilon$, expected around $10^{-5}$ to $10^{-2}$. Such a new dark force gauge boson below the GeV range would naturally mediate TeV range dark matter annihilation, and interaction with ordinary matter. Existing constraints from collider searches and beam dump experiments leave much of this preferred $(m_{A'},\epsilon)$ phase space unexplored. The detection of electroproduced $A'$ on a heavy target from their decay into $e^{+}e^{-}$ pairs comes with a copious QED trident background. Using the high luminosity and precision beam available from Jlab, and combining a silicon microstrip vertex tracker with a lead-tungstate calorimeter for triggering, the HPS experiment will hunt for the $A'$ in a large $\epsilon$ range, with unique sensitivity in the lower coupling range using vertexing to suppress the trident background. In this talk, the experimental setup and goals of the HPS experiment will be presented.
The fully electromagnetic particle-in-cell code WarpX is being developed by a team of the U.S. DOE Exascale Computing Project (with additional non-U.S. collaborators on part of the code) to enable the modeling of chains of tens to hundreds of plasma accelerator stages on exascale supercomputers, for future collider designs. The code is combining the latest algorithmic advances (e.g., Lorentz boosted frame and pseudo-spectral Maxwell solvers) with mesh refinement and runs on the latest computer processing unit and graphical processing unit (GPU) architectures. In this paper, we summarize the strategy that was adopted to port WarpX to GPUs, report on the weak parallel scaling of the pseudo-spectral electromagnetic solver, and then present solutions for decreasing the time spent in data exchanges from guard regions between subdomains. In Sec. IV, we demonstrate the simulations of a chain of three consecutive multi-GeV laser-driven plasma accelerator stages.
Quantum mechanical calculations for material modeling using Kohn–Sham density functional theory (DFT) involve the solution of a nonlinear eigenvalue problem for N smallest eigenvector-eigenvalue pairs, with N proportional to the number of electrons in the material system. Here, these calculations are computationally demanding and have asymptotic cubic scaling complexity with the number of electrons. Large-scale matrix eigenvalue problems arising from the discretization of the Kohn–Sham DFT equations employing a systematically convergent basis traditionally rely on iterative orthogonal projection methods, which are shown to be computationally efficient and scalable on massively parallel computing architectures. However, as the size of the material system increases, these methods are known to incur dominant computational costs through the Rayleigh–Ritz projection step of the discretized Kohn–Sham Hamiltonian matrix and the subsequent subspace diagonalization of the projected matrix. This work explores the potential of polynomial expansion approaches based on recursive Fermi-operator expansion as an alternative to the subspace diagonalization of the projected Hamiltonian matrix to reduce the computational cost. Subsequently, we perform a detailed comparison of various recursive polynomial expansion approaches to the traditional approach of explicit diagonalization on both multi-node central processing unit and graphics processing unit architectures and assess their relative performance in terms of accuracy, computational efficiency, scaling behavior, and energy efficiency.
We report on the use of the adaptively compressed exchange (ACE) operator to accelerate many-body perturbation theory (MBPT) calculations, including G 0 W 0 and the Bethe–Salpeter equation (BSE), for hybrid density functional theory starting points. We show that by approximating the exact exchange operator with the low-rank ACE operator, substantial computational savings can be achieved with systematically controllable errors in the quasiparticle energies computed with full-frequency G 0 W 0 and the optical absorption spectra and vertical excitation energies computed by solving the BSE within density matrix perturbation theory. Our implementation makes use of the ACE-accelerated electronic Hamiltonian to carry out both G 0 W 0 and BSE without explicitly computing empty states. We show the robustness of the approach and present the computational gains obtained on both the central processing unit and graphics processing unit nodes. In conclusion, our work will facilitate the exploration and evaluation of fine-tuned hybrid starting points aimed at enhancing the accuracy of MBPT calculations without involving computationally demanding self-consistency in Hedin’s equations.
In recent years, there has been an increasing interest in space-qualified processors such as multi-core central processing units and graphics processing units that can withstand the adverse effects of space radiation. These processors can allow parallel programming to perform tasks that typically demand high computational power. One can study guidance schemes that can take advantage of these currently developing processors and provide more robust guidance. Software for Multi-model Autonomous Real-time Trajectories (SMART) guidance can identify robust trajectories by running an onboard Monte Carlo analysis. SMART guidance can take advantage of knowledge updates obtained from the onboard sensors, allowing it to consider the off-nominal cases that it would not typically encounter during the offline trajectory analysis. This work uses the SMART guidance for the powered divert at Mars simulation in Program to Optimize and Simulated Trajectories- II.
In recent years, there has been an increasing interest in space-qualified processors such as multi-core central processing units and graphics processing units that can withstand the adverse effects of space radiation. These processors can allow parallel programming to perform tasks that typically demand high computational power. One can study guidance schemes that can take advantage of these currently developing processors and provide more robust guidance. Software for Multi-model Autonomous Real-time Trajectories (SMART) guidance can identify robust trajectories by running an onboard Monte Carlo analysis. SMART guidance can take advantage of knowledge updates obtained from the onboard sensors, allowing it to consider the off-nominal cases that it would not typically encounter during the offline trajectory analysis. This work uses the SMART guidance for the powered divert at Mars simulation in Program to Optimize and Simulated Trajectories- II.