High-Throughput Characterization of Porous Materials Using Graphics Processing Units
Explore the source record for details and available documents.
SEARCH · Engineering Papers
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
Explore the source record for details and available documents.
The subject of this chapter is the modeling and simulation of the flow through full wind farms, also known as wind plants, which are collections of many wind turbines within a region working together to supply power to the grid like any other type of power plant. This topic is interesting and exciting because of the increasing variety of and uses for wind plant flow modeling and simulation. For example, Figure 6.1 shows the computed flow field from a high-fidelity simulation of the Lillgrund offshore wind farm using computational fluid dynamics (CFD) with large-eddy simulation (LES) turbulence modeling and actuator line rotor aerodynamics models. There are many ways to model this type of flow, ranging from very simple models of wind turbine wakes and their interactions that can run in a matter of seconds on a laptop to high-fidelity, turbulence -resolving LES of the atmospheric flow and wakes that require days of use of thousands of cores on a high-performance computing cluster. The uses of wind plant flow simulations are equally varied. Wind plant layout designers run hundreds or thousands of simulations over all possible wind directions and speeds using simple models to predict a future wind plant's annual energy production. Researchers use LESs of the wind plant to study phenomena such as wind turbine wake interactions with each other and with the atmospheric boundary layer (ABL). As computational resources become more readily available, it is even becoming common for turbine manufacturers to use CFD of some form to model wind plants in complex terrain to help in the wind turbine siting and selection process. We are beginning to see the expensive, high-fidelity tools being used to create data and knowledge that improves or enables the creation of new cheaper, lower fidelity tools. We are also beginning to see the low- and high-fidelity tools used together to perform uncertainty quantification of energy estimates and optimization of wind plant layouts. As new computing technologies mature, such as graphics processing unit (GPU)-based computing, the future of full wind plant flow modeling is exciting.
For over a decade now, physical and energy constraints have limited clock speed improvements in commodity microprocessors. Instead, chipmakers have been pushed into producing lower-power, multi-core processors such as Graphical Processing Units (GPU), ARM CPUs, and Intel MICs. Broad-based efforts from manufacturers and developers have been devoted to making these processors user-friendly enough to perform general computations. However, extracting performance from a larger number of cores, as well as specialized vector or SIMD units, requires special care in algorithm design and code optimization. One of the most computationally challenging problems in high-energy particle experiments is finding and fitting the charged-particle tracks during event reconstruction. This is expected to become by far the dominant problem at the High-Luminosity Large Hadron Collider (HL-LHC), for example. Today the most common track finding methods are those based on the Kalman filter. Experience with Kalman techniques on real tracking detector systems has shown that they are robust and provide high physics performance. This is why they are currently in use at the LHC, both in the trigger and offine. Previously we reported on the significant parallel speedups that resulted from our investigations to adapt Kalman filters to track fitting and track building on Intel Xeon and Xeon Phi. Here, we discuss our progresses toward the understanding of these processors and the new developments to port the Kalman filter to NVIDIA GPUs.
The advent of computing resources with co-processors, for example Graphics Processing Units (GPU) or Field-Programmable Gate Arrays (FPGA), for use cases like the CMS High-Level Trigger (HLT) or data processing at leadership-class supercomputers imposes challenges for the current data processing frameworks. These challenges include developing a model for algorithms to offload their computations on the co-processors as well as keeping the traditional CPU busy doing other work. The CMS data processing framework, CMSSW, implements multithreading using the Intel Threading Building Blocks (TBB) library, that utilizes tasks as concurrent units of work. In this paper we will discuss a generic mechanism to interact effectively with non-CPU resources that has been implemented in CMSSW. In addition, configuring such a heterogeneous system is challenging. In CMSSW an application is configured with a configuration file written in the Python language. The algorithm types are part of the configuration. The challenge therefore is to unify the CPU and co-processor settings while allowing their implementations to be separate. We will explain how we solved these challenges while minimizing the necessary changes to the CMSSW framework. We will also discuss on a concrete example how algorithms would offload work to NVIDIA GPUs using directly the CUDA API.
The CMS experiment started to utilize Graphics Processing Units (GPU) to accelerate the online reconstruction and event selection running on its High Level Trigger (HLT) farm in the 2022 data taking period. The projections of the HLT farm to the High-Luminosity LHC foresee a significant use of compute accelerators in the LHC Run 4 and onwards in order to keep the cost, size, and power budget of the farm under control. This direction of leveraging compute accelerators has synergies with the increasing use of HPC resources in HEP computing, as HPC machines are employing more and more compute accelerators that are predominantly GPUs today. In this work we review the features developed for the CMS data processing framework, CMSSW, to support the effective utilization of both compute accelerators and many-core CPUs within a highly concurrent task-based framework. We measure the impact of various design choices for the scheduling of heterogeneous algorithms on the event processing throughput, using the Run-3 HLT application as a realistic use case.
Fast, digital signal processing (DSP) has many applications. Typical hardware options for performing DSP are field-programmable gate arrays (FPGAs), application-specific integrated DSP chips, or general purpose personal computer systems. This paper presents a novel DSP platform that has been developed for feedback control on the HBT-EP tokamak device. The system runs all signal processing exclusively on a Graphics Processing Unit (GPU) to achieve real-time performance with latencies below 8 μs. Signals are transferred into and out of the GPU using PCI Express peer-to-peer direct-memory-access transfers without involvement of the central processing unit or host memory. Tests were performed on the feedback control system of the HBT-EP tokamak using forty 16-bit floating point inputs and outputs each and a sampling rate of up to 250 kHz. Signals were digitized by a D-TACQ ACQ196 module, processing done on an NVIDIA GTX 580 GPU programmed in CUDA, and analog output was generated by D-TACQ AO32CPCI modules.
We present a low-prefactor, cubically scaling scaled-opposite-spin second-order Møller-Plesset perturbation theory (SOS-MP2) method which is highly suitable for massively parallel architectures like graphics processing units (GPU). The scaling is reduced from O(N{sup 5}) to O(N{sup 3}) by a reformulation of the MP2-expression in the atomic orbital basis via Laplace transformation and the resolution-of-the-identity (RI) approximation of the integrals in combination with efficient sparse algebra for the 3-center integral transformation. In contrast to previous works that employ GPUs for post Hartree-Fock calculations, we do not simply employ GPU-based linear algebra libraries to accelerate the conventional algorithm. Instead, our reformulation allows to replace the rate-determining contraction step with a modified J-engine algorithm, that has been proven to be highly efficient on GPUs. Thus, our SOS-MP2 scheme enables us to treat large molecular systems in an accurate and efficient manner on a single GPU-server.
The forces acting on the atoms as well as the stress tensor are crucial ingredients for calculating the structural and dynamical properties of systems in the condensed phase. Here, these derivatives of the total energy are evaluated for the second-order Møller-Plesset perturbation energy (MP2) in the framework of the resolution of identity Gaussian and plane waves method, in a way that is fully consistent with how the total energy is computed. This consistency is non-trivial, given the different ways employed to compute Coulomb, exchange, and canonical four center integrals, and allows, for example, for energy conserving dynamics in various ensembles. Based on this formalism, a massively parallel algorithm has been developed for finite and extended system. The designed parallel algorithm displays, with respect to the system size, cubic, quartic, and quintic requirements, respectively, for the memory, communication, and computation. All these requirements are reduced with an increasing number of processes, and the measured performance shows excellent parallel scalability and efficiency up to thousands of nodes. Additionally, the computationally more demanding quintic scaling steps can be accelerated by employing graphics processing units (GPU’s) showing, for large systems, a gain of almost a factor two compared to the standard central processing unit-only case. In this way, the evaluation of the derivatives of the RI-MP2 energy can be performed within a few minutes for systems containing hundreds of atoms and thousands of basis functions. With good time to solution, the implementation thus opens the possibility to perform molecular dynamics (MD) simulations in various ensembles (microcanonical ensemble and isobaric-isothermal ensemble) at the MP2 level of theory. Geometry optimization, full cell relaxation, and energy conserving MD simulations have been performed for a variety of molecular crystals including NH{sub 3}, CO{sub 2}, formic acid, and benzene.
The fully electromagnetic particle-in-cell code WarpX is being developed by a team of the U.S. DOE Exascale Computing Project (with additional non-U.S. collaborators on part of the code) to enable the modeling of chains of tens to hundreds of plasma accelerator stages on exascale supercomputers, for future collider designs. The code is combining the latest algorithmic advances (e.g., Lorentz boosted frame and pseudo-spectral Maxwell solvers) with mesh refinement and runs on the latest computer processing unit and graphical processing unit (GPU) architectures. In this paper, we summarize the strategy that was adopted to port WarpX to GPUs, report on the weak parallel scaling of the pseudo-spectral electromagnetic solver, and then present solutions for decreasing the time spent in data exchanges from guard regions between subdomains. In Sec. IV, we demonstrate the simulations of a chain of three consecutive multi-GeV laser-driven plasma accelerator stages.
We present a reduced scaling and exact reformulation of state specific complete active space second-order perturbation (CASPT2) analytical gradients in terms of the MP2 and Fock derivatives using the supporting subspace method. This work follows naturally from the supporting subspace formulation of the CASPT2 energy in terms of the MP2 energy using dressed orbitals and Fock builds. For a given active space configuration, the terms corresponding to the MP2-gradient can be evaluated with O(N5) operations, while the rest of the calculations can be computed with O(N3) operations using Fock builds, Fock gradients, and linear algebra. When tensor-hyper-contraction is applied simultaneously, the computational cost can be further reduced to O(N4) for a fixed active space size. The new formulation enables efficient implementation of CASPT2 analytical gradients by leveraging the existing graphical processing unit (GPU)-based MP2 and Fock routines. We present benchmark results that demonstrate the accuracy and performance of the new method. Example applications of the new method in ab initio molecular dynamics simulation and constrained geometry optimization are given.
The routine use of electronic structures in many chemical simulation applications calls for efficient and easy ways to access electronic structure programs. Here, we describe how the graphics processing unit (GPU) accelerated electronic structure program TeraChem can be set up as an electronic structure server, to be easily accessed by third-party client programs. We exploit Google’s protocol buffer framework for data serialization and communication. The client interface, called TeraChem protocol buffers (TCPB), has been designed for ease of use and compatibility with multiple programming languages, such as C++, Fortran, and Python. To demonstrate the ease of coupling third-party programs with electronic structures using TCPB, we have incorporated the TCPB client into Amber for quantum mechanics/molecular mechanics (QM/MM) simulations. The TCPB interface saves time with GPU initialization and I/O operations, achieving a speedup of more than 2× compared to a prior file-based implementation for a QM region with ~250 basis functions. We demonstrate the practical application of TCPB by computing the free energy profile of p-hydroxybenzylidene-2,3-dimethylimidazolinone (p-HBDI - )—a model chromophore in green fluorescent proteins—on the first excited singlet state using Hamiltonian replica exchange for enhanced sampling. All calculations in this work have been performed with the non-commercial freely-available version of TeraChem, which is sufficient for many QM region sizes in common use.
We review the GPAW open-source Python package for electronic structure calculations. GPAW is based on the projector-augmented wave method and can solve the self-consistent density functional theory (DFT) equations using three different wave-function representations, namely real-space grids, plane waves, and numerical atomic orbitals. The three representations are complementary and mutually independent and can be connected by transformations via the real-space grid. This multi-basis feature renders GPAW highly versatile and unique among similar codes. By virtue of its modular structure, the GPAW code constitutes an ideal platform for the implementation of new features and methodologies. Moreover, it is well integrated with the Atomic Simulation Environment (ASE), providing a flexible and dynamic user interface. In addition to ground-state DFT calculations, GPAW supports many-body GW band structures, optical excitations from the Bethe–Salpeter Equation, variational calculations of excited states in molecules and solids via direct optimization, and real-time propagation of the Kohn–Sham equations within time-dependent DFT. A range of more advanced methods to describe magnetic excitations and non-collinear magnetism in solids are also now available. In addition, GPAW can calculate non-linear optical tensors of solids, charged crystal point defects, and much more. Recently, support for graphics processing unit (GPU) acceleration has been achieved with minor modifications to the GPAW code thanks to the CuPy library. We end the review with an outlook, describing some future plans for GPAW.
The evolution of molecular dynamics (MD) simulations has been intimately linked to that of computing hardware. For decades following the creation of MD, simulations have improved with computing power along the three principal dimensions of accuracy, atom count (spatial scale), and duration (temporal scale). Since the mid-2000s, computer platforms have, however, failed to provide strong scaling for MD, as scale-out central processing unit (CPU) and graphics processing unit (GPU) platforms that provide substantial increases to spatial scale do not lead to proportional increases in temporal scale. Important scientific problems therefore remained inaccessible to direct simulation, prompting the development of increasingly sophisticated algorithms that present significant complexity, accuracy, and efficiency challenges. While bespoke MD-only hardware solutions have provided a path to longer timescales for specific physical systems, their impact on the broader community has been mitigated by their limited adaptability to new methods and potentials. In this work, we show that a novel computing architecture, the Cerebras wafer scale engine, completely alters the scaling path by delivering unprecedentedly high simulation rates up to 1.144 M steps/s for 200 000 atoms whose interactions are described by an embedded atom method potential. This enables direct simulations of the evolution of materials using general-purpose programmable hardware over millisecond timescales, dramatically increasing the space of direct MD simulations that can be carried out. In this paper, we provide an overview of advances in MD over the last 60 years and present our recent result in the context of historical MD performance trends.
Accurate and efficient gradients of molecular energy with respect to nuclear degrees of freedom are essential for geometry optimization and molecular dynamics, including simulations that go beyond the Born–Oppenheimer regime. A common approach involves deriving analytical formulas for new electronic structure methods, which is often conceptually difficult and requires tedious coding. Here, we implement analytical, semi-numerical, and automatic differentiation (AD)-based gradient pathways for semiempirical Hamiltonian models in the PYSEQM software package, leveraging both graphics processing unit (GPU) and central processing unit (CPU) architectures. We further extend these capabilities to excited states calculated using the configuration interaction singles and time-dependent Hartree–Fock ansätze. We benchmark wall time, peak memory usage, and accuracy across three molecular families of varying chemical complexity, including systems of up to a thousand atoms. For ground-state simulations, analytical and AD gradients achieve near-identical GPU runtimes, while semi-numerical gradients are slower on GPU but remain competitive on CPU. For excited states, both analytical and custom AD approaches using implicit differentiation show similar performance and low memory requirements, whereas gradients with full AD are memory-limited. AD gradients match analytical ones in accuracy across all tested systems, aided by a quaternion-based diatomic frame rotation for two-center quantities that ensures smooth energy surfaces. Overall, automatic differentiation emerges as a practical alternative to analytical gradients in semiempirical quantum chemistry, offering high accuracy while allowing seamless integration in AI-driven workflows and popular packages, such as PyTorch and JAX. Our results provide actionable guidance for selecting optimal gradient strategies in large-scale ground- and excited-state molecular dynamics simulations.
We present three new eclipsing white-dwarf/M-dwarf binary systems discovered during a search for transiting planets around M-dwarfs. Unlike most known eclipsing systems of this type, the optical and infrared emission is dominated by the M-dwarf components, and the systems have optical colors and discovery light curves consistent with being Jupiter-radius transiting planets around early M-dwarfs. We detail the PTF/M-dwarf transiting planet survey, part of the Palomar Transient Factory (PTF). We present a graphics processing unit (GPU)-based box-least-squares search for transits that runs approximately 8 Multiplication-Sign faster than similar algorithms implemented on general purpose systems. For the discovered systems, we decompose low-resolution spectra of the systems into white-dwarf and M-dwarf components, and use radial velocity measurements and cooling models to estimate masses and radii for the white dwarfs. The systems are compact, with periods between 0.35 and 0.45 days and semimajor axes of approximately 2 R{sub Sun} (0.01 AU). The M-dwarfs have masses of approximately 0.35 M{sub Sun }, and the white dwarfs have hydrogen-rich atmospheres with temperatures of around 8000 K and have masses of approximately 0.5 M{sub Sun }. We use the Robo-AO laser guide star adaptive optics system to tentatively identify one of the objects as a triple system. We also use high-cadence photometry to put an upper limit on the white-dwarf radius of 0.025 R{sub Sun} (95% confidence) in one of the systems. Accounting for our detection efficiency and geometric factors, we estimate that 0.08%{sub -0.05%}{sup +0.10%} (90% confidence) of M-dwarfs are in these short-period, post-common-envelope white-dwarf/M-dwarf binaries where the optical light is dominated by the M-dwarf. The lack of detections at shorter periods, despite near-100% detection efficiency for such systems, suggests that binaries including these relatively low-temperature white dwarfs are preferentially found at relatively large orbital radii. Similar eclipsing binary systems can have arbitrarily small eclipse depths in red bands and generate plausible small-planet-transit light curves. As such, these systems are a source of false positives for M-dwarf transiting planet searches. We present several ways to rapidly distinguish these binaries from transiting planet systems.
We present graphics processing unit (GPU) implementations of two fast force calculation methods based on series expansions of the Poisson equation. One method is the self-consistent field (SCF) method, which is a Fourier-like expansion of the density field in some basis set; the other method is the multipole expansion (MEX) method, which is a Taylor-like expansion of the Green's function. MEX, which has been advocated in the past, has not gained as much popularity as SCF. Both are particle-field methods and optimized for collisionless galactic dynamics, but while SCF is a 'pure' expansion, MEX is an expansion in just the angular part; thus, MEX is capable of capturing radial structure easily, while SCF needs a large number of radial terms. We show that despite the expansion bias, these methods are more accurate than direct techniques for the same number of particles. The performance of our GPU code, which we call ETICS, is profiled and compared to a CPU implementation. On the tested GPU hardware, a full force calculation for one million particles took ∼0.1 s (depending on expansion cutoff), making simulations with as many as 10{sup 8} particles fast for a comparatively small number of nodes.
Recent advances in general relativistic magnetohydrodynamic simulations have expanded and improved our understanding of the dynamics of black-hole accretion disks. However, current simulations do not capture the thermodynamics of electrons in the low density accreting plasma. This poses a significant challenge in predicting accretion flow images and spectra from first principles. Because of this, simplified emission models have often been used, with widely different configurations (e.g., disk- versus jet-dominated emission), and were able to account for the observed spectral properties of accreting black holes. Exploring the large parameter space introduced by such models, however, requires significant computational power that exceeds conventional computational facilities. In this paper, we use GRay, a fast graphics processing unit (GPU) based ray-tracing algorithm, on the GPU cluster El Gato, to compute images and spectra for a set of six general relativistic magnetohydrodynamic simulations with different magnetic field configurations and black-hole spins. We also employ two different parametric models for the plasma thermodynamics in each of the simulations. We show that, if only the spectral properties of Sgr A* are used, all 12 models tested here can fit the spectra equally well. However, when combined with the measurement of the image size of the emission using the Event Horizon Telescope, current observations rule out all models with strong funnel emission, because the funnels are typically very extended. Our study shows that images of accretion flows with horizon-scale resolution offer a powerful tool in understanding accretion flows around black holes and their thermodynamic properties.
QMCPACK is an open source quantum Monte Carlo package for ab-initio electronic structure calculations. It supports calculations of metallic and insulating solids, molecules, atoms, and some model Hamiltonians. Implemented real space quantum Monte Carlo algorithms include variational, diffusion, and reptation Monte Carlo. QMCPACK uses Slater-Jastrow type trial wave functions in conjunction with a sophisticated optimizer capable of optimizing tens of thousands of parameters. The orbital space auxiliary field quantum Monte Carlo method is also implemented, enabling cross validation between different highly accurate methods. The code is specifically optimized for calculations with large numbers of electrons on the latest high performance computing architectures, including multicore central processing unit (CPU) and graphical processing unit (GPU) systems. We detail the program’s capabilities, outline its structure, and give examples of its use in current research calculations. The package is available at http://www.qmcpack.org.