Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “supercomputer”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,369 records · Page 76

HEPCloud Operations at Fermilab—The First Five Years

The HEPCloud Facility at Fermilab has now been in production operation for five years. This facility is a unified provisioning gateway to US high performance computing centers, including NERSC, OLCF, and ALCF, other large supercomputers run by the NSF, and commercial clouds. HEPCloud delivers hundreds of millions of core-hours yearly for CMS. HEPCloud also serves other Fermilab experiments including DUNE, Mu2e, Muon g-2, and NOvA. In this paper we present the practical considerations of operating a distributed facility such as HEPCloud. We also mention some of the interesting research and development that HEPCloud has been used for including GPU-based machine learning inference servers, and tests of Quantum Computing.

Timm, Steven [Fermilab]↗

Towards district scale flood simulations using conventional and anisotropic porosity shallow water models with high-resolution topographic information

Current topographic survey technology provides high-resolution (HR) datasets for urban environments. Incorporating this HR information in models aiming to provide flood risk assessment is desirable because the flood wave propagation is depending on the urban topographic features, i.e. buildings, bridges and street networks. Conceptual, numerical and practical challenges arise from the application of shallow water models to HR urban flood modeling. For example, numerical challenges are occurrence of wet-dry fronts, geometric discontinuities in the urban environment and discontinuous solutions, i.e. shock waves. These challenges can be overcome by using a Godunov-type scheme. However, the computational cost of this type of schemes is high, such that HR two-dimensional shallow water simulations with practical relevance have to be run on supercomputers. The porous shallow water model is an alternative approach that aims to reduce computational cost by using a coarse resolution and accounting for unresolved processes by means of the porosity terms. Usually, a speedup between two and three orders of magnitude in comparison to HR simulations can be obtained. This study presents preliminary results of a practical test case concerning pluvial flooding in a district of the city of Nice, France, caused by the intense rainfall event on October 3rd, 2015. HR topography data set on a 1 m resolution is available for the district, whereby street features of infra-metric dimensions have been included. A reference solution is calculated by a HR shallow water model on a 1 m by 1 m structured computational grid. The porous shallow water model is run on a 10 m by 10 m grid and the influence of the drag source term is studied. The model calculations show a large deviation, which is caused by the poor meshing strategy of the porous shallow water (AP) model. The study also summarizes practical challenges that arise during the application of the AP and HR models to a large urban catchment. The main difficulty is to obtain a good mesh. In smaller scale investigations, the mesh is currently constructed by hand such that the cell edges align with buildings. This approach is not feasible for large scale urban catchments with a large number of buildings. Future steps that have to be taken, such as a strategy for automatic mesh generation, are reported on.

42 ENGINEERING↗

How large can the electron to proton mass ratio be in particle-in-cell simulations of unstable systems?

Particle-in-cell simulations are widely used as a tool to investigate instabilities that develop between a collisionless plasma and beams of charged particles. However, even on contemporary supercomputers, it is not always possible to resolve the ion dynamics in more than one spatial dimension with such simulations. The ion mass is thus reduced below 1836 electron masses, which can affect the plasma dynamics during the initial exponential growth phase of the instability and during the subsequent nonlinear saturation. The goal of this article is to assess how far the electron to ion mass ratio can be increased, without changing qualitatively the physics. It is first demonstrated that there can be no exact similarity law, which balances a change in the mass ratio with that of another plasma parameter, leaving the physics unchanged. Restricting then the analysis to the linear phase, a criterion allowing to define a maximum ratio is explicated in terms of the hierarchy of the linear unstable modes. The criterion is applied to the case of a relativistic electron beam crossing an unmagnetized electron-ion plasma.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗

Modeling laser wakefield accelerator experiments with ultrafast particle-in-cell simulations in boosted frames

The development of new laser systems at the 10 Petawatt range will push laser wakefield accelerators to novel regimes, for which theoretical scalings predict the possibility to accelerate electron bunches up to tens of GeVs in meter-scale plasmas. Numerical simulations will play a crucial role in testing, probing, and optimizing the physical parameters and the setup of future experiments. Fully kinetic simulations are computationally very demanding, pushing the limits of today's supercomputers. In this paper, the recent developments in the OSIRIS framework [R. A. Fonseca et al., Lect. Notes Comput. Sci. 2331, 342 (2002)] are described, in particular the boosted frame scheme, which leads to a dramatic change in the computational resources required to model laser wakefield accelerators. Results from one-to-one modeling of the next generation of laser systems are discussed, including the confirmation of electron bunch acceleration to the energy frontier.

43 PARTICLE ACCELERATORS↗

Atomic-Orbital Close-Coupling Calculations Of Electron Capture From Hydrogen Atoms Into Highly Excited Rydberg States Of Multiply Charged Ions

Collisions of neutral hydrogen atoms with multiply charged ions have been studied in the past using the semi-classical atomic-orbital close-coupling method. We present total and state-resolved cross sections for charge exchange as well as ionization. The advent of supercomputers and parallel programming facilities now allow treatment of collision systems that have been out of reach before, because much larger basis sets involving high quantum numbers are now feasible.

74 ATOMIC AND MOLECULAR PHYSICS↗

Spacecraft charging analysis with the implicit particle-in-cell code iPic3D

We present the first results on the analysis of spacecraft charging with the implicit particle-in-cell code iPic3D, designed for running on massively parallel supercomputers. The numerical algorithm is presented, highlighting the implementation of the electrostatic solver and the immersed boundary algorithm; the latter which creates the possibility to handle complex spacecraft geometries. As a first step in the verification process, a comparison is made between the floating potential obtained with iPic3D and with Orbital Motion Limited theory for a spherical particle in a uniform stationary plasma. Second, the numerical model is verified for a CubeSat benchmark by comparing simulation results with those of PTetra for space environment conditions with increasing levels of complexity. In particular, we consider spacecraft charging from plasma particle collection, photoelectron and secondary electron emission. The influence of a background magnetic field on the floating potential profile near the spacecraft is also considered. Although the numerical approaches in iPic3D and PTetra are rather different, good agreement is found between the two models, raising the level of confidence in both codes to predict and evaluate the complex plasma environment around spacecraft.

42 ENGINEERING↗

A divide-conquer-recombine algorithmic paradigm for large spatiotemporal quantum molecular dynamics simulations

We introduce an extension of the divide-and-conquer (DC) algorithmic paradigm called divide-conquer-recombine (DCR) to perform large quantum molecular dynamics (QMD) simulations on massively parallel supercomputers, in which interatomic forces are computed quantum mechanically in the framework of density functional theory (DFT). In DCR, the DC phase constructs globally informed, overlapping local-domain solutions, which in the recombine phase are synthesized into a global solution encompassing large spatiotemporal scales. For the DC phase, we design a lean divide-and-conquer (LDC) DFT algorithm, which significantly reduces the prefactor of the O(N) computational cost for N electrons by applying a density-adaptive boundary condition at the peripheries of the DC domains. Our globally scalable and locally efficient solver is based on a hybrid real-reciprocal space approach that combines: (1) a highly scalable real-space multigrid to represent the global charge density; and (2) a numerically efficient plane-wave basis for local electronic wave functions and charge density within each domain. Hybrid space-band decomposition is used to implement the LDC-DFT algorithm on parallel computers. A benchmark test on an IBM Blue Gene/Q computer exhibits an isogranular parallel efficiency of 0.984 on 786 432 cores for a 50.3 × 10{sup 6}-atom SiC system. As a test of production runs, LDC-DFT-based QMD simulation involving 16 661 atoms is performed on the Blue Gene/Q to study on-demand production of hydrogen gas from water using LiAl alloy particles. As an example of the recombine phase, LDC-DFT electronic structures are used as a basis set to describe global photoexcitation dynamics with nonadiabatic QMD (NAQMD) and kinetic Monte Carlo (KMC) methods. The NAQMD simulations are based on the linear response time-dependent density functional theory to describe electronic excited states and a surface-hopping approach to describe transitions between the excited states. A series of techniques are employed for efficiently calculating the long-range exact exchange correction and excited-state forces. The NAQMD trajectories are analyzed to extract the rates of various excitonic processes, which are then used in KMC simulation to study the dynamics of the global exciton flow network. This has allowed the study of large-scale photoexcitation dynamics in 6400-atom amorphous molecular solid, reaching the experimental time scales.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

Modeling of the charge-state separation at ITEP experimental facility for material science based on a Bernas ion source

The experiment automation system is supposed to be developed for experimental facility for material science at ITEP, based on a Bernas ion source. The program CAMFT is assumed to be involved into the program of the experiment automation. CAMFT is developed to simulate the intense charged particle bunch motion in the external magnetic fields with arbitrary geometry by means of the accurate solution of the particle motion equation. Program allows the consideration of the bunch intensity up to 10{sup 10} ppb. Preliminary calculations are performed at ITEP supercomputer. The results of the simulation of the beam pre-acceleration and following turn in magnetic field are presented for different initial conditions.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗

Efficient linear-scaling second-order Møller-Plesset perturbation theory: The divide–expand–consolidate RI-MP2 model

The Resolution of the Identity second-order Møller-Plesset perturbation theory (RI-MP2) method is implemented within the linear-scaling Divide-Expand-Consolidate (DEC) framework. In a DEC calculation, the full molecular correlated calculation is replaced by a set of independent fragment calculations each using a subset of the total orbital space. The number of independent fragment calculations scales linearly with the system size, rendering the method linear-scaling and massively parallel. The DEC-RI-MP2 method can be viewed as an approximation to the DEC-MP2 method where the RI approximation is utilized in each fragment calculation. The individual fragment calculations scale with the fifth power of the fragment size for both methods. However, the DEC-RI-MP2 method has a reduced prefactor compared to DEC-MP2 and is well-suited for implementation on massively parallel supercomputers, as demonstrated by test calculations on a set of medium-sized molecules. The DEC error control ensures that the standard RI-MP2 energy can be obtained to the predefined precision. The errors associated with the RI and DEC approximations are compared, and it is shown that the DEC-RI-MP2 method can be applied to systems far beyond the ones that can be treated with a conventional RI-MP2 implementation.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

Accurate reaction-diffusion operator splitting on tetrahedral meshes for parallel stochastic molecular simulations

Spatial stochastic molecular simulations in biology are limited by the intense computation required to track molecules in space either in a discrete time or discrete space framework, which has led to the development of parallel methods that can take advantage of the power of modern supercomputers in recent years. We systematically test suggested components of stochastic reaction-diffusion operator splitting in the literature and discuss their effects on accuracy. We introduce an operator splitting implementation for irregular meshes that enhances accuracy with minimal performance cost. We test a range of models in small-scale MPI simulations from simple diffusion models to realistic biological models and find that multi-dimensional geometry partitioning is an important consideration for optimum performance. We demonstrate performance gains of 1-3 orders of magnitude in the parallel implementation, with peak performance strongly dependent on model specification.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

Numerical modeling of laser-driven experiments aiming to demonstrate magnetic field amplification via turbulent dynamo

The universe is permeated by magnetic fields, with strengths ranging from a femtogauss in the voids between the filaments of galaxy clusters to several teragauss in black holes and neutron stars. The standard model behind cosmological magnetic fields is the nonlinear amplification of seed fields via turbulent dynamo to the values observed. We have conceived experiments that aim to demonstrate and study the turbulent dynamo mechanism in the laboratory. Here, we describe the design of these experiments through simulation campaigns using FLASH, a highly capable radiation magnetohydrodynamics code that we have developed, and large-scale three-dimensional simulations on the Mira supercomputer at the Argonne National Laboratory. The simulation results indicate that the experimental platform may be capable of reaching a turbulent plasma state and determining the dynamo amplification. As a result, we validate and compare our numerical results with a small subset of experimental data using synthetic diagnostics.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗

Green’s function coupled cluster simulation of the near-valence ionizations of DNA-fragments

Accurate description of the ionization process in DNA is crucial to the understanding of the DNA damage under exposure to ionizing radiation and the exploration of the potential application of DNA strands in nanoelectronics. In this work, by employing our recently developed Green’s function coupled-cluster library on supercomputing facilities, we have studied the spectral functions of several guanine–cytosine (G–C) base pair structures ([G–C]n, n = 1–3) for the first time in a relatively broad near-valence regime ([−25.0, −5.0] eV) in the coupled-cluster with singles and doubles level. Our focus is to give a preliminary many-body coupled-cluster understanding and guideline of the vertical ionization energy (VIE), spectral profile, and ionization feature changes of these systems as the system size expands in this near-valence regime. The results show that, as the system size expands, even though the lowest VIEs keep decreasing, the changes of spectral function profile and the relative peak positions get unexpectedly smaller. Further analysis of the ionized states associated with the most intensive peak in the spectral functions reveals non-negligible |2h, 1p⟩’s in the ionized wave functions of the considered G–C base pair systems. The leading |2h, 1p⟩’s associated with the main ionizations from the cytosine part of the G–C base pairs feature a transition from the intra-base-pair cytosine π → π* excitation to the inter-base-pair electron excitation as the size of G–C base pairs expands, which also indicates the minimum quantum region in the many-body calculations of DNA systems.

Peng, Bo (ORCID:0000000242267294)↗

Scalable molecular dynamics on CPU and GPU architectures with NAMD

NAMD is a molecular dynamics program designed for high-performance simulations of very large biological objects on CPU- and GPU-based architectures. NAMD offers scalable performance on petascale parallel supercomputers consisting of hundreds of thousands of cores, as well as on inexpensive commodity clusters commonly found in academic environments. It is written in C++ and leans on Charm++ parallel objects for optimal performance on low-latency architectures. NAMD is a versatile, multipurpose code that gathers state-of-the-art algorithms to carry out simulations in apt thermodynamic ensembles, using the widely popular CHARMM, AMBER, OPLS, and GROMOS biomolecular force fields. Here, we review the main features of NAMD that allow both equilibrium and enhanced-sampling molecular dynamics simulations with numerical efficiency. We describe the underlying concepts utilized by NAMD and their implementation, most notably for handling long-range electrostatics; controlling the temperature, pressure, and pH; applying external potentials on tailored grids; leveraging massively parallel resources in multiple-copy simulations; and hybrid quantum-mechanical/molecular-mechanical descriptions. We detail the variety of options offered by NAMD for enhanced-sampling simulations aimed at determining free-energy differences of either alchemical or geometrical transformations and outline their applicability to specific problems. Last, we discuss the roadmap for the development of NAMD and our current efforts toward achieving optimal performance on GPU-based architectures, for pushing back the limitations that have prevented biologically realistic billion-atom objects to be fruitfully simulated, and for making large-scale simulations less expensive and easier to set up, run, and analyze. NAMD is distributed free of charge with its source code at www.ks.uiuc.edu.

high-performance computing↗

Toward exascale whole-device modeling of fusion devices: Porting the GENE gyrokinetic microturbulence code to GPU

GENE solves the five-dimensional gyrokinetic equations to simulate the development and evolution of plasma microturbulence in magnetic fusion devices. The plasma model used is close to first principles and computationally very expensive to solve in the relevant physical regimes. In order to use the emerging computational capabilities to gain new physics insights, several new numerical and computational developments are required. Here, we focus on the fact that it is crucial to efficiently utilize GPUs (graphics processing units) that provide the vast majority of the computational power on such systems. In this paper, we describe the various porting approaches considered and given the constraints of the GENE code and its development model, justify the decisions made, and describe the path taken in porting GENE to GPUs. We introduce a novel library called gtensor that was developed along the way to support the process. Performance results are presented for the ported code, which in a single node of the Summit supercomputer achieves a speed-up of almost 15× compared to running on central processing unit (CPU) only. Typical GPU kernels are memory-bound, achieving about 90% of peak. Our analysis shows that there is still room for improvement if we can refactor/fuse kernels to achieve higher arithmetic intensity. We also performed a weak parallel scalability study, which shows that the code runs well on a massively parallel system, but communication costs start becoming a significant bottleneck.

Germaschewski, K. (ORCID:0000000284956354)↗

Multi-level parallelization of quantum-chemical calculations

Here, strategies for multiple-level parallelizations of quantum-mechanical calculations are discussed, with an emphasis on using groups of workers for performing parallel tasks. These parallel programming models can be used for a variety ab initio quantum chemistry approaches, including the fragment molecular orbital method and replica-exchange molecular dynamics. Strategies for efficient load balancing on problems of increasing granularity are introduced and discussed. A four-level parallelization is developed based on a multi-level hierarchical grouping, and a high parallel efficiency is achieved on the Theta supercomputer using 131 072 OpenMP threads.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

Accuracy of the explicit energy-conserving particle-in-cell method for under-resolved simulations of capacitively coupled plasma discharges

The traditional explicit electrostatic momentum-conserving particle-in-cell algorithm requires strict resolution of the electron Debye length to deliver numerical stability and accuracy. The explicit electrostatic energy-conserving particle-in-cell algorithm alleviates this constraint with minimal modification to the traditional algorithm, retaining its simplicity, ease of parallelization, and acceleration on modern supercomputing architectures. In this article, we apply the algorithm to model a one-dimensional radio frequency capacitively coupled plasma discharge relevant to industrial applications. The energy-conserving approach closely matches the results from the momentum-conserving algorithm and retains accuracy even for cell sizes up to 8 times the electron Debye length. For even larger cells, the algorithm loses accuracy due to poor resolution of steep gradients within the radio frequency sheath. Accuracy can be recovered by adopting a non-uniform grid, which resolves the sheath and allows for cell sizes up to 32 times the electron Debye length in the quasi-neutral bulk of the discharge. The effect is an up to 8 times reduction in the number of required simulation cells, an improvement that can compound in higher-dimensional simulations. We therefore consider the explicit energy-conserving algorithm as a promising approach to significantly reduce the computational cost of full-scale device simulations and a pathway to delivering kinetic simulation capabilities of use to industry.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗

Accelerated kinetic model for global macro stability studies of high-beta fusion reactors

The field reversed configuration (FRC), such as studied in the C-2W experiment at TAE Technologies, is an attractive candidate for realizing a nuclear fusion reactor. In an FRC, kinetic ion effects play the majority role in macroscopic stability, which allows global stability studies to make use of fluid-kinetic hybrid (also referred to as Ohm's law) models wherein ions are treated kinetically while electrons are treated as a fluid. The development and validation of such a hybrid particle-in-cell algorithm in the Exascale Computing Project code WarpX are reported here. Implementation of this model in the WarpX framework benefits from the numerical efficiency of WarpX as well as its scalability on large HPC systems and portability to different architectures. Performance benchmarks of the new algorithm for large, 3-dimensional, full device simulations from the Perlmutter supercomputer are presented. Results of a series of FRC simulations are discussed in which the impact of two-fluid effects on the tilt-mode growth rate was studied. It was observed that, in agreement with previous Hall-MHD studies, two-fluid effects have a stabilizing impact on the tilt mode.

Physics↗

Special Topic on High Performance Computing in Chemical Physics

Computational modeling and simulation have become indispensable scientific tools in virtually all areas of chemical, biomolecular, and materials systems research. Computation can provide unique and detailed atomic level information that is difficult or impossible to obtain through analytical theories and experimental investigations. In addition, recent advances in micro-electronics have resulted in computer architectures with unprecedented computational capabilities, from the largest supercomputers to common desktop computers. In conclusion, combined with the development of new computational domain science methodologies and novel programming models and techniques, this has resulted in modeling and simulation resources capable of providing results at or better than experimental chemical accuracy and for systems in increasingly realistic chemical environments.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗