Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “supercomputers”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,009 records · Page 56

Low-synch Gram–Schmidt with delayed reorthogonalization for Krylov solvers

The parallel strong-scaling of iterative methods is often determined by the number of global reductions at each iteration. Low-synch Gram-Schmidt algorithms are applied here to the Arnoldi algorithm to reduce the number of global reductions and therefore to improve the parallel strong-scaling of iterative solvers for nonsymmetric matrices such as the GMRES and the Krylov-Schur iterative methods. In the Arnoldi context, the factorization is "left-looking" and processes one column at a time. Among the methods for generating an orthogonal basis for the Arnoldi algorithm, the classical Gram-Schmidt algorithm, with reorthogonalization (CGS2) requires three global reductions per iteration. A new variant of CGS2 that requires only one reduction per iteration is presented and applied to the Arnoldi algorithm. Delayed CGS2 (DCGS2) employs the minimum number of global reductions per iteration (one) for a one-column at-a-time algorithm. The main idea behind the new algorithm is to group global reductions by rearranging the order of operations. DCGS2 must be carefully integrated into an Arnoldi expansion or a GMRES solver. Numerical stability experiments assess robustness for Krylov-Schur eigenvalue computations. Performance experiments on the ORNL Summit supercomputer then establish the superiority of DCGS2 over CGS2.

97 MATHEMATICS AND COMPUTING↗

AI and extreme scale computing to learn and infer the physics of higher order gravitational wave modes of quasi-circular, spinning, non-precessing black hole mergers

We use artificial intelligence (AI) to learn and infer the physics of higher order gravitational wave modes of quasi-circular, spinning, non precessing binary black hole mergers. We trained AI models using 14 million waveforms, produced with the surrogate model NRHybSur3dq8, that include modes up to $\ell$ ≤ 4 and (5,5), except for (4,0) and (4,1), that describe binaries with mass-ratios $\textit{q}$ ≤ 8, individual spins $s^z_{\{1,2\}} \in$[–0.8,0.8], and inclination angle $θ \in$ [0,π]. Our probabilistic AI surrogates can accurately constrain the mass-ratio, individual spins, effective spin, and inclination angle of numerical relativity waveforms that describe such signal manifold. We compared the predictions of our AI models with Gaussian process regression, random forest, k-nearest neighbors, and linear regression, and with traditional Bayesian inference methods through the PyCBC Inference toolkit, finding that AI outperforms all these approaches in terms of accuracy, and are between three to four orders of magnitude faster than traditional Bayesian inference methods. Our AI surrogates were trained within 3.4 hours using distributed training on 1,536 NVIDIA V100 GPUs in the Summit supercomputer.

79 ASTRONOMY AND ASTROPHYSICS↗

Turbulence theories and statistical closure approaches

When discussing research in physics and in science more generally, it is common to ascribe equal importance to the three components of the scientific trinity: theoretical, experimental, and computational studies. This review will explore the future of modern turbulence theory by tracing its history, which began in earnest with Kolmogorov’s 1941 analysis of turbulence cascade and inertial range [A.N. Kolmogorov, Dokl. Akad. Nauk SSSR, 30, 299, (1941); 32, 19, (1941)]. The 80th Anniversary of Kolmogorov’s landmark study is a welcome opportunity to survey the achievements and evaluate the future of the theoretical approach of turbulence research. Over the years, turbulence theories have been critically important in laying the foundation of our understanding of the nature of turbulent flows. In particular, the Direct Interaction Approximation (DIA) [R.H. Kraichnan, J. Fluid Mech., 5, 497 (1959)] and its subsequent development, known as the statistical closure approach, can be identified as perhaps the most profound single advancement. The remarkable success of the statistical closure has furnished a platform to study such essential concepts as the energy transfer process and interacting scales, and the roles of the straining and sweeping motions. More recently, the quasi-Lagrangian formulation of V. L’vov & I. Procaccia and Kraichnan’s solvable passive scalar model provided powerful ways to explore another fundamental aspect of turbulent flows, the phenomena of intermittency, and the associated anomalous scaling exponents. In the meantime, the theory of fluid equilibria has been developed to describe the large-scale structures that can emerge from turbulent cascades of two-dimensional and geophysical flows at a later time. And yet, despite all these successes, analytical treatments suffer from mathematical complexities. As a result, the utility of theoretical approaches has been limited to relatively idealized flows. On the other hand, in recent decades, computational abilities and experimental facilities have reached an unprecedented scale. Looking beyond the horizon, the imminent deployment of exascale supercomputers will generate complete datasets of the entire flow field of key benchmark flows, allowing researchers to extract additional measurements concerning fully developed, complex turbulent flow fields far beyond those available from the statistical closure theories. Some other developments that could potentially influence the future course of turbulence theories include the advancement of machine learning, artificial intelligence, and data science; likely disruptions arising from the advent of quantum computation; and the increasingly prominent role of turbulence research in providing more accurate climate scientific data. Finally, turbulence theorists can leverage these developments by asking the right questions and developing advanced, sophisticated frameworks that will be able to predict and correlate vast amounts of data from the other two components of the trinity.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

ParCAT: Parallel Climate Analysis Toolkit

Climate science is employing increasingly complex models and simulations to analyze the past and predict the future of Earth's climate. This growth in complexity is creating a widening gap between the data being produced and the ability to analyze the datasets. Parallel computing tools are necessary to analyze, compare, and interpret the simulation data. The Parallel Climate Analysis Toolkit (ParCAT) provides basic tools to efficiently use parallel computing techniques to make analysis of these datasets manageable. The toolkit provides the ability to compute spatio-temporal means, differences between runs or differences between averages of runs, and histograms of the values in a data set. ParCAT is implemented as a command-line utility written in C. This allows for easy integration in other tools and allows for use in scripts. This also makes it possible to run ParCAT on many platforms – from laptops to supercomputers. ParCAT outputs NetCDF files so it is compatible with existing utilities such as Panoply and UV-CDAT. This paper describes ParCAT and presents results from some example runs on the Titan system at ORNL.

97 MATHEMATICS AND COMPUTING↗

MapReduce SVM Game

Despite technological advances making computing devices faster, smaller, and more prevalent in today's age, data generation and collection has outpaced data processing capabilities. Simply having more compute platforms does not provide a means of addressing challenging problems in the big data era. Rather, alternative processing approaches are needed and the application of machine learning to big data is hugely important. The MapReduce programming paradigm is an alternative to conventional supercomputing approaches, and requires less stringent data passing constrained problem decompositions. Rather, MapReduce relies upon defining a means of partitioning the desired problem so that subsets may be computed independently and recom- bined to yield the net desired result. However, not all machine learning algorithms are amenable to such an approach. Game-theoretic algorithms are often innately distributed, consisting of local interactions between players without requiring a central authority and are iterative by nature rather than requiring extensive retraining. Effectively, a game-theoretic approach to machine learning is well suited for the MapReduce paradigm and provides a novel, alternative new perspective to addressing the big data problem. In this paper we present a variant of our Support Vector Machine (SVM) Game classifier which may be used in a distributed manner, and show an illustrative example of applying this algorithm.

97 MATHEMATICS AND COMPUTING↗

Asynchronous Two-Level Checkpointing Scheme for Large-Scale Adjoints in the Spectral-Element Solver Nek5000

Adjoints are an important computational tool for large-scale sensitivity evaluation, uncertainty quantification, and derivative-based optimization. An essential component of their performance is the storage/recomputation balance in which efficient checkpointing methods play a key role. We introduce a novel asynchronous two-level adjoint checkpointing scheme for multistep numerical time discretizations targeted at large-scale numerical simulations. The checkpointing scheme combines bandwidth-limited disk checkpointing and binomial memory checkpointing. Based on assumptions about the target petascale systems, which we later demonstrate to be realistic on the IBM Blue Gene/Q system Mira, we create a model of the expected performance of our checkpointing approach and validate it using the highly scalable Navier-Stokes spectralelement solver Nek5000 on small to moderate subsystems of the Mira supercomputer. In turn, this allows us to predict optimal algorithmic choices when using all of Mira. We also demonstrate that two-level checkpointing is significantly superior to single-level checkpointing when adjoining a large number of time integration steps. To our knowledge, this is the first time two-level checkpointing had been designed, implemented, tuned, and demonstrated on fluid dynamics codes at large scale of 50k+ cores.

Adjoints↗

Generating unstructured nuclear reactor core meshes in parallel

Recent advances in supercomputers and parallel solver techniques have enabled users to run large simulations problems using millions of processors. Techniques for multiphysics nuclear reactor core simulations are under active development in several countries. Most of these techniques require large unstructured meshes that can be hard to generate in a standalone desktop computers because of high memory requirements, limited processing power, and other complexities. We have previously reported on a hierarchical lattice-based approach for generating reactor core meshes. Here, we describe efforts to exploit coarse-grained parallelism during reactor assembly and reactor core mesh generation processes. We highlight several reactor core examples including a very high temperature reactor, a full-core model of the Korean MONJU reactor, a ¼ pressurized water reactor core, the fast reactor Experimental Breeder Reactor-II core with a XX09 assembly, and an advanced breeder test reactor core. The times required to generate large mesh models, along with speedups obtained from running these problems in parallel, are reported. A graphical user interface to the tools described here has also been developed.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗

G-protein coupled receptors: advances in simulation and drug discovery

G-protein coupled receptors (GPCRs), the largest family of human membrane proteins, mediate cellular signaling and represent primary targets of about one third of currently marketed drugs. GPCRs undergo highly dynamic structural transitions during signal transduction, from binding of extracellular ligands to coupling with intracellular effector proteins. Molecular dynamics (MD) simulations have been utilized to investigate GPCR signaling mechanisms (such as pathways of ligand binding and receptor activation/deactivation) and to design novel small-molecule drug candidates. Future research directions point towards modeling cooperative binding of multiple orthosteric and allosteric ligands to GPCRs, GPCR oligomerization and interactions of GPCRs with different intracellular signaling proteins. Lastly, through methodological and supercomputing advances, MD simulations will continue to provide important insights into GPCR signaling mechanisms and further facilitate structure-based drug design.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

SKA shakes hands with Summit

Recently, a full-scale data processing workflow (Fig. 1) of the Square Kilometre Array (SKA) Phase 1 was successfully executed on the world’s fastest supercomputer Summit, proving that scientists have the expertise, software tools and computing resources to process the SKA data.

79 ASTRONOMY AND ASTROPHYSICS↗

Xcompact3D: An open-source framework for solving turbulence problems on a Cartesian mesh

Xcompact3D is a Fortran 90–95 open-source framework designed for fast and accurate simulations of turbulent flows, targeting CPU-based supercomputers. It is an evolution of the flow solver Incompact3D which was initially designed in France in the mid-90’s for serial processors to solve the incompressible Navier–Stokes equations. Incompact3D was then ported to parallel High Performance Computing (HPC) systems in the early 2010’s. Very recently the capabilities of Incompact3D have been extended so that it can now tackle more flow regimes (from incompressible flows to compressible flows at low Mach numbers), resulting in the design of a new user-friendly framework called Xcompact3D. The present manuscript presents an overview of Xcompact3D with a particular focus on its functionalities, its ready-to-run simulations and a few case studies to demonstrate its impact.

17 WIND ENERGY↗

MPLBM-UT: Multiphase LBM library for permeable media analysis

MPLBM-UT is a specialized lattice-Boltzmann library that makes running single- and two-phase flow simulations in porous media accessible to everyone. We provide a suite of tools to pre-process computational domains for simulation, to set up custom boundary conditions, to run simulations, to post-process simulation outputs, and to visualize simulation results and data. All of these tools are easily accessible to users through the mplbm_utils Python package included in and automatically installed with MPLBM-UT. The high-performance, highly parallel library Palabos is used as the solver backend. MPLBM-UT is easily deployed in a variety of systems, from laptops to supercomputer clusters. MPLBM-UT also features multiple examples and benchmark templates that allow for fast prototyping of different porous media problems. We also provide an interface for reading in different file types and downloading domains from the Digital Rocks Portal to perform simulations.

97 MATHEMATICS AND COMPUTING↗

HARD: A performance portable radiation hydrodynamics code based on FleCSI framework

Hydrodynamics And Radiation Diffusion (HARD) is an open-source application for high-performance simulations of compressible hydrodynamics with radiation-diffusion coupling. Built on the FleCSI (Bergen et al., 2021 [1]) (Flexible Computational Science Infrastructure) framework, HARD expresses its computational units as tasks whose execution can be orchestrated by multiple back-end runtimes, including Legion (Bauer et al., 2012 [2]), MPI (Forum, 1994 [3]), and HPX (Kaiser et al., 2020 [4]). Node-level parallelism is handled through Kokkos (Edwards et al., 2014 [5]), providing a single-source, portable code base that runs efficiently on laptops, small homogeneous clusters, and the largest heterogeneous supercomputers currently available. To ensure scientific reliability, HARD includes a regression test suite that automatically reproduces canonical verification problems such as the Sod and LeBlanc shock tubes, and the Sedov blast wave, comparing numerical solutions against known analytical results. The project is distributed under an OSI-approved license, hosted on GitHub, and accompanied by reproducible build scripts and continuous integration workflows. This combination of performance portability, verification infrastructure, and community-focused development makes HARD a sustainable platform for advancing radiation hydrodynamics research across multiple domains.

97 MATHEMATICS AND COMPUTING↗

Arsenic moiety in gallium arsenide is responsible for neuronal apoptosis and behavioral alterations in rats

Gallium arsenide (GaAs), an intermetallic semiconductor finds widespread applications in high frequency microwave and millimeter wave, and ultra fast supercomputers. Extensive use of GaAs has led to increased exposure to humans working in semiconductor industry. GaAs has the ability to dissociate into its constitutive moieties at physiological pH and might be responsible for the oxidative stress. The present study was aimed at evaluating, the principle moiety (Ga or As) in GaAs to cause neurological dysfunction based on its ability to cause apoptosis, in vivo and in vitro and if this neuronal dysfunction translated to neurobehavioral changes in chronically exposed rats. Result indicated that arsenic moiety in GaAs was mainly responsible for causing oxidative stress via increased reactive oxygen species (ROS) and nitric oxide (NO) generation, both in vitro and in vivo. Increased ROS further caused apoptosis via mitochondrial driven pathway. Effects of oxidative stress were also confirmed based on alterations in antioxidant enzymes, GPx, GST and SOD in rat brain. We noted that ROS induced oxidative stress caused changes in the brain neurotransmitter levels, Acetylcholinesterase and nitric oxide synthase, leading to loss of memory and learning in rats. The study demonstrates for the first time that the slow release of arsenic moiety from GaAs is mainly responsible for oxidative stress induced apoptosis in neuronal cells causing behavioral changes.

60 APPLIED LIFE SCIENCES↗

Comparative evaluation of deep learning workloads for leadership-class systems

Deep learning (DL) workloads and their performance at scale are becoming important factors to consider as we design, develop and deploy next-generation high-performance computing systems. Since DL applications rely heavily on DL frameworks and underlying compute (CPU/GPU) stacks, it is essential to gain a holistic understanding from compute kernels, models, and frameworks of popular DL stacks, and to assess their impact on science-driven, mission-critical applications. At Oak Ridge Leadership Computing Facility (OLCF), we employ a set of micro and macro DL benchmarks established through the Collaboration of Oak Ridge, Argonne, and Livermore (CORAL) to evaluate the AI readiness of our next-generation supercomputers. In this paper, we present our early observations and performance benchmark comparisons between the Nvidia V100 based Summit system with its CUDA stack and an AMD MI100 based testbed system with its ROCm stack. We take a layered perspective on DL benchmarking and point to opportunities for future optimizations in the technologies that we consider.

Yin, Junqi↗

Numerical investigation of the role of free-stream turbulence in boundary-layer separation

The aerodynamic performance of lifting surfaces operating at low Reynolds number conditions is impaired by laminar separation. In most cases, transition to turbulence occurs in the separated shear layer as a result of a series of strong hydrodynamic instability mechanisms. Although the understanding of these mechanisms has been significantly advanced over the past decades, key questions remain unanswered about the influence of external factors such as free-stream turbulence (FST) and others on transition and separation. The present study is driven by the need for more accurate predictions of separation and transition phenomena in ‘real world’ applications, where elevated levels of FST can play a significant role (e.g. turbomachinery). Numerical investigations have become an integral part in the effort to enhance our understanding of the intricate interactions between separation and transition. Due to the development of advanced numerical methods and the increase in the performance of supercomputers with parallel architecture, it has become feasible for low Reynolds number application ( $O(10^{5})$ ) to carry out direct numerical simulations (DNS) such that all relevant spatial and temporal scales are resolved without the use of turbulence modelling. Because the employed high-order accurate DNS are characterized by very low levels of background noise, they lend themselves to transition research where the amplification of small disturbances, sometimes even growing from numerical round-off, can be examined in great detail. When comparing results from DNS and experiment, however, it is beneficial, if not necessary, to increase the background disturbance levels in the DNS to levels that are typical for the experiment. For the current work, a numerical model that emulates a realistic free-stream turbulent environment was adapted and implemented into an existing Navier–Stokes code based on a vorticity–velocity formulation. The role FST plays in the transition process was then investigated for a laminar separation bubble forming on a flat plate. FST was shown to cause the formation of the well-known Klebanoff mode that is represented by streamwise-elongated streaks inside the boundary layer. Increasing the FST levels led to accelerated transition, a reduction in bubble size and better agreement with the experiments. Moreover, the stage of linear disturbance growth due to the inviscid shear-layer instability was found to not be ‘bypassed’.

Mechanics↗

Small-scale properties from exascale computations of turbulence on a $\mathbf{32\,768^3}$ periodic cube

To study the physics of small-scale properties of homogeneous isotropic turbulence at increasingly high Reynolds numbers, direct numerical simulation results have been obtained for forced isotropic turbulence at Taylor-scale Reynolds number R λ = 2500 on a 32 768 3 three-dimensional periodic domain using a GPU pseudo-spectral code on a 1.1 exaflop GPU supercomputer (Frontier). These simulations employ the multi-resolution independent simulation (MRIS) technique (Yeung & Ravikumar 2020, Phys. Rev. Fluids, vol. 5, 110517) where ensemble averaging is performed over multiple short segments initiated from velocity fields at modest resolution, and subsequently taken to higher resolution in both space and time. Reynolds numbers are increased by reducing the viscosity with the large-scale forcing parameters unchanged. Although MRIS segments at the highest resolution for each Reynolds number last for only a few Kolmogorov time scales, small-scale physics in the dissipation range is well captured – for instance, in the probability density functions and higher moments of the dissipation rate and enstrophy density, which appear to show monotonic trends persisting well beyond the Reynolds number range in prior works in the literature. Attainment of range of length and time scales consistent with classical scaling also reinforces the potential utility of the present high-resolution data for studies of short-time-scale turbulence physics at high Reynolds numbers where full-length simulations spanning many large-eddy time scales are still not accessible. A single snapshot of the 32 768 3 data is publicly available for further analyses via the Johns Hopkins Turbulence Database.

intermittency↗

Key role of the REC lobe during CRISPR–Cas9 activation by ‘sensing’, ‘regulating’, and ‘locking’ the catalytic HNH domain

Understanding the conformational dynamics of CRISPR (clustered regularly interspaced short palindromic repeat)–Cas9 is of the utmost importance for improving its genome editing capability. Here, molecular dynamics simulations performed using Anton-2 – a specialized supercomputer capturing micro-to-millisecond biophysical events in real time and at atomic-level resolution – reveal the activation process of the endonuclease Cas9 toward DNA cleavage. Over the unbiased simulation, we observe that the spontaneous approach of the catalytic domain HNH to the DNA cleavage site is accompanied by a remarkable structural remodeling of the recognition (REC) lobe, which exerts a key role for DNA cleavage. Specifically, the significant conformational changes and the collective conformational dynamics of the REC lobe indicate a mechanism by which the REC1–3 regions ‘sense’ nucleic acids, ‘regulate’ the HNH conformational transition, and ultimately ‘lock’ the HNH domain at the cleavage site, contributing to its catalytic competence. By integrating additional independent simulations and existing experimental data, we provide a solid validation of the activated HNH conformation, which had been so far poorly characterized, and we deliver a comprehensive understanding of the role of REC1–3 in the activation process. Considering the importance of the REC lobe in the specificity of Cas9, this study poses the basis for fully understanding how the REC components control the cleavage of off-target sequences, laying the foundation for future engineering efforts toward improved genome editing.

59 BASIC BIOLOGICAL SCIENCES↗

All-Atom Biomolecular Simulation in the Exascale Era

Exascale supercomputers have opened the door to dynamic simulations, facilitated by AI/ML techniques, that model biomolecular motions over unprecedented length and time scales. This new capability holds the potential to revolutionize our understanding of fundamental biological processes. Herein we report on some of the major advances that were discussed at a recent CECAM workshop in Pisa, Italy, on the topic with a primary focus on atomic-level simulations. First, we highlight examples of current large-scale biomolecular simulations and the future possibilities enabled by crossing the exascale threshold. Next, we discuss challenges to be overcome in optimizing the usage of these powerful resources. Finally, we close by listing several grand challenge problems that could be investigated with this new computer architecture.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗