Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “parallel performance”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 361 records · Page 20

Performance of the OVERFLOW-MLP and LAURA-MLP CFD Codes on the NASA Ames 512 CPU Origin System

The shared memory Multi-Level Parallelism (MLP) technique, developed last year at NASA Ames has been very successful in dramatically improving the performance of important NASA CFD codes. This new and very simple parallel programming technique was first inserted into the OVERFLOW production CFD code in FY 1998. The OVERFLOW-MLP code's parallel performance scaled linearly to 256 CPUs on the NASA Ames 256 CPU Origin 2000 system (steger). Overall performance exceeded 20.1 GFLOP/s, or about 4.5x the performance of a dedicated 16 CPU C90 system. All of this was achieved without any major modification to the original vector based code. The OVERFLOW-MLP code is now in production on the inhouse Origin systems as well as being used offsite at commercial aerospace companies. Partially as a result of this work, NASA Ames has purchased a new 512 CPU Origin 2000 system to further test the limits of parallel performance for NASA codes of interest. This paper presents the performance obtained from the latest optimization efforts on this machine for the LAURA-MLP and OVERFLOW-MLP codes. The Langley Aerothermodynamics Upwind Relaxation Algorithm (LAURA) code is a key simulation tool in the development of the next generation shuttle, interplanetary reentry vehicles, and nearly all "X" plane development. This code sustains about 4-5 GFLOP/s on a dedicated 16 CPU C90. At this rate, expected workloads would require over 100 C90 CPU years of computing over the next few calendar years. It is not feasible to expect that this would be affordable or available to the user community. Dramatic performance gains on cheaper systems are needed. This code is expected to be perhaps the largest consumer of NASA Ames compute cycles per run in the coming year.The OVERFLOW CFD code is extensively used in the government and commercial aerospace communities to evaluate new aircraft designs. It is one of the largest consumers of NASA supercomputing cycles and large simulations of highly resolved full aircraft are routinely undertaken. Typical large problems might require 100s of Cray C90 CPU hours to complete. The dramatic performance gains with the 256 CPU steger system are exciting. Obtaining results in hours instead of months is revolutionizing the way in which aircraft manufacturers are looking at future aircraft simulation work. Figure 2 below is a current state of the art plot of OVERFLOW-MLP performance on the 512 CPU Lomax system. As can be seen, the chart indicates that OVERFLOW-MLP continues to scale linearly with CPU count up to 512 CPUs on a large 35 million point full aircraft RANS simulation. At this point performance is such that a fully converged simulation of 2500 time steps is completed in less than 2 hours of elapsed time. Further work over the next few weeks will improve the performance of this code even further.The LAURA code has been converted to the MLP format as well. This code is currently being optimized for the 512 CPU system. Performance statistics indicate that the goal of 100 GFLOP/s will be achieved by year's end. This amounts to 20x the 16 CPU C90 result and strongly demonstrates the viability of the new parallel systems rapidly solving very large simulations in a production environment.

Taft, James R.↗

Iterative sampling of expensive simulations for faster deep surrogate training

Deep neural network (DNN) surrogates of expensive physics simulations are enabling a rapid change in the way that common experimental design and analysis tasks are approached. Surrogate models allow simulations to be performed in parallel and separately from downstream tasks, thereby enabling analyses that would be impossible with the simulation in-the-loop; surrogates based on DNNs can effectively emulate diverse non-scalar data of the types collected in fusion and laboratory-astrophysics experiments. The challenge is in training the surrogate model, for which large ensembles of physics simulations must be run, preferably without wasting computational effort on uninteresting simulations. Here, in this paper, we present an iterative sampling scheme that can preferentially propose simulations in interesting regions of parameter space without neglecting unexplored regions, allowing high-quality and wide-ranging surrogate models to be trained using 2–3 times fewer simulations compare to space-filling designs. Our approach uses an explicit importance function defined on the simulation output space, balanced against a measure of simulation density which serves as a proxy for surrogate accuracy. It is easy to implement and can be tuned to find interesting simulations early in the study, allowing surrogates to be trained quickly and refined as new simulations become available; this represents an important step towards the routine generation of deep surrogate models quickly enough to be truly relevant to experimental work.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗

Phoenix: A Scalable Streaming Hypergraph Analysis Framework

We present Phoenix, a scalable hypergraph analytics framework for data analytics and knowledge discovery that was implemented on the leadership class computing platforms at Oak Ridge National Laboratory (ORNL). Our software framework comprises a distributed implementation of a streaming server architecture which acts as a gateway for various hypergraph generators/external sources to connect. Phoenix has the capability to utilize diverse hypergraph generators, including HyGen, a very large-scale hypergraph generator developed by ORNL. Phoenix incorporates specific algorithms for efficient data representation by exploiting hidden structures of the hypergraphs. Our experimental results demonstrate Phoenix’s scalable and stable performance on massively parallel computing platforms. Phoenix’s superior performance is due to the merging of high-performance computing with data analytic.

Kurte, Kuldeep↗

Enriched immersed finite element and isogeometric analysis: algorithms and data structures

Immersed finite element methods provide a convenient analysis framework for problems involving geometrically complex domains, such as those found in topology optimization and microstructures for engineered materials. However, their implementation remains a major challenge due to, among other things, the need to apply nontrivial stabilization schemes and generate custom quadrature rules. This article introduces the robust and computationally efficient algorithms and data structures comprising an immersed finite element preprocessing framework. The input to the preprocessor consists of a background mesh and one or more geometries defined on its domain. The output is structured into groups of elements with custom quadrature rules formatted such that common finite element assembly routines may be used without or with only minimal modifications. The key to the preprocessing framework is the construction of material topology information, concurrently with the generation of a quadrature rule, which is then used to perform enrichment and generate stabilization rules. While the algorithmic framework applies to a wide range of immersed finite element methods using different types of meshes, integration, and stabilization schemes, the preprocessor is presented within the context of the extended isogeometric analysis. This method utilizes a structured B-spline mesh, a generalized Heaviside enrichment strategy considering the material layout within individual basis functions’ supports, and face-oriented ghost stabilization. Using a set of examples, the effectiveness of the enrichment and stabilization strategies is demonstrated alongside the preprocessor’s robustness in geometric edge cases. Additionally, the performance and parallel scalability of the implementation are evaluated.

Computer implementation↗

Causal explicit algorithm for heat conduction in a plasma

Hyperbolic heat conduction extends standard Spitzer-Harm heat conduction by including a term proportional to the time derivative of the heat flux. The new term arises from a kinetic derivation of the heat flux that includes higher order corrections. Here we present a causal explicit numerical algorithm for solving the nonlinear hyperbolic heat conduction equation in an unmagnetized plasma. The maximum stable timestep for the causal explicit algorithm scales linearly with the cell size, owing to the hyperbolic nature of the problem. This is in contrast to the quadratic scaling of the maximum stable timestep with the cell size for the parabolic forward time centered space algorithm. The favorable scaling of the timestep with the cell size enables a practical explicit implementation of heat conduction in high-performance massively parallel plasma codes. In particular, we have implemented the causal explicit algorithm in the laser plasma interaction code pF3D. We verify the CE algorithm and analyze its convergence rate by simulating a harmonic mode, which has an analytic solution within the context of the HHC model. We also compare simulations using the CE algorithm to those using the forward time centered space algorithm on a pair of test problems: evolution in time of a Gaussian temperature perturbation in a uniform plasma and heat transport in the presence of inverse bremsstrahlung heating by a Gaussian laser speckle.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗

High-order multirate explicit time-stepping schemes for the baroclinic-barotropic split dynamics in primitive equations

In order to treat the multiple time scales of ocean dynamics in an efficient manner, the baroclinic-barotropic splitting technique has been widely used for solving the primitive equations for ocean modeling. Based on the framework of strong stability-preserving Runge-Kutta approach, we propose two high-order multirate explicit time-stepping schemes (SSPRK2-SE and SSPRK3-SE) for the resulting split system in this paper. The proposed schemes allow for a large time step to be used for the three-dimensional baroclinic (slow) mode and a small time step for the two-dimensional barotropic (fast) mode, in which each of the two mode solves just need to satisfy their respective CFL conditions for numerical stability. Specifically, at each time step, the baroclinic velocity is first computed by advancing the baroclinic mode and fluid thickness of the system with the large time-step and the assistance of some intermediate approximations of the barotropicmode obtained by substepping with the small time step; then the barotropic velocity is corrected by using the small time step to re-advance the barotropic mode un-der an improved barotropic forcing produced by interpolation of the forcing terms from the preceding baroclinic mode solves; lastly, the fluid thickness is updated by coupling the baroclinic and barotropic velocities. Additionally, numerical inconsistencies on the discretized sea surface height caused by the mode splitting are relieved via a reconciliation process with carefully calculated flux deficits. Here, two benchmark tests from the “MPAS-Ocean” platform are carried out to numerically demonstrate the performance and parallel scalability of the proposed SSPRK-SE schemes.

54 ENVIRONMENTAL SCIENCES↗

Xcompact3D: An open-source framework for solving turbulence problems on a Cartesian mesh

Xcompact3D is a Fortran 90–95 open-source framework designed for fast and accurate simulations of turbulent flows, targeting CPU-based supercomputers. It is an evolution of the flow solver Incompact3D which was initially designed in France in the mid-90’s for serial processors to solve the incompressible Navier–Stokes equations. Incompact3D was then ported to parallel High Performance Computing (HPC) systems in the early 2010’s. Very recently the capabilities of Incompact3D have been extended so that it can now tackle more flow regimes (from incompressible flows to compressible flows at low Mach numbers), resulting in the design of a new user-friendly framework called Xcompact3D. The present manuscript presents an overview of Xcompact3D with a particular focus on its functionalities, its ready-to-run simulations and a few case studies to demonstrate its impact.

17 WIND ENERGY↗

Drive-pressure optimization in ramp-wave compression experiments through differential evolution

Ramp-wave dynamic-compression experiments are used to examine quasi-isentropic loading paths in materials. The gradual and continuous increase in pressure created by ramp waves make these types of experiments ideal for studying nonequilibrium material behavior, such as solidification kinetics. In ramp-wave compression experiments, the input drive pressure to the experimental setup may be exerted through one of a number of different mechanisms (e.g., magnetic fields, gas-gun-driven impactors, or high-energy lasers) and is generally required for simulating such experiments. Yet, regardless of the specific mechanism, this drive pressure cannot be measured directly (measurements are generally taken at a location near the back of the experimental setup through a transparent window), leading to an inverse problem where one must determine the drive pressure at the front of the experimental setup (i.e., the input) that corresponds to the particle velocity (the output) measured near the back of the experimental setup. Furthermore, we solve this inverse problem using a heuristic optimization algorithm, known as differential evolution, coupled with a multiphysics, hydrodynamics code that simulates the compression of the experimental setup. By running many rounds of forward simulations of the experimental setup, our optimization process iteratively searches for a drive pressure that is optimized to closely reproduce the experimentally measured particle velocity near the back of the experimental setup. While our optimization methodology requires a significant number of hydrodynamics simulations to be conducted, many of these can be performed in parallel, which greatly reduces the time cost of our methodology. One novel aspect of our method for determining the drive pressure is that it does not require physical modeling of the drive mechanism and can thus be broadly applied to many types of ramp-compression experiments, regardless of the drive mechanism.

75 CONDENSED MATTER PHYSICS, SUPERCONDUCTIVITY AND↗

Accelerating the density-functional tight-binding method using graphical processing units

Acceleration of the density-functional tight-binding (DFTB) method on single and multiple graphical processing units (GPUs) was accomplished using the MAGMA linear algebra library. Herein two major computational bottlenecks of DFTB ground-state calculations were addressed in our implementation: the Hamiltonian matrix diagonalization and the density matrix construction. The code was implemented and benchmarked on two different computer systems: (1) the SUMMIT IBM Power9 supercomputer at the Oak Ridge National Laboratory Leadership Computing Facility with 1–6 NVIDIA Volta V100 GPUs per computer node and (2) an in-house Intel Xeon computer with 1–2 NVIDIA Tesla P100 GPUs. The performance and parallel scalability were measured for three molecular models of 1-, 2-, and 3-dimensional chemical systems, represented by carbon nanotubes, covalent organic frameworks, and water clusters.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

Temperature dependence of the energy band gap in ZrTe 5 : Implications for the topological phase

Using Landau level spectroscopy, we determine the temperature dependence of the energy band gap in zirconium pentatelluride (ZrTe 5 ). We find that the band gap reaches E g = (5±1) meV at low temperatures and increases monotonously when the temperature is raised. So this implies that ZrTe 5 is a weak topological insulator, with non-inverted ordering of electronic bands in the center of the Brillouin zone. Our magneto-transport experiments performed in parallel show that the resistivity anomaly in ZrTe 5 is not connected with the temperature dependence of the band gap.

75 CONDENSED MATTER PHYSICS, SUPERCONDUCTIVITY AND↗

DistGANS- Distributed Generative Adversarial Neural Networks

DistGANs is a Python package to perform distributed training of conditional generative adversarial neural networks for multi-class labeled image data. DistGANs partitionins the training data according to data labels, and enhances scalability by performing a parallel training where multiple generators are concurrently trained, each one of them focusing on a single data label.

Lupo Pasini, Massimiliano [Oak Ridge National Lab.↗

pinacles

Predicting Interactions of Aerosol and Clouds in Large Eddy Simulation (PINACLES) is a high-performance, massively parallel, anelastic atmospheric model written in Python

Pressel, Kyle↗

MARBLES (Multi-scale Adaptively Refined Boltzmann LatticE Solver) [SWR-23-37]

MARBLES (Multi-scale Adaptively Refined Boltzmann LatticE Solver) is an open-source computational fluid dynamics package powered by the lattice Boltzmann equations and built on AMReX. In the lattice Boltzmann method, local collisions between meso-scale fictitious particles drive the governing equations which enables MARBLES to easily simulate flow around complex and/or moving geometry without the generation of a body-conforming mesh. Using AMReX data structures and operations ensures a high level of computational performance and parallel scaling on heterogenous architectures while also naturally supporting locally enhanced grid resolution and fidelity through automatic mesh refinement. New domains and problem definitions are easily specified through an input file with examples and guidance on all options and variables provided in the MARBLES documentation.

Henry de Frahan, Marc↗

PFLOTRAN 5

PFLOTRAN leverages massively parallel, high performance computing to simulate large-scale non-isothermal multiphase flow, multicomponent reactive transport and electrical resistivity tomography (ERT) problems in the subsurface environment. Researchers have employed PFLOTRAN to simulate these Earth system processes on leadership class supercomputers for over two decades. The code is designed to predict the future estate of environmental systems and better inform stakeholders in the regulatory decision making process (e.g., fate of contaminants, long-term stewardship for nuclear waste, impact of climate change, etc.). A diverse team of scientists oversees PFLOTRAN development and maintenance under an open-source licensing agreement and manages contributions from an international community of researchers.

Hammond, Glenn↗

ALEGRA Parallel Scaling for Shock in a Heterogeneous Structure

We investigate the strong and weak parallel scaling performance of the ALEGRA multiphysics finite element program when solving a problem involving shock propagation through a heterogeneous material. We determine that ALEGRA scales well over a wide range of problem sizes, cores, and element sizes, and that scaling generally improves as the minimum element size in the mesh increases.

36 MATERIALS SCIENCE↗

Geometry, Disorder and Phase Transitions in Topological States of Matter

The quantum Hall effect is the birthplace of topological states of matter, a major theme at the forefront of condensed matter physics in the past two decades. The fractional quantum Hall (FQH) effect revolutionized our understanding of phases of electronic matter. FQH states support exotic fractionally charged excitations that obey Abelian or non-Abelian fractional statistics, which are topological excitations that result from the underlying topological order. During this project, our group discovered a previously unrecognized geometric degree of freedom of incompressible FQH states and studied that for a variety of gapped FQH states. We brought this new concept into direct contact with experiments for the first time by generalizing it to Fermi-liquid states of composite fermions. Using the newly formulated powerful infinite Density Matrix Renormalization Group method, our numerical calculations yielded a parameter free prediction that was found to be in excellent agreement with experimental findings on electron systems in semiconductor heterostructures. In parallel, we performed extensive numerical studies on different, competing phases at various Landau level filling factors, and quantum phase transitions that result from such a competition, e.g. Abelian-non-Abelian phase transitions in bilayer systems. We studied geometrical excitations dubbed “gravitons” (because of their analogy with excitations in the theory of gravitation) and ways to excite and detect them, and explored how they couple with topological excitations. In graphene-based chiral materials, we realized the ability to tune through different incompressible and compressible states in a single Landau level, and found appropriate experimental parameters for the exploration of universal Luttinger liquid behavior not obtained in semiconductor-based electron systems. We showed that topological systems had a very different response from nontopological systems to strong disorder (many-body localization) as well as periodic drives.

75 CONDENSED MATTER PHYSICS, SUPERCONDUCTIVITY AND↗

Verification and Performance Impact of the New Parallel MCNP6.3 Particle Track Output Capability for Subcritical Multiplication Simulations [Slides]

A separate MCNP6.3 V&V document reports on all the default calculations for all test suites. This report does not include the subcritical multiplication benchmark suite. After some additional clean-up and finalizing the post-processing and documentation steps, the subcritical multiplication benchmark suite will be released in the next version of our vnvstats repository. We tested the new HDF5 PTRAC feature in MCNP6.3 and found encouraging outcomes. Identical results coming out of the simulation with respect to the legacy PTRAC results. The overall runtime for all simulations is reduced by ~20% with the new HDF5 PTRAC capability. We consider giving the new HDF5 PTRAC features a try and using it for all subcritical multiplication and any other relevant (PTRAC) calculations.

97 MATHEMATICS AND COMPUTING↗

Developing And Scaling an OpenFOAM Model to Study Turbulent Flow in a HFIR Coolant Channel

Improving the understanding of how computational fluid dynamics (CFD) direct numerical simulations (DNS) of flows in the High Flux Isotope Reactor (HFIR) perform when run in parallel using the high performance computing (HPC) platform Summit at the Oak Ridge Leadership Computing Facility (OLCF) is of particular importance to boost the computational tools used to support HFIR conversion to low enriched fuel (LEU). Evaluation of scaling performance was driven by the increasing importance of graphics processing unit (GPU) usage in HPC, which is becoming the standard for modern supercomputers such as Summit. The desired results are to obtain a strong positive correlation between the computational resources dedicated to a problem and the relative speed-up of the simulation in comparison to a benchmark. This capability will allow substantially improvement in HFIR flow analytical capabilities, specifically when predicting turbulence properties at high Reynolds numbers. The study leverages previous simulation results performed with code PHASTA (finite element) on HPC platforms Cori (NERSC) and Theta (ALCF) [1] with computing options provided in the computing platform OpenFOAM (finite volume) at OLCF. Transitioning from PHASTA to OpenFOAM will (1) eliminate dependence on third-party software for mesh generation and manipulation, (2) reduce resource needs by employing modern architectures, and (3) build expertise for future modeling of HFIR-specific problems like heat transfer in involute geometry, entrance effects, flow structure in channel corners, and so on—all important issues when defining the available thermal margins in the transition to LEU. CPUs and GPUs differ significantly in their architecture and utilization, as discussed in the literature [2]. The most important differences are in the approach to computations and their memory. A single GPU contains a large quantity of cores, enabling it to perform with a much higher throughput than a CPU, but execution requires a different approach. GPU codes execute instructions using the Single-Instruction Multiple-Thread (SIMT) approach in which a single instruction is used for groups of threads called warps. A warp typically consists of 32 threads which must execute the same set of instructions, although on separate threads. Alternately, a CPU has far fewer cores that are much more flexible in their operation, excelling at quickly performing more complex serial computations. This is why GPUs have greater throughput when properly utilized. The second important difference is seen when comparing their memory spaces. Limited memory allocations and CPU–GPU communications cause a significant bottleneck in GPU-accelerated programs. Further study was required to properly take advantage of GPU resources. A comprehensive analysis of code performance and the model-specific features of turbulence constitutes the core of this work. In this study, a DNS simulation of HFIR channel turbulence was performed with the finite volume CFD code OpenFOAM v2112 and CUDA v11.0 on Red Hat Enterprise Linux v8.2. The OpenFOAM installation had AMGx integrated to enable GPU acceleration and utilizes the PETSc4FOAM library. The computational resources and the problem size were scaled on CPU and CPU + GPU architectures to gain a better understanding of the performance of a DNS problem on modern computing hardware. The study aimed to analyze the scaling of the code exclusively on CPUs and then to examine the scaling of the codes with GPU acceleration enabled. Scaling studies included CPU and GPU acceleration on a mesh of varying resolution to analyze the impact of problem size relative to computational resources. In the course of preparing the GPU configuration on Summit, mainly using the AMGX solvers, difficulties were encountered stemming from constant changes resulting from extensive ongoing development activities and the changing environment. This resulted in the inability to complete the GPU portion of the work. The code was compiled and tested, but production runs to assess acceleration were not performed because the used discretional compute time allocation expired as year-end approached. The Summit HPC platform is scheduled for decommissioning in 2024, making it unattractive for future use with Nvidia-based GPUs. Therefore, the work will be moved onto NERSC machines in FY24. An application was prepared and submitted, and sufficient node-hours were awarded to continue the research in the next calendar year. This report summarizes work performed thus far, which mostly focused on CPU OpenFOAM computing.

21 SPECIFIC NUCLEAR REACTORS AND ASSOCIATED PLANTS↗