Engineering Papers⌕ Search

Engineering topics

Slattery, Stuart R.

Publications and source records attributed to Slattery, Stuart R..

Enabling particle applications for exascale computing platforms

The Exascale Computing Project (ECP) is invested in co-design to assure that key applications are ready for exascale computing. Within ECP, the Co-design Center for Particle Applications (CoPA) is addressing challenges faced by particle-based applications across four “sub-motifs”: short-range particle–particle interactions (e.g., those which often dominate molecular dynamics (MD) and smoothed particle hydrodynamics (SPH) methods), long-range particle–particle interactions (e.g., electrostatic MD and gravitational N-body), particle-in-cell (PIC) methods, and linear-scaling electronic structure and quantum molecular dynamics (QMD) algorithms. Our crosscutting co-designed technologies fall into two categories: proxy applications (or “apps”) and libraries. Proxy apps are vehicles used to evaluate the viability of incorporating various types of algorithms, data structures, and architecture-specific optimizations and the associated trade-offs; examples include ExaMiniMD, CabanaMD, CabanaPIC, and ExaSP2. Libraries are modular instantiations that multiple applications can utilize or be built upon; CoPA has developed the Cabana particle library, PROGRESS/BML libraries for QMD, and the SWFFT and fftMPI parallel FFT libraries. Success is measured by identifiable “lessons learned” that are translated either directly into parent production application codes or into libraries, with demonstrated performance and/or productivity improvement. The libraries and their use in CoPA’s ECP application partner codes are also addressed.

97 MATHEMATICS AND COMPUTING↗

ArborX: A Performance Portable Geometric Search Library

Searching for geometric objects that are close in space is a fundamental component of many applications. The performance of search algorithms comes to the forefront as the size of a problem increases both in terms of total object count as well as in the total number of search queries performed. Scientific applications requiring modern leadership-class supercomputers also pose an additional requirement of performance portability, i.e., being able to efficiently utilize a variety of hardware architectures. In this article, we introduce a new open-source C++ search library, ArborX, which we have designed for modern supercomputing architectures. Herein, we examine scalable search algorithms with a focus on performance, including a highly efficient parallel bounding volume hierarchy implementation, and propose a flexible interface making it easy to integrate with existing applications. We demonstrate the performance portability of ArborX on multi-core CPUs and GPUs and compare it to the state-of-the-art libraries such as Boost.Geometry.Index and nanoflann.

97 MATHEMATICS AND COMPUTING↗

Multigroup Monte Carlo on GPUs: Comparison of history- and event-based algorithms

This article presents an investigation of the performance of different multigroup Monte Carlo transport algorithms on GPUs with a discussion of both history-based and event-based approaches. Several algorithmic improvements are introduced for both approaches. By modifying the history-based algorithm that is traditionally favored in CPU-based MC codes to occasionally filter out dead particles to reduce thread divergence, performance exceeds that of either the pure history-based or event-based approaches. The impacts of several algorithmic choices are discussed, including performance studies on Kepler and Pascal generation NVIDIA GPUs for fixed source and eigenvalue calculations. Single-device performance equivalent to 20–40 CPU cores on the K40 GPU and 60–80 CPU cores on the P100 GPU is achieved. Last, in addition, nearly perfect multi-device parallel weak scaling is demonstrated on more than 16,000 nodes of the Titan supercomputer.

97 MATHEMATICS AND COMPUTING↗

Analysis of Monte Carlo accelerated iterative methods for sparse linear systems: Analysis of Monte Carlo accelerated iterative methods for sparse linear systems

Here, we consider hybrid deterministic-stochastic iterative algorithms for the solution of large, sparse linear systems. Starting from a convergent splitting of the coefficient matrix, we analyze various types of Monte Carlo acceleration schemes applied to the original preconditioned Richardson (stationary) iteration. We expect that these methods will have considerable potential for resiliency to faults when implemented on massively parallel machines. We also establish sufficient conditions for the convergence of the hybrid schemes, and we investigate different types of preconditioners including sparse approximate inverses. Numerical experiments on linear systems arising from the discretization of partial differential equations are presented.

97 MATHEMATICS AND COMPUTING↗

L3:PHI.CMD.P13.02 Support for CILC L1 Milestone Using STAR-CCM+

This report documents work performed to support Consortium for the Advanced Simulation of LWRs (CASL) modeling of Chalk River Unidentified Deposit (CRUD) Induced Power Shift (CIPS) and CRUD Induced Local Corrosion (CILC) using the Cicada package. The work documented here is intended to complement current and future CIPS and CILC modeling activities in CASL. We provide tools for crud and corrosion-related simulation and analysis by developing a better understanding of the interplay between the coupled physics that describe the phenomena at different time and length scales. We intend to use these models to better inform future simulation capability and development.

21 SPECIFIC NUCLEAR REACTORS AND ASSOCIATED PLANTS↗

DTK C/Fortran Interface Development for NEAMS FSI Simulations

This report documents the development of DataTransferKit (DTK) C and Fortran interfaces for fluid-structure-interaction (FSI) simulations in NEAMS. In these simulations, the codes Nek5000 and Diablo are being coupled within the SHARP framework to study flow-induced vibration (FIV) in reactor steam generators. We will review the current Nek5000/Diablo coupling algorithm in SHARP and the current state of the solution transfer scheme used in this implementation. We will then present existing DTK algorithms which may be used instead to provide an improvement in both flexibility and scalability of the current SHARP implementation. We will show how these can be used within the current FSI scheme using a new set of interfaces to the algorithms developed by this work. These new interfaces currently expose the mesh-free solution transfer algorithms in DTK, a C++ library, and are written in C and Fortran to enable coupling of both Nek5000 and Diablo in their native Fortran language. They have been compiled and tested on Cooley, the test-bed machine for Mira at ALCF.

21 SPECIFIC NUCLEAR REACTORS AND ASSOCIATED PLANTS↗

Crashworthiness Models for Automotive Batteries - Report on Project 2088-A031-15 for DOT/NHTSA

Safety is a key element of any device designed to store energy, in particular electrochemical batteries, which convert energy of chemical reactions to electrical energy. Safety considerations are especially important when applied to large automotive batteries designed for propulsion of electric vehicles (EV). The high amount of energy stored in EV battery packs translates to higher probability of fire in case of severe deformation of battery compartment due to automotive crash or impact caused by road debris. While such demand for safety has resulted in heavier protection of battery enclosure, the mechanisms leading to internal short circuit due to deformation of the battery are not well understood even on the level of a single electrochemical cell. Moreover, not all internal shorts result in thermal runaway, and thus a criterion for catastrophic failure needs to be developed. This report summarizes the effort to pinpoint the critical deformation necessary to trigger a short via experimental study on large format automotive Li-ion cells in a rigid spherical indentation configuration. Cases of single cells and cell stacks undergoing indentation were investigated. Mechanical properties of cell components were determined via experimental testing and served as input for constitutive models of Finite Element (FE) analysis. The ability of the model to predict the behavior of cell(s) under spherical indentation and to predict failure leading to internal short circuit was validated against experiments. The necessity of resolving pairs of negative and positive electrodes in the FE formulation is clearly demonstrated by comparing layer-resolved simulations with simulations involving batteries with homogenized material properties. Finally, a coupled solution of electrochemical-electrical-thermal (EET) problem on a Nissan Leaf battery module was demonstrated towards the goal of extending the simulations to module level.

25 ENERGY STORAGE↗

GPU Acceleration of History-Based Multigroup Monte Carlo

Accurate solutions of the radiation transport equation play an important role in the analysis of many nuclear systems. Because there is no need to discretize the problem, Monte Carlo methods offer the most accurate possible solutions to the transport equation. However, significant computational effort is required to reduce the stochastic noise inherent to Monte Carlo methods. Current trends in high performance computing are moving towards vectorized, single instruction multiple data (SIMD), architectures such as Intel Xeon Phis and graphics processing units (GPUs) due to their favorable ratio of performance to power consumption. The challenge of adapting Monte Carlo transport to vectorized computing architectures was first addressed by Brown with an algorithm known as event-based Monte Carlo. In this approach, rather than simulating individual particle histories from birth to death, groups of particles are processed one event at a time. This allows each event batch to perform the same operations in lockstep, enabling use of the vector hardware. Although modern vectorized hardware is more versatile than the early vector machines, reducing so-called thread divergence is nonetheless still an important consideration for algorithm design. Many of the algorithms proposed so far for Monte Carlo transport on advanced architectures have focused on variants of event-based transport. Some evidence, however, suggests that low-level thread divergence, resulting from short-lived branching statements, may not be detrimental to overall performance on GPUs as long as the effect on achievable memory bandwidth is considered. This paper characterizes the performance of a traditional history-based Monte Carlo algorithm on GPUs. We show that with only minor modifications to traditional algorithms, significant performance gains relative to CPUs are possible. (authors)

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗

Mesh-free data transfer algorithms for partitioned multiphysics problems: Conservation, accuracy, and parallelism

In this study we analyze and extend mesh-free algorithms for three-dimensional data transfer problems in partitioned multiphysics simulations. We first provide a direct comparison between a mesh-based weighted residual method using the common-refinement scheme and two mesh-free algorithms leveraging compactly supported radial basis functions: one using a spline interpolation and one using a moving least square reconstruction. Through the comparison we assess both the conservation and accuracy of the data transfer obtained from each of the methods. We do so for a varying set of geometries with and without curvature and sharp features and for functions with and without smoothness and with varying gradients. Our results show that the mesh-based and mesh-free algorithms are complementary with cases where each was demonstrated to perform better than the other. We then focus on the mesh-free methods by developing a set of algorithms to parallelize them based on sparse linear algebra techniques. This includes a discussion of fast parallel radius searching in point clouds and restructuring the interpolation algorithms to leverage data structures and linear algebra services designed for large distributed computing environments. The scalability of our new algorithms is demonstrated on a leadership class computing facility using a set of basic scaling studies. Finally, these scaling studies show that for problems with reasonable load balance, our new algorithms for both spline interpolation and moving least square reconstruction demonstrate both strong and weak scalability using more than 100,000 MPI processes with billions of degrees of freedom in the data transfer operation.

97 MATHEMATICS AND COMPUTING↗

A spectral analysis of the domain decomposed Monte Carlo method for linear systems

The domain decomposed behavior of the adjoint Neumann-Ulam Monte Carlo method for solving linear systems is analyzed using the spectral properties of the linear oper- ator. Relationships for the average length of the adjoint random walks, a measure of convergence speed and serial performance, are made with respect to the eigenvalues of the linear operator. In addition, relationships for the effective optical thickness of a domain in the decomposition are presented based on the spectral analysis and diffusion theory. Using the effective optical thickness, the Wigner rational approxi- mation and the mean chord approximation are applied to estimate the leakage frac- tion of random walks from a domain in the decomposition as a measure of parallel performance and potential communication costs. The one-speed, two-dimensional neutron diffusion equation is used as a model problem in numerical experiments to test the models for symmetric operators with spectral qualities similar to light water reactor problems. We find, in general, the derived approximations show good agreement with random walk lengths and leakage fractions computed by the numerical experiments.

97 MATHEMATICS AND COMPUTING↗

Crash Models for Advanced Automotive Batteries: A Review of the Current State of the Art

Safety is a critical aspect of lithium-ion (Li-ion) battery design. Impact/crash conditions can trigger a complex interplay of mechanical contact, heat generation and electrical discharge, which can result in adverse thermal events. The cause of these thermal events has been linked to internal contact between the opposite electrodes, i.e. internal short circuit. The severity of the outcome is influenced by the configuration of the internal short circuit and the battery state. Different loading conditions and battery states may lead to micro (soft) shorts where material burnout due to generated heat eliminates contact between the electrodes, or persistent (hard) shorts which can lead to more significant thermal events and potentially damage the entire battery system and beyond. Experimental characterization of individual battery components for the onset of internal shorts is limited, since it is impractical to canvas all possible variations in battery state of charge, operating conditions, and impact loading in a timely manner. This report provides a survey of modeling and simulation approaches and documents a project initiated and funded by DOT/NHTSA to improve modeling and simulation capabilities in order to design tests that provide leading indicators of failure in batteries. In this project, ORNL has demonstrated a computational infrastructure to conduct impact simulations of Li-ion batteries using models that resolve internal structures and electro-thermo-chemical and mechanical conditions. Initial comparisons to abuse experiments on cells and cell strings conducted at ORNL and Naval Surface Warfare Center (NSWC) at Carderock MD for parameter estimation and model validation have been performed. This research has provided insight into the mechanisms of deformation in batteries (both at cell and electrode level) and their relationship to the safety of batteries.

33 ADVANCED PROPULSION SYSTEMS↗

An approach for coupled-code multiphysics core simulations from a common input

This study describes an approach for coupled-code multiphysics reactor core simulations that is being developed by the Virtual Environment for Reactor Applications (VERA) project in the Consortium for Advanced Simulation of Light-Water Reactors (CASL). In this approach a user creates a single problem description, called the “VERAIn” common input file, to define and setup the desired coupled-code reactor core simulation. A preprocessing step accepts the VERAIn file and generates a set of fully consistent input files for the different physics codes being coupled. The problem is then solved using a single-executable coupled-code simulation tool applicable to the problem, which is built using VERA infrastructure software tools and the set of physics codes required for the problem of interest. The approach is demonstrated by performing an eigenvalue and power distribution calculation of a typical three-dimensional 17 × 17 assembly with thermal–hydraulic and fuel temperature feedback. All neutronics aspects of the problem (cross-section calculation, neutron transport, power release) are solved using the Insilico code suite and are fully coupled to a thermal–hydraulic analysis calculated by the Cobra-TF (CTF) code. The single-executable coupled-code (Insilico-CTF) simulation tool is created using several VERA tools, including LIME (Lightweight Integrating Multiphysics Environment for coupling codes), DTK (Data Transfer Kit), Trilinos, and TriBITS. Parallel calculations are performed on the Titan supercomputer at Oak Ridge National Laboratory using 1156 cores, and a synopsis of the solution results and code performance is presented. Finally, ongoing development of this approach is also briefly described.

97 MATHEMATICS AND COMPUTING↗

A Multiple-Set Overlapping-Domain Decomposed Monte Carlo Synthetic Acceleration Method for Linear Systems

We present a multiple-set overlapping-domain decomposed strategy for parallelizing the Monte Carlo Synthetic Acceleration method. Monte Carlo Synthetic Acceleration methods use the Neumann-Ulam class of Monte Carlo solvers for linear systems to accelerate a fixed-point iteration sequence. Effective parallel algorithms for these methods require the parallelization of the underlying Neumann-Ulam solvers. To do this in a domain decomposed environment, we borrow strategies traditionally implemented in Monte Carlo particle transport to parallelize the problem. The parallel Neumann-Ulam and multiple-set overlapping-domain decomposition algorithms are presented along with parallel scaling data for the resulting implementation using the Titan Cray XK7 machine at Oak Ridge National Laboratory.

Computer Science↗