Engineering Papers⌕ Search

Engineering topics

Bell, John B.

Publications and source records attributed to Bell, John B..

The Pele Simulation Suite for Reacting Flows at Exascale

In this work, we present the Pele suite of software tools for compressible and incompressible reacting flows. The Pele suite leverages several different libraries, notably AMReX and SUNDIALS, to achieve performance portability on heterogeneous computing architectures across the supercomputing landscape. The Pele suite is comprised of PeleC, a compressible reacting flow block-structured adaptive mesh refinement solver, PeleLMeX, a low-Mach number reacting flow block-structured adaptive mesh refinement solver, Pele-Physics, a library for transport, thermodynamics, finite rate chemistry, soot, spray and radiation physics. The objective of this paper is (i) to present the code development efforts necessary to achieve highly effective and scalable applications for exascale machines and (ii) to detail the performance results of the Combustion-Pele project applications on Oak Ridge National Laboratory's Frontier. We show good weak and strong scaling results for both PeleC and PeleLMeX up to more than 50 billion cells on more than 4096 Frontier graphics processing unit nodes. We also present a capability demonstration simulation of a dual-fuel pulse compression ignition engine (six adaptive mesh refinement levels, and 60 billion cells or 2.1 trillion degrees of freedom) on Frontier, to date one of the largest simulations performed on the first exascale-class supercomputer.

adaptive mesh refinement↗

Surface coverage dynamics for reversible dissociative adsorption on finite linear lattices

Dissociative adsorption onto a surface introduces dynamic correlations between neighboring sites not found in non-dissociative absorption. We study surface coverage dynamics where reversible dissociative adsorption of dimers occurs on a finite linear lattice. We derive analytic expressions for the equilibrium surface coverage as a function of the number of reactive sites, N, and the ratio of the adsorption and desorption rates. Using these results, we characterize the finite size effect on the equilibrium surface coverage. For comparable N’s, the finite size effect is significantly larger when N is even than when N is odd. Moreover, as N increases, the size effect decays more slowly in the even case than in the odd case. The finite-size effect becomes significant when adsorption and desorption rates are considerably different. These finite-size effects are related to the number of accessible configurations in a finite system where the odd-even dependence arises from the limited number of accessible configurations in the even case. We confirm our analytical results with kinetic Monte Carlo simulations. We also analyze the surface-diffusion case where adsorbed atoms can hop into neighboring sites. As expected, the odd-even dependence disappears because more configurations are accessible in the even case due to surface diffusion.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

Fluctuating hydrodynamics and the Rayleigh–Plateau instability

The Rayleigh–Plateau instability occurs when surface tension makes a fluid column become unstable to small perturbations. At nanometer scales, thermal fluctuations are comparable to interfacial energy densities. Consequently, at these scales, thermal fluctuations play a significant role in the dynamics of the instability. These microscopic effects have previously been investigated numerically using particle-based simulations, such as molecular dynamics (MD), and stochastic partial differential equation–based hydrodynamic models, such as stochastic lubrication theory. In this paper, we present an incompressible fluctuating hydrodynamics model with a diffuse-interface formulation for binary fluid mixtures designed for the study of stochastic interfacial phenomena. An efficient numerical algorithm is outlined and validated in numerical simulations of stable equilibrium interfaces. We present results from simulations of the Rayleigh–Plateau instability for long cylinders pinching into droplets for Ohnesorge numbers of Oh = 0.5 and 5.0. Both stochastic and perturbed deterministic simulations are analyzed and ensemble results show significant differences in the temporal evolution of the minimum radius near pinching. Short cylinders, with lengths less than their circumference, were also investigated. As previously observed in MD simulations, we find that thermal fluctuations cause these to pinch in cases where a perturbed cylinder would be stable deterministically. Finally, we show that the fluctuating hydrodynamics model can be applied to study a broader range of surface tension–driven phenomena.

97 MATHEMATICS AND COMPUTING↗

Staggered scheme for the compressible fluctuating hydrodynamics of multispecies fluid mixtures

Here, we present a numerical formulation for the solution of nonisothermal, compressible Navier-Stokes equations with thermal fluctuations to describe mesoscale transport phenomena in multispecies fluid mixtures. The novelty of our numerical method is the use of staggered grid momenta along with a finite volume discretization of the thermodynamic variables to solve the resulting stochastic partial differential equations. The key advantages of the numerical scheme are that it significantly simplifies the discretization of diffusive and stochastic momentum fluxes into a more compact form, and it provides an unambiguous prescription of boundary conditions involving pressure. The staggered grid scheme more accurately reproduces the equilibrium static structure factor of hydrodynamic fluctuations in gas mixtures compared to a collocated scheme described previously by Balakrishnan et al. [Phys. Rev. E 89, 013017 (2014)1539-375510.1103/PhysRevE.89.013017]. The numerical method is tested for ideal noble gases mixtures under various nonequilibrium conditions, such as applied thermal and concentration gradients, to assess the role of cross-diffusion effects, such as Soret and Dufour, on the long-ranged correlations of hydrodynamic fluctuations, which are also more accurately reproduced compared to the collocated scheme. We numerically study giant nonequilibrium fluctuations driven by concentration gradients and fluctuation-driven Rayleigh-Taylor instability in gas mixtures. Wherever applicable, excellent agreement is observed with theory and measurements from the direct simulation Monte Carlo method.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

PeleC: An adaptive mesh refinement solver for compressible reacting flows

Reacting flow simulations for combustion applications require extensive computing capabilities. Leveraging the AMReX library, the Pele suite of combustion simulation tools targets the largest supercomputers available and future exascale machines. We introduce PeleC, the compressible solver in the Pele suite, and detail its capabilities, including complex geometry representation, chemistry integration, and discretization. We present a comparison of development efforts using both OpenACC and AMReX’s C++ performance portability framework for execution on multiple GPU architectures. We discuss relevant details that have allowed PeleC to achieve high performance and scalability. PeleC’s performance characteristics are measured through relevant simulations on multiple supercomputers. The success of PeleC’s design for exascale is exhibited through demonstration of a 160 billion cell simulation and weak scaling onto 100% of Summit, an NVIDIA-based GPU supercomputer at Oak Ridge National Laboratory. Our results provide confidence that PeleC will enable future combustion science simulations with unprecedented fidelity.

97 MATHEMATICS AND COMPUTING↗

A weighted state redistribution algorithm for embedded boundary grids

State redistribution is an algorithm that stabilizes cut cells for embedded boundary grid methods. This work extends the earlier algorithm in several important ways. First, state redistribution is extended to three spatial dimensions. Second, we discuss several algorithmic changes and improvements motivated by the more complicated cut cell geometries that can occur in higher dimensions. In particular, we introduce a weighted version with less dissipation in an easily generalizable framework. Third, we demonstrate that state redistribution can also stabilize a solution update that includes both advective and diffusive contributions. Notably, the stabilization algorithm is shown to be effective for incompressible as well as compressible reacting flows. Finally, we discuss the implementation of the algorithm for several exascale-ready simulation codes based on AMReX, demonstrating ease of use in combination with domain decomposition, hybrid parallelism and complex physics.

97 MATHEMATICS AND COMPUTING↗

Thermal fluctuations in the dissipation range of homogeneous isotropic turbulence

In this work, using fluctuating hydrodynamics we investigate the effect of thermal fluctuations in the dissipation range of homogeneous isotropic turbulence. Simulations confirm theoretical predictions that the energy spectrum is dominated by these fluctuations at length scales comparable to the Kolmogorov length. We also find that the extreme intermittency in the far-dissipation range predicted by Kraichnan is replaced by Gaussian thermal equipartition.

42 ENGINEERING↗

A coupled discontinuous Galerkin-Finite Volume framework for solving gas dynamics over embedded geometries

Herein, we present a computational framework for solving the equations of inviscid gas dynamics using structured grids with embedded geometries. The novelty of the proposed approach is the use of high-order discontinuous Galerkin (dG) schemes and a shock-capturing Finite Volume (FV) scheme coupled via an hp adaptive mesh refinement (hp-AMR) strategy that offers high-order accurate resolution of the embedded geometries. The hp-AMR strategy is based on a multi-level block-structured domain partition in which each level is represented by block-structured Cartesian grids and the embedded geometry is represented implicitly by a level set function. The intersection of the embedded geometry with the grids produces the implicitly-defined mesh that consists of a collection of regular rectangular cells plus a relatively small number of irregular curved elements in the vicinity of the embedded boundaries. High-order quadrature rules for implicitly-defined domains enable high-order accuracy resolution of the curved elements with a cell-merging strategy to address the small-cell problem. The hp-AMR algorithm treats the system with a second-order finite volume scheme at the finest level to dynamically track the evolution of solution discontinuities while using dG schemes at coarser levels to provide high-order accuracy in smooth regions of the flow. On the dG levels, the methodology supports different orders of basis functions on different levels. The space-discretized governing equations are then advanced explicitly in time using high-order Runge-Kutta algorithms. Numerical tests are presented for two-dimensional and three-dimensional problems involving an ideal gas. The results are compared with both analytical solutions and experimental observations and demonstrate that the framework provides high-order accuracy for smooth flows and accurately captures solution discontinuities.

97 MATHEMATICS AND COMPUTING↗

An Adaptive Semi-Implicit Scheme for Simulations of Unsteady Viscous Compressible Flows

A numerical scheme for simulation of unsteady, viscous, compressible flows is considered. The scheme employs an explicit discretization of the inviscid terms of the Navier-Stokes equations and an implicit discretization of the viscous terms. The discretization is second order accurate in both space and time. Under appropriate assumptions, the implicit system of equations can be decoupled into two linear systems of reduced rank. These are solved efficiently using a Gauss-Seidel method with multigrid convergence acceleration. When coupled with a solution-adaptive mesh refinement technique, the hybrid explicit-implicit scheme provides an effective methodology for accurate simulations of unsteady viscous flows. The methodology is demonstrated for both body-fitted structured grids and for rectangular (Cartesian) grids.

Steinthorsson, Erlendur↗