Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “block structured meshes”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 55 records · Page 3

A Moving Embedded Boundary Approach for the Compressible Navier-Stokes Equations in a Block-Structured Adaptive Refinement Framework

A computational technique has been developed to perform compressible flow simulations involving moving boundaries using an embedded boundary approach within the block-structured adaptive mesh refinement (SAMR) framework of AMReX [1], [91], [92]. We leverage the SAMR capability to obtain quantitatively accurate results whilst using robust, second-order finite volume schemes. A conservative, unsplit, cut-cell approach is utilized and a ghost-cell approach is developed for computing the flux on the moving, embedded boundary faces. A third-order least-squares formulation has been developed to compute the wall velocity gradients, and was found to significantly improve the performance of the solver in terms of the quantitative comparison of surface quantities such as the skin friction coefficient. Various test cases are performed to validate the method, and compared with analytical, experimental, and other numerical results in literature. Inviscid and viscous test cases are performed that span a wide regime of flow speeds - acoustic (harmonically pulsating sphere), smooth flows (expansion fan created by a receding piston) and flows with shocks (shock-cylinder interaction, shock-wedge interaction, pitching NACA 0012 airfoil and shock-cone interaction). A closed system with moving boundaries - an oscillating piston in a cylinder, showed that the percentage error in mass within the system decreases with refinement, demonstrating that the numerical scheme is conservative with grid refinement, but is not discretely conservative. Viscous test cases involve that of a horizontally moving cylinder at Re = 40, an inline oscillating cylinder at Re = 100, and a transversely oscillating cylinder at Re = 185. The judicious use of adaptive mesh refinement with appropriate refinement criteria to capture the regions of interest leads to well-resolved flow features, and good quantitative comparison is observed with the results available in literature.

adaptive refinement↗

Nyx: A MASSIVELY PARALLEL AMR CODE FOR COMPUTATIONAL COSMOLOGY

We present a new N-body and gas dynamics code, called Nyx, for large-scale cosmological simulations. Nyx follows the temporal evolution of a system of discrete dark matter particles gravitationally coupled to an inviscid ideal fluid in an expanding universe. The gas is advanced in an Eulerian framework with block-structured adaptive mesh refinement; a particle-mesh scheme using the same grid hierarchy is used to solve for self-gravity and advance the particles. Computational results demonstrating the validation of Nyx on standard cosmological test problems, and the scaling behavior of Nyx to 50,000 cores, are presented.

79 ASTRONOMY AND ASTROPHYSICS↗

Outflow Feedback Regulated Massive Star Formation in Parsec-Scale Cluster Forming Clumps

We investigate massive star formation in turbulent, magnetized, parsec-scale clumps of molecular clouds including protostellar outflow feedback using three dimensional numerical simulations of effective resolution 2048{sup 3}. The calculations are carried out using a block structured adaptive mesh refinement code that solves the ideal MHD equations including self-gravity and implements accreting sink particles. We find that, in the absence of regulation by magnetic fields and outflow feedback, massive stars form readily in a turbulent, moderately condensed clump of {approx} 1,600 M{sub {circle_dot}} (containing {approx} 10{sup 2} initial Jeans masses), along with a cluster of hundreds of lower mass stars. The massive stars are fed at high rates by (1) transient dense filaments produced by large-scale turbulent compression at early times, and (2) by the clump-wide global collapse resulting from turbulence decay at late times. In both cases, the bulk of the massive star's mass is supplied from outside a 0.1 pc-sized 'core' that surrounds the star. In our simulation, the massive star is clump-fed rather than core-fed. The need for large-scale feeding makes the massive star formation prone to regulation by outflow feedback, which directly opposes the feeding processes. The outflows reduce the mass accretion rates onto the massive stars by breaking up the dense filaments that feed the massive star formation at early times, and by collectively slowing down the global collapse that fuel the massive star formation at late times. The latter is aided by a moderate magnetic field of strength in the observed range (corresponding to a dimensionless clump mass-to-flux ratio {lambda} {approx} a few); the field allows the outflow momenta to be deposited more efficiently inside the clump. We conclude that the massive star formation in our simulated turbulent, magnetized, parsec-scale clump is outflow-regulated and clump-fed (ORCF for short). An important implication is that the formation of low-mass stars in a dense clump can affect the formation of massive stars in the same clump, through their outflow feedback on the clump dynamics. In a companion paper, we discuss the properties of the lower mass cluster members formed along with the massive stars, including their mass distribution and spatial clustering.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

A structured multi-block solution-adaptive mesh algorithm with mesh quality assessment

The dynamic solution adaptive grid algorithm, DSAGA3D, is extended to automatically adapt 2-D structured multi-block grids, including adaption of the block boundaries. The extension is general, requiring only input data concerning block structure, connectivity, and boundary conditions. Imbedded grid singular points are permitted, but must be prevented from moving in space. Solutions for workshop cases 1 and 2 are obtained on multi-block grids and illustrate both increased resolution of and alignment with the solution. A mesh quality assessment criteria is proposed to determine how well a given mesh resolves and aligns with the solution obtained upon it. The criteria is used to evaluate the grid quality for solutions of workshop case 6 obtained on both static and dynamically adapted grids. The results indicate that this criteria shows promise as a means of evaluating resolution.

Ingram, Clint L.↗

ENZO: AN ADAPTIVE MESH REFINEMENT CODE FOR ASTROPHYSICS

This paper describes the open-source code Enzo, which uses block-structured adaptive mesh refinement to provide high spatial and temporal resolution for modeling astrophysical fluid flows. The code is Cartesian, can be run in one, two, and three dimensions, and supports a wide variety of physics including hydrodynamics, ideal and non-ideal magnetohydrodynamics, N-body dynamics (and, more broadly, self-gravity of fluids and particles), primordial gas chemistry, optically thin radiative cooling of primordial and metal-enriched plasmas (as well as some optically-thick cooling models), radiation transport, cosmological expansion, and models for star formation and feedback in a cosmological context. In addition to explaining the algorithms implemented, we present solutions for a wide range of test problems, demonstrate the code's parallel performance, and discuss the Enzo collaboration's code development methodology.

79 ASTRONOMY AND ASTROPHYSICS↗

Adaptive Mesh Refinement Simulations for Turbulent Reacting Flow

With the increased availability of exascale computing hardware, detailed simulations of realistic devices can be performed at practically relevant time and length scales. Insights into the multiscale driving mechanisms in compressible reacting flow systems with complex geometry, such as combustors, can be used for design optimization and technology improvements. However, to effectively perform these simulations, advanced numerical algorithms must be used to maintain solution accuracy without incurring undue computational costs. PeleC, part of the Pele suite of codes, leverages block-structured adaptive mesh refinement (AMR) through the AMReX library to capture fine-scale flow features in compressible reacting flows. In this talk, we discuss recent improvements to the numerical algorithms, particularly in regard to describing flows at complex boundary structures, and PeleC's performance on exascale computing hardware. We will demonstrate that PeleC is well-suited for modern, extreme-scale, heterogenous compute platforms.

combustion↗

COSMOLOGICAL ADAPTIVE MESH REFINEMENT MAGNETOHYDRODYNAMICS WITH ENZO

In this work, we present EnzoMHD, the extension of the cosmological code Enzo to include the effects of magnetic fields through the ideal magnetohydrodynamics approximation. We use a higher order Godunov method for the computation of interface fluxes. We use two constrained transport methods to compute the electric field from those interface fluxes, which simultaneously advances the induction equation and maintains the divergence of the magnetic field. A second-order divergence-free reconstruction technique is used to interpolate the magnetic fields in the block-structured adaptive mesh refinement framework already extant in Enzo. This reconstruction also preserves the divergence of the magnetic field to machine precision. We use operator splitting to include gravity and cosmological expansion. We then present a series of cosmological and non-cosmological test problems to demonstrate the quality of solution resulting from this combination of solvers.

79 ASTRONOMY AND ASTROPHYSICS↗

MFIX-Exa: Performance prediction of multiphase energy conversion devices

MFIX-Exa targets the high-fidelity CFD-DEM model in which particles are unresolved by the fluid grid (typically using a fluid mesh approximately twice the particle diameter) but discrete particle dynamics including collisions are resolved with a simple linear-spring dashpot model. The fluid field is solved with a modern low Mach number formation of a cell-centered, nodal-pressure approximate projection method using a Godunov scheme . Physics capabilities include: complex geometries via embedded boundaries (EBs), open and closed system ideal gas equations of state, species transport and heterogeneous chemi- cal reactions. MFIX-Exa is built on the AMReX software framework, the ECP Block-Structured Adaptive Mesh Refinement (AMR) Co-Design Center, which allows the code to be portable and performant.Further integration into the ECP ecosystem includes HYPRE linear solvers and Ascent for in situ visualization. Current project efforts focus on scaling up realistic simulations to the KPP of the challenge problem: NETs 50kW chemical looping reactor, shakedown and scaling on JSLE and OLCF TDS systems and reaching for any remaining performance improvements.

Musser, Jordan↗

Early experiences on the OLCF Frontier system with AthenaPK and Parthenon–Hydro

The Oak Ridge Leadership Computing Facility (OLCF) has been preparing the nation's first exascale system, Frontier, for production and end users. Frontier is based on HPE Cray's new EX architecture and Slingshot interconnect and features 74 cabinets of optimized 3rd Gen AMD EPYC CPUs for HPC and AI and AMD Instinct 250X accelerators. As a part of this preparation, “real-world” user codes have been selected to help assess the functionality, performance, and usability of the system. This article describes early experiences using the system in collaboration with the Hamburg Observatory for two selected codes, which have since been adopted in the OLCF test harness. Experiences discussed include efforts to resolve performance variability and per-cycle slowdowns. Results are shown for a performance portable astrophysical magnetohydronamics code, AthenaPK, and a mini-application stressing the core functionality of a performance portable block-structured adaptive mesh refinement framework, Parthenon-Hydro. Here, these results show good scaling characteristics to the full system. At the largest scale, the Parthenon-Hydro miniapp reaches a total of $1.7$ $\times$ $10^{13}$ zone-cycles/s on 9216 nodes (73,728 logical GPUs) at ≈92% weak scaling parallel efficiency (starting from a single node using a second-order, finite-volume method).

97 MATHEMATICS AND COMPUTING↗

A hybrid adaptive multiresolution approach for the efficient simulation of reactive flows

Computational studies that use block-structured adaptive mesh refinement (AMR) approaches suffer from unnecessarily high mesh resolution in regions adjacent to important solution features. This deficiency limits the performance of AMR codes. In this work a novel hybrid adaptive multiresolution (HAMR) approach to AMR-based calculations is introduced to address this issue. The multiresolution (MR) smoothness indicators are used to identify regions of smoothness on the mesh where the computational cost of individual physics solvers may be decreased by replacing direct calculations with interpolation. We suggest an approach to balance the errors due to the adaptive discretization and the interpolation of physics quantities such that the overall accuracy of the HAMR solution is consistent with that of the MR-driven AMR solution. The performance of the HAMR scheme is evaluated for a range of test problems, from pure hydrodynamics to turbulent combustion.

97 MATHEMATICS AND COMPUTING↗

CG-Kit: Code Generation Toolkit for performant and maintainable variants of source code applied to Flash-X hydrodynamics simulations

CG-Kit is a new Code Generation tool-Kit that we have developed as a part of the solution for portability and maintainability for multiphysics computing applications. The development of CG-Kit is rooted in the urgent need created by the shifting landscape of high-performance computing platforms and the algorithmic complexities of a particular large-scale multiphysics application: Flash-X. To efficiently use computing resources on a heterogeneous node, an application must have a map of computation to resources and a mechanism to move the data and computation to the resources according to the map. Most existing performance portability solutions are focussed on abstracting the expression of computations so that a unified source code can be specialized to run on different resources. However, such an approach is insufficient for a code like Flash-X, which has a multitude of code components that can be assembled in various permutations and combinations to form different instances of applications. Similar challenges apply to any code that has composability, where a single specified way of apportioning work among devices may not be optimal. Additionally, use cases arise where the optimal control flow of computation may differ for different devices while the underlying numerics remain identical. This combination leads to unique challenges including handling an existing large code base in Fortran and/or C/C++, subdivision of code into a great variety of units supporting a wide range of physics and numerical methods, different parallelization techniques for distributed and shared memory systems and accelerator devices, and heterogeneity of computing platforms requiring coexisting variants of parallel algorithms. All of these challenges demand that scientific software developers apply existing knowledge about domain applications, algorithms, and computing platforms to determine custom abstractions and granularity for code generation. There is a critical lack of tools to tackle those problems. CG-Kit is designed to fill this gap by providing a user with the ability to express their desired control flow and computation-to-resource map in the form a pseudocode-like recipe. It consists of standalone tools that can be combined into highly specific and, we argue, highly effective portability and maintainability toolchains. Here we present the design of our new tools: parametrized source trees, control flow graphs, and recipes. The tools are implemented in Python. They are agnostic to the programming language of the source code targeted for code generation. In conclusion, we demonstrate the capabilities of the toolkit with two examples, first, multithreaded variants of the basic AXPY operation, and second, variants of parallel algorithms within a hydrodynamics solver, called Spark, from Flash-X that operates on block-structured adaptive meshes.

Algorithmic portability↗

NUMERICAL SIMULATIONS OF WIND ACCRETION IN SYMBIOTIC BINARIES

About half of the binary systems are close enough to each other for mass to be exchanged between them at some point in their evolution, yet the accretion mechanism in wind accreting binaries is not well understood. We study the dynamical effects of gravitational focusing by a binary companion on winds from late-type stars. In particular, we investigate the mass transfer and formation of accretion disks around the secondary in detached systems consisting of an asymptotic giant branch (AGB) mass-losing star and an accreting companion. The presence of mass outflows is studied as a function of mass-loss rate, wind temperature, and binary orbital parameters. A two-dimensional hydrodynamical model is used to study the stability of mass transfer in wind accreting symbiotic binary systems. In our simulations we use an adiabatic equation of state and a modified version of the isothermal approximation, where the temperature depends on the distance from the mass losing star and its companion. The code uses a block-structured adaptive mesh refinement method that allows us to have high resolution at the position of the secondary and resolve the formation of bow shocks and accretion disks. We explore the accretion flow between the components and formation of accretion disks for a range of orbital separations and wind parameters. Our results show the formation of stream flow between the stars and accretion disks of various sizes for certain orbital configurations. For a typical slow and massive wind from an AGB star the flow pattern is similar to a Roche lobe overflow with accretion rates of 10% of the mass loss from the primary. Stable disks with exponentially decreasing density profiles and masses of the order 10{sup -4} solar masses are formed when wind acceleration occurs at several stellar radii. The disks are geometrically thin with eccentric streamlines and close to Keplerian velocity profiles. The formation of tidal streams and accretion disks is found to be weakly dependent on the mass loss from the AGB star. Our simulations of gravitationally focused wind accretion in symbiotic binaries show the formation of stream flows and enhanced accretion rates onto the compact component. We conclude that mass transfer through a focused wind is an important mechanism in wind accreting interacting binaries and can have a significant impact on the evolution of the binary itself and the individual components.

99 GENERAL AND MISCELLANEOUS↗

CASTRO: A NEW COMPRESSIBLE ASTROPHYSICAL SOLVER. II. GRAY RADIATION HYDRODYNAMICS

We describe the development of a flux-limited gray radiation solver for the compressible astrophysics code, CASTRO. CASTRO uses an Eulerian grid with block-structured adaptive mesh refinement based on a nested hierarchy of logically rectangular variable-sized grids with simultaneous refinement in both space and time. The gray radiation solver is based on a mixed-frame formulation of radiation hydrodynamics. In our approach, the system is split into two parts, one part that couples the radiation and fluid in a hyperbolic subsystem, and another parabolic part that evolves radiation diffusion and source-sink terms. The hyperbolic subsystem is solved explicitly with a high-order Godunov scheme, whereas the parabolic part is solved implicitly with a first-order backward Euler method.

79 ASTRONOMY AND ASTROPHYSICS↗

CASTRO: A NEW COMPRESSIBLE ASTROPHYSICAL SOLVER. III. MULTIGROUP RADIATION HYDRODYNAMICS

We present a formulation for multigroup radiation hydrodynamics that is correct to order O(v/c) using the comoving-frame approach and the flux-limited diffusion approximation. We describe a numerical algorithm for solving the system, implemented in the compressible astrophysics code, CASTRO. CASTRO uses a Eulerian grid with block-structured adaptive mesh refinement based on a nested hierarchy of logically rectangular variable-sized grids with simultaneous refinement in both space and time. In our multigroup radiation solver, the system is split into three parts: one part that couples the radiation and fluid in a hyperbolic subsystem, another part that advects the radiation in frequency space, and a parabolic part that evolves radiation diffusion and source-sink terms. The hyperbolic subsystem and the frequency space advection are solved explicitly with high-order Godunov schemes, whereas the parabolic part is solved implicitly with a first-order backward Euler method. Our multigroup radiation solver works for both neutrino and photon radiation.

79 ASTRONOMY AND ASTROPHYSICS↗

Meeting the Challenges of Modeling Astrophysical Thermonuclear Explosions: Castro, Maestro, and the AMReX Astrophysics Suite

We describe the AMReX suite of astrophysics codes and their application to modeling problems in stellar astrophysics. Maestro is tuned to efficiently model subsonic convective flows while Castro models the highly compressible flows associated with stellar explosions. Both are built on the block-structured adaptive mesh refinement library AMReX. Together, these codes enable a thorough investigation of stellar phenomena, including Type Ia supernovae and X-ray bursts. We describe these science applications and the approach we are taking to make these codes performant on current and future many-core and GPU-based architectures.

79 ASTRONOMY AND ASTROPHYSICS↗

On Using Linux Kernel Huge Pages with FLASH, an Astrophysical Simulation Code

We present efforts at improving the performance of FLASH, a multi-scale, multi-physics simulation code principally for astrophysical applications, by using huge pages on Ookami, an HPE Apollo 80 A64FX platform. FLASH is written principally in modern Fortran and makes use of the PARAMESH library to manage a block-structured adaptive mesh. We explored options for enabling the use of huge pages with several compilers, but we were only able to successfully use huge pages when compiling with the Fujitsu compiler. As a result, the use of huge pages substantially reduced the number of translation lookaside buffer misses, but overall performance gains were marginal.

79 ASTRONOMY AND ASTROPHYSICS↗

Emu v1.1

Emu is a particle-in-cell code for solving the neutrino quantum kinetic equations in 1, 2, or 3 spatial dimensions with arbitrary angular resolution in order to simulate neutrino flavor transformation in neutron star merger and core collapse supernova environments. Emu represents the neutrino distribution function as a set of particles, each of which represent a collection of neutrinos and antineutrinos with unique position and momentum. Each particle carries two density matrices to define the flavor state of the neutrinos and antineutrinos it represents. Emu includes the vacuum, matter, and neutrino self-interaction potentials. Emu calculates the self-interaction potential using PIC deposition and interpolation algorithms that efficiently compute the local number density and flux at particle locations. Emu is implemented in C++ and is based on the AMReX library for high-performance, block-structured adaptive mesh refinement. Emu is parallelized with MPI + OpenMP for CPUs and MPI + CUDA for GPUs.

Willcox, Donald↗

mesoflow [SWR-22-56]

Mesoflow is a continuum scale simulation tool developed specifically for modeling transport and chemistry at the mesoscale. Our solver utilizes Cartesian block-structured adaptive mesh refinement to resolve complex surface morphologies (of catalysts/biomass particles among others) directly obtained from X-ray tomography data. An immersed boundary based formulation enables rapid representation of complex geometries prevalent in most mesoporous interfaces. The solver is developed on top of open-source performance portable library, AMReX, providing parallel execution capabilities on current and upcoming high-performance-computing (HPC) architectures. Our flexible software framework enables integration of complex chemical mechanisms at heterogenous interfaces and time-split algorithms for circumventing highly disparate reaction and flow time-scales. Our current studies indicate a ten-fold performance gain by using graphics-processing-units (GPU) compared to a single processor for representative problem sizes (2 million cell mesh).

Sitaraman, Hariswaran↗