Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Parallel Fluid Dynamic Solver”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

135 records · Page 8
All sourcesNASA NTRSDOE OSTI
DOE OSTI2024

Scalable Multiphysics Block Preconditioning for Low Mach Number Compressible Resistive MHD with Application to Magnetic Confinement Fusion

This study investigates multiphysics block preconditioners that are critical in devising scalable Newton–Krylov iterative solvers for longer time-scale fully implicit fluid plasma models. The specific model of interest is the visco-resistive, low Mach number, compressible magnetohydrodynamics (MHD) model. This model describes the dynamics of conducting fluids in the presence of electromagnetic fields and can be used to study aspects of astrophysical phenomena, important science and technology applications, and basic plasma physics. The specific application of interest that motivates this study is the macroscopic simulation of longer time-scale stability and disruptions of magnetic confinement fusion devices, specifically the ITER Tokamak. The computational solution of the governing balance equations for mass, momentum, heat transfer, and magnetic induction for resistive MHD systems can be extremely challenging. These difficulties arise from both the strong nonlinear, nonsymmetric coupling of fluid and electromagnetic phenomena as well as the significant range of time and length scales that the interactions of these physical mechanisms produce. To handle the range of time and spatial scales of interest, a fully implicit unstructured variational multiscale finite element formulation is employed. For the scalable solution of the Newton linearized systems, fully coupled block preconditioners are designed to leverage algebraic multigrid subsolves. In conclusion, results are presented for the strong and weak scaling of the method as well as the robustness of these techniques for a large range of Lundquist numbers.

97 MATHEMATICS AND COMPUTING↗
NASA NTRS2016

Simulation of Sweep-Jet Flow Control, Single Jet and Full Vertical Tail

This work is a simulation technology demonstrator, of sweep jets used to suppress boundary layer separation and increase maximum achievable load coefficients. A sweep jet is a discrete Coanda jet that oscillates in the plane parallel to an aerodynamic surface. It injects mass and momentum in the approximate stream wise direction. It also generate turbulent eddies at the oscillation frequency, which are typically large relative to boundary layer turbulence, and which augmenting mixing across the boundary layer to attack flow separation. Simulations of a fluidic oscillator, the sweep jet emerging from the oscillator, and the suppression of boundary layer separation by an array of sweep jets are performed. Simulation results are compared to data from a dedicated CFD validation experiment of a single oscillator and its sweep jet, and from a study of a full-scale Boeing 757 vertical tail augmented with an array of sweep jets.2, 20 A critical step in the work is the development of realistic time-dependent sweep-jet in flow boundary conditions, derived from the results of the single-oscillator simulations, which create the sweep jets in the full-tail simulations. Simulations were performed using the Over flow CFD solver, with high-order spatial discretization and a range of turbulence modeling. Good results were obtained for all flows simulated, when suitable turbulence modeling was used.

flow control↗
DOE OSTI2022

kynema-fmb [SWR-23-07]

Kynema-FMB (FKA: Kynema) is an open-source performance portable flexible multibody (FMB) dynamics solver designed for time-domain simulations. While originally tailored for wind turbine structural dynamics, the formulation and implementation are those of a general flexible-multidbody dynamics solver that can readily be applied to a wide range of systems. Kynema was designed with a narrow focus, namely to provide a lightweight, fast, accurate FMD solver for coupling to computational-fluid-dynamics (CFD) codes, especially the CFD codes in the Kynema suite, for fluid-structure-interaction (FSI) simulations. Kynema-FMB is equipped to model systems that can be represented as a collection of beams and rigid bodies that are connected through constraints. Degrees of freedom are defined in the inertial/global frame of reference and include displacements and rotations (formally as rotation matrices, but stored as quaternions). The underlying formulation is built on a Lie-group time integrator designed for index-3 differential-algebraic equations, which is second-order accurate in time (Bruls et al., 2012). Beam models are based on geometrically exact beam theory and are discretized as high-order spectral finite elements similar to those in BeamDyn (Wang et al., 2017). The governing equations for a FMD system like a wind turbine constitute a highly nonlinear system of constrained partial-differential equations. Kynema-FMB uses analytical Jacobians in the nonlinear-system solves in each time step. Linear systems use sparse storage and several third-party sparse-linear-system solvers are enabled. Ill conditioning of linear systems is mitigated with preconditioning described in Bottasso et al, 2008. Kynema-FMB is integrated with a simple open-source controller (ROSCO). There is an application programming interface (API) for coupling to geometry-resolved CFD (like that in Sharma et al., 2023) and actuator-force CFD (like that in Kuhn et al., 2025). In the latter, for actuator-line models, Kynema-FMB includes an internal blade-element solver that depends on user-provided lookup tables for coefficients of lift and drag, i.e., aerodynamic polars. Kynema-FMB is written in C++ and leverages Kokkos and Kokkos-Kernels (KokkosEcosystem) as its performance portability layer enabling simulations on both CPU and GPU systems. The repository is equipped with extensive automated testing at the unit and regression/system levels. The following describes the high-level development objectives conceived for Kynema: *Kynema will follow modern software development best practices, including test-driven development (TDD), version control, hierarchical automated testing, and continuous integration (CI) for a robust development environment. *The core data structures are memory efficient and enable vectorization and parallelization at multiple levels. *Data structures are data-oriented to exploit methods for accelerated computing including high utilization of chip resources (e.g., single instruction multiple data (SIMD) instruction sets) and parallelization using GP-GPUs. *The computational algorithms incorporate robust open-source libraries for mathematical operations, resource allocation, and data management. *The API design considers multiple stakeholder needs and ensure integration with existing and future ecosystems for data science, machine learning, and AI. *Kynema-FMB is written in modern C++ and leverages Kokkos as its performance-portability library with inspiration from the kynema stack.

Sprague, MichaelA.↗
NASA NTRS1994

Accelerated panel methods using the fast multipole method

Panel methods are commonly used in computational fluid dynamics for the solution of potential flow problems. The methods are a numerical technique based on the surface distribution of singularity elements. The solution is the process of finding the strength of the singularity elements distributed over the body's surface. This process involves the solution of the matrix problem Pq = p' for a set of unknowns q. The Fast Multipole Method is used to directly compute q without using matrix solvers. The algorithm works in O(N) time for N points, a great improvement over standard matrix solvers. In panel methods, the surface of a body is divided into a series of quadrilateral panels. The methods involve the computation of the influence of all other panels on each individual panel. The influence is based on the surface distribution, though this can be approximated by the area for distant panels. An alternative approximation, though with arbitrary accuracy, is to develop a multipole expansion about the center of the panel to describe the effect of a given panel on distant points in space. The expansion is based on the moments of the panel, thus allow the use of various surface distributions without changing the basic algorithm, just the computation of the various moments. The expansions are then manipulated in a tree walk to develop Taylor series expansions about a point in space which describe the effect of all distant panels on any point within a volume of convergence. The effect of near panels then needs to be computed directly, but the effect of all distant panels can be computed by simply evaluating the resulting expansion. The Fast Multipole Method has been applied to panel methods for the solution of source and doublet distributions. A major feature of the algorithm is that the algorithm does not change to derive the potential and velocity for sources and doublets. The same expansions can be used for both sources and doublets. Since the velocity is related to the potential, and the doublet potential is related to the z-component of the source velocity, all values can be derived from the same expansion by taking a series of partial derivatives. This requires more expansion terms to be kept since terms are lost in the process of taking partial derivatives. Thus to maintain accuracy for the doublet computation, more terms are required than if just evaluating for sources. The resulting Fast Multipole code should then parallelize better than classical panel methods due to the locality of data dependencies found in the Fast Multipole Method. Theoretically the parallelized code should execute in O(log N) time with O(N) processors, though this is not practical. Ongoing work includes implementing the parallel accelerated panel method, including methods to improve the load balancing of the problem by taking advantage of the known geometry of panels, and to encorporate sensitivity analysis into the algorithm.

Leathrum, James F., Jr.↗
NASA NTRSDate not supplied

Particle Interaction Physics Model Formulation for Plume-Surface Interaction Erosion and Cratering

As part of the Game Changing Development (GCD) Program, funded by NASA’s Space Technology Mission Directorate (STMD), the development of simulation capability for the prediction of extra-terrestrial Plume Surface Interaction (PSI) environments has been undertaken by the Fluid Dynamics Branch at NASA/MSFC. The Predictive Simulation Capability (PSC) Element is focused on creating simulation capability for the reliable and accurate prediction of PSI in Martian (~650 Pa) and Lunar (vacuum) ambient environments. In addition to the predictive simulation capability, the GCD Program also contains a companion Ground Testing Element for development of focused datasets for validation of predictive capability as well as a Flight-focused Instrumentation Element. This paper will present the status of implementing and maturing particle-particle interaction constituent physics models essential in simulating the landing surface granular material flow under PSI effects. This gas-particle multi-phase interaction modeling of plume impingement flow on the extra-terrestrial soil material is performed with the Gas-Granular Flow Solver (GGFS) addressed in a companion paper. The response of regolith particle flow induced by lander PSI requires accurate representation of the regolith granular material fluidic behavior and gas-granular interactions. The lunar regolith, as the extreme example, is poorly sorted with broad particle size distributions and large fines content. It has significant cohesion, due to interlocking particle shapes for the very jagged particles. The combination of particle shape and size distribution has been identified as major drivers in the complex particle flow response and resulting crater shape characteristics of extraterrestrial granular material. Constituent models for spherical particles can be formulated directly from particle kinetics theory. Complex particle shapes can be modeled by gluing together elemental spherical shapes into composite particles, requiring a Discrete Element Model (DEM) particle kinetics modeling approach to extract data and formulate constituent models. Mixture constituent models for poly-disperse mixtures (i.e, containing distribution of particle sizes) have recently been developed. The required non-spherical particle mixture granular material response closure models are then obtained through small-scale unit physics DEM simulations for the range of particle shapes, mixtures and packing densities. The granular material response closure models are then implemented in the Eulerian granular flow formulation. This DEM-based constituent model extraction process and formulation of poly-disperse particle mixtures has been successfully developed by small business and academic partners in the development of the Gas-Granular Flow Solver (GGFS) simulation program simulation framework. The currently implemented capabilities have reached the capability level of modeling bi-disperse, non-spherical particle mixtures is being continuously extended towards computational modeling of full range irregular particle mixtures. Under the GCD project, this technology is being further developed, transferred to NASA analysts, and matured towards application readiness. The predictive simulation capability team under the GCD project has acquired the modeling tools and processes of the DEM based constituent model formulation from the GGFS development team and is developing the capability to replicate the existing process. This is the first important step towards the ability of the NASA team to independently perform such model development in a production setting. Further efforts are underway to migrate the DEM based model simulation process performed with the academic based tools to more capable Open Source, highly parallelized simulation tools for efficient operation on NASA HPC assets. Evaluation of the currently implemented (such as mono-disperse and bi-disperse spherical and irregular shape particle constituent model applications) and continuously evolving full-range particle physics models in the GGFS tool is performed by the NASA team to advance application readiness of the simulations. Application testing for complex PSI erosions and cratering scenarios such as the Apollo LM is performed for axi-symmetric and full 3D simulations to aid the tool developers in achieving practical application readiness for NASA projects. Important validation and application testing will further be performed against experimental data generated under the GCD PSI project experimental component.

Peter A Liever↗
DOE OSTI2016

Single-Bubble Flow Boiling Phenomena - Interface Tracking Simulation

Boiling, as one of the most efficient heat transfer mechanisms, is widely used in various engineering systems. Better understanding and modeling of this process remains a major challenge in multiphase flow research. The distribution of vapor in boiling system affects the heat transfer rate and may cause unfavorable conditions, such as heater burn-out. A number of flow regimes have been observed experimentally. In general, there are three regimes under boiling conditions: nucleate boiling, transition boiling, and film boiling. The nucleate boiling regime is categorized into partial boiling regime and fully developed nucleate boiling, according to behavior of bubble dynamics and heat transfer characteristics. In the first region, as the wall temperature increases, the fraction of the wall surface subject to nucleate boiling increases until bubble formation occupies the entire heated surface. Heat transfer in this region is a complex mixture of single-phase forced convection and nucleate boiling. As bubble density increases, bubbles turn to coalesce and form insulating vapor patches, which impedes liquid flow back to the surface. The heat transfer rate under this condition reduces dramatically. This point is called Critical Heat Flux (CHF), which leads to boiling crisis. The behavior of boiling crisis is dependent on local fluid conditions due to bubble growth and departure. Experiments are the ultimate tool for boiling model development and verification, but they still lack some capabilities needed for better understanding of the local mechanism of this phenomena. It's difficult to capture many important details at the pressure/temperature conditions of interest using experimental techniques, such as full three dimensional measurements of velocity and temperature distribution and estimates of heat flux partitioning during the boiling process. The interface tracking simulation (ITS) is one of the promising approaches to describe heat transfer of boiling phenomena and their underlying mechanisms. The advances of high performance computing (HPC) in recent years made it possible to apply DNS to a wide variety of adiabatic flows. Some work has been done to implement evaporation / condensation models as well. Lee and Nydahl performed a numerical simulation of bubble growth and departure assuming that the bubble has the shape of a truncated sphere and a micro-layer exists during bubble growth. A 2-D cylindrical bubbles simulation has been done by Son with a simplified model to predict the heat flux in a thin liquid micro-layer. A direct simulation of 2D film boiling is conducted by Juric and Tryggvason. The film boiling, below critical pressure with a modified Level Set Method, is studied by Son and Dhir. Ose and Kunugi have developed a boiling and condensation model on subcooled boiling phenomena. Fuchs performed numerical simulations of bubble growth and departure considering wall conduction effects. Fully transient heat and fluid flow is modeled with a free surface of a rising bubble and a periodic calculation of the whole cycling of a growing, detaching, and rising bubble. Mukherjee and Kandikar have done a numerical simulation of an evaporating meniscus on a moving heated surface with level set method. The heat transfer associated with the advancing and receding contact angle has been studied. To develop the capability of physics-based boiling process, in the presented work the energy and momentum equations are coupled to simulate single-bubble growth scenarios. For a single bubble simulation in infinite liquid, the analytical solution exists and can be compared with the numerical results for verification purposes. The presented capabilities will be utilized in the large scale DNS of boiling flows in the future. The finite-element based code, PHASTA is used for the simulation. It is a parallel, hierarchic, higher-order accurate, adaptive, stabilized, transient analysis flow solver, which has been shown to be an effective tool for a wide range of single and two-phase flow problems. It has been shown to be highly scalable on top supercomputers ({approx}85% scaling on up to 3 million mesh partitions using a 92 billion element mesh . PHASTA has been shown to reliably predict the details of adiabatic bubbly flows as well as single-phase flows with heat transfer. To develop the capability of physics-based boiling process, in the presented work the energy and momentum equations are coupled to simulate a single-bubble growth scenario. For a single bubble growth in superheated water, the analytical solution exists and can be compared with the numerical results for verification purposes. The second-order accurate spatial and time discretization is used for the flow solver in the presented work. (authors)

22 GENERAL STUDIES OF NUCLEAR REACTORS↗
DOE OSTI2015

Partnership for Edge Physics Simulation (EPSI)

We propose to develop advanced simulation codes, based upon an extreme parallelism, first principles kinetic approach, to address the challenges associated with the edge region of magnetically confined plasmas. This work is relevant to both existing magnetic fusion facilities and essential for next-generation burning plasma experiments, such as ITER where success is critically dependent upon H-mode operation achieving an edge pedestal of sufficient height for good core plasma performance without producing deleterious large scale edge localized instabilities. The plasma edge presents a well-known set of multi-physics, multi-scale problems involving complex 3D magnetic geometry. Perhaps the greatest computational challenge is the lack of scale separation – temporal scales for drift waves, Alfven waves, ELM dynamics for example have strong overlap. Similar overlap occurs on the spatial scales for the ion poloidal gyro-radius, drift wave and pedestal width. The traditional approach of separating fusion problems into weakly interacting spatial or temporal domains clearly breaks down in the edge. A full kinetic model (full-f model) must be solved to understand and predict the edge physics including non-equilibrium thermodynamic issues arising from the magnetic topology (the open field lines producing a spatially sensitive velocity hole), plasma wall interactions, neutral and atomic physics. The plan here is to model these phenomena within a comprehensive first principles set of equations without the need for the insurmountable multiple-codes coupling issues by building on the XGC1 code developed under the SciDAC Proto-FSP Center for Plasma Edge Simulation (CPES). This proposal includes the critical participants in the XGC1 development. We propose enhancing the capability of XGC1 by including all the important turbulence physics contained in kinetic ion and electron electromagnetic dynamics, by extending the PIC technology to incorporate several positive features found in the Eulerian grid technology, combining the coarse grained XGC0 code into it with a multiscale projective time integration technique to be developed in the project for experimental time scale simulation of the edge multi-physics, and by implementing modern computational technologies to move forward with the extreme scale heterogeneous hardware/software platform development. Ultra-fast edge localized instabilities, involving compressional Alven modes, will be studied using a proper MHD or two-fluid codes employing kinetic closure information from XGC. The proposed research integrates the most important physical processes on overlapping temporal and spatial scales, to study (i) transitions and thresholds for enhanced confinement regimes, (ii) the predictive understanding of the edge pedestal formation, structure and dynamics, (iii) the effect of the edge plasma on the core confinement, (iv) the physics of Edge Localized Modes and their suppression or mitigation via external control techniques, and (v) the heat load on the material wall. Our approach will implement physical models that are valid in their relevant collisionality regimes for current experiments and future burning plasma devices. Comparisons with experimental data will be made at multiple levels including characterization of turbulence dynamics, flux-gradient responses in the energy, particle and momentum channels, and observations of global phenomena such as L-H transitions, momentum generation, ELM dynamics, ELM-free regimes and the tokamak density limit. The proposal team has established strong connections, through joint membership, with other organizations that will be critical for success. These include FASTMath, SUPER and QUEST Institutes respectively for meshing tools and solvers; code performance tuning and uncertainty quantification as well as a proposed Data Management and Visualization Institute.. We will also be teaming with ongoing fusion SciDAC’s: CSWPI for RF-edge plasmas interactions; CEMM for the M3D MHD simulations and with PEPSC for incorporation of energetic particle physics. Finally, the team includes representatives from each of the major U.S. experimental facilities who will coordinate the validation program and the development of synthetic diagnostics.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗
DOE OSTI2017

Mechanisms of interfacial reactivity in near surface and extreme environments

The local water structure surrounding ions in aqueous solutions greatly affects their chemical properties such as reaction rates, ion association, and proton and electron transport. These properties result in the behavior of ions in natural aqueous environments. For example ore transport is facilitated by chloride ion pair formation and the reaction of ions in an interface is strongly dependent on the dehydration of the ion hydration shell. We are developing the use of high-­resolution XAFS observations and 1st principles based MD-­XAFS analysis (spectra simulated using 1st principle methods with no adjustable parameters, AIMD) to interpret the solution properties of strongly interacting aqueous solutes under arbitrary pressure and temperature conditions. In the 1st principle MD-­XAFS method density functional theory (DFT) based MD simulations(Car and Parrinello, 1985) are used to generate a large ensemble of structural snap shots of the hydration region. These are then used to generate scattering intensities. I emphasize three points about this novel approach to analyzing XAFS data. 1st: As illustrated in Figure 1, the level of agreement between the calculated and observed intensities is considerably higher than has been obtained by any XAFS analysis to date (note 2nd shell region, R> 2 Å). 2nd: This result was obtained from a parameter free simulation with no fitting of the interaction potentials to any data. This supports the use of these methods for more difficult environments and more complex solutes (polyions). 3rd: New information about the shell structure (Figure 1) is now available because of this more detailed agreement. We note also that both multiple scattering and second shell features are well represented in the analysis. As far as we know this is the 1st analysis of second shell structure and multiple scattering. Excellent agreement has been obtained for most of the third row metal ions: Ca 2+ , Zn 2+ , Cu 2+ , Ni 2+ , Co 2+ , Mn 2+ , Fe 3+ , Cr 3+ . Calculations on these systems are demanding because of their open electronic shells, and high ionic charge. Principal Investigator: Professor John Weare (University of California, San Diego) The prediction of the interactions of geochemical fluids with minerals, nanoparticles, and colloids under extreme near surface conditions of temperature (T) and pressure (P) is a grand challenge research need in geosciences (U.S. DOE 2007, Basic Research Needs for Geosciences: Facilitating the 21st Energy Systems.). To evaluate the impact of these processes on energy production and management strategies it is necessary to have a high level of understanding of the interaction between complex natural fluids and mineral formations. This program emphasizes 1st principle parameter free simulations of complex chemical processes in solutions, in the mineral phase, and in the interfaces between these phases The development of new computational tools (with emphasis on oxide materials and reaction dynamics) tailored to treat wide range of conditions and time scales experienced in such geochemical applications is has been developed. Because of the sensitivity of the interaction in these systems to electronic structure and local bonding environments, and of the need to describe bond breaking/formation, our simulations are based on interactions calculated at the electronic structure level (ab-initio molecular dynamics, AIMD). The progress in the computational aspects of program may be summarized in terms of the following themes (objectives); Development of efficient parameter free dynamical simulation technology based on 1st principles force and energy calculations especially adapted for geochemical applications (e.g., mineral, interfaces and aqueous solutions) (continuing program); Calculation of the dynamics of water structure of in the surface-water interface of transition metal oxides and oxihydroxides; and Development of improved (beyond DFT+GGA) electronic structure calculations for minerals and the interface region that more accurately calculate electron correlation, spin density, and localization. The focus of the program is also on the iron oxide and oxihydroxide minerals and Fe 2+ (aq)/Fe 3+ (cr) oxidation in the mineral solution interface region. These methods included the development of model Hamilitonian methods that can be solved to near convergence for single site models (DMFT) and many-body perturbation methods (MP2, GW); Development of time decomposition methods to extend time scales of molecular dynamics (MD) simulations and support the use of high complexity electronic structure calculations (MP2, CCSD(T)) of forces for use in dynamical simulations where very high chemical accuracy is required (microsolvated reactions in absorbed surface layers); and The development of a new linear scaling finite element solver for eigenvalue problem that supports solution of quantum problems with unusual potential and boundary values. Application progress of the above new simulation technology to problems of geochemical interests includes; The prediction of metal oxide surface structure and the reduction/oxidation of Fe 3+ (cr)/Fe 2+ (aq) in metal oxide (hematite, goethite)/solution interfaces. Result: water interacts strongly with the 001 Hematite surface; interaction of water with the 100 goethite is weak; The study of ion solvation and the composition of ion hydration shells under extreme conditions (focus on Fe 3+/2+ , Al 3+ and Mg 2+ and their hydroxide speciation). Result: Ion association in water solutions can be calculated from 1st principle methods. Efficient sampling of the free energy requires more development; The continued development of new high resolution analysis of XAFS scattering of disordered systems (particularly Al, Mg) and of XANES calculations for aqueous ions. Result: EXAFS spectra can be calculated to high accuracy with DFT level dynamic simulations; The exploration of electron localization and electron transport in metal oxides (highly correlated materials). Result: Proper description of electron localization requires levels of calculation beyond DFT; and Localization of electrons in DFT type Hamiltonians was studied. Result: For very Dirac high exchange new solutions (New unphysical bifrucations) to the eigenvalue problem are found. The program was highly collaborative involving faculty and students in mathematics, physics and computer science departments as well as coworkers at the Pacific Northwest National Laboratories (PNNL). The students in this program had the opportunity to develop skills in the development of methods, the implementation of method on high performance parallel computers and the application of these methods to problem in geochemical science. Much of the software that was developed was incorporated in the NWchem software package maintained by PNNL.

58 GEOSCIENCES↗
DOE OSTI2024

Developing And Scaling an OpenFOAM Model to Study Turbulent Flow in a HFIR Coolant Channel

Improving the understanding of how computational fluid dynamics (CFD) direct numerical simulations (DNS) of flows in the High Flux Isotope Reactor (HFIR) perform when run in parallel using the high performance computing (HPC) platform Summit at the Oak Ridge Leadership Computing Facility (OLCF) is of particular importance to boost the computational tools used to support HFIR conversion to low enriched fuel (LEU). Evaluation of scaling performance was driven by the increasing importance of graphics processing unit (GPU) usage in HPC, which is becoming the standard for modern supercomputers such as Summit. The desired results are to obtain a strong positive correlation between the computational resources dedicated to a problem and the relative speed-up of the simulation in comparison to a benchmark. This capability will allow substantially improvement in HFIR flow analytical capabilities, specifically when predicting turbulence properties at high Reynolds numbers. The study leverages previous simulation results performed with code PHASTA (finite element) on HPC platforms Cori (NERSC) and Theta (ALCF) [1] with computing options provided in the computing platform OpenFOAM (finite volume) at OLCF. Transitioning from PHASTA to OpenFOAM will (1) eliminate dependence on third-party software for mesh generation and manipulation, (2) reduce resource needs by employing modern architectures, and (3) build expertise for future modeling of HFIR-specific problems like heat transfer in involute geometry, entrance effects, flow structure in channel corners, and so on—all important issues when defining the available thermal margins in the transition to LEU. CPUs and GPUs differ significantly in their architecture and utilization, as discussed in the literature [2]. The most important differences are in the approach to computations and their memory. A single GPU contains a large quantity of cores, enabling it to perform with a much higher throughput than a CPU, but execution requires a different approach. GPU codes execute instructions using the Single-Instruction Multiple-Thread (SIMT) approach in which a single instruction is used for groups of threads called warps. A warp typically consists of 32 threads which must execute the same set of instructions, although on separate threads. Alternately, a CPU has far fewer cores that are much more flexible in their operation, excelling at quickly performing more complex serial computations. This is why GPUs have greater throughput when properly utilized. The second important difference is seen when comparing their memory spaces. Limited memory allocations and CPU–GPU communications cause a significant bottleneck in GPU-accelerated programs. Further study was required to properly take advantage of GPU resources. A comprehensive analysis of code performance and the model-specific features of turbulence constitutes the core of this work. In this study, a DNS simulation of HFIR channel turbulence was performed with the finite volume CFD code OpenFOAM v2112 and CUDA v11.0 on Red Hat Enterprise Linux v8.2. The OpenFOAM installation had AMGx integrated to enable GPU acceleration and utilizes the PETSc4FOAM library. The computational resources and the problem size were scaled on CPU and CPU + GPU architectures to gain a better understanding of the performance of a DNS problem on modern computing hardware. The study aimed to analyze the scaling of the code exclusively on CPUs and then to examine the scaling of the codes with GPU acceleration enabled. Scaling studies included CPU and GPU acceleration on a mesh of varying resolution to analyze the impact of problem size relative to computational resources. In the course of preparing the GPU configuration on Summit, mainly using the AMGX solvers, difficulties were encountered stemming from constant changes resulting from extensive ongoing development activities and the changing environment. This resulted in the inability to complete the GPU portion of the work. The code was compiled and tested, but production runs to assess acceleration were not performed because the used discretional compute time allocation expired as year-end approached. The Summit HPC platform is scheduled for decommissioning in 2024, making it unattractive for future use with Nvidia-based GPUs. Therefore, the work will be moved onto NERSC machines in FY24. An application was prepared and submitted, and sufficient node-hours were awarded to continue the research in the next calendar year. This report summarizes work performed thus far, which mostly focused on CPU OpenFOAM computing.

21 SPECIFIC NUCLEAR REACTORS AND ASSOCIATED PLANTS↗