A time-relaxation reduced order model for the turbulent channel flow
Not Available
Tsai, Ping-Hsuan (ORCID:0009000323887580)↗
Engineering topics
Fischer, Paul
Publications and source records attributed to Fischer, Paul.
A Study of the Transition to Turbulence in a Bed of 67 Spherical Pebbles
Packed beds are commonly found in many engineering systems and have been widely studied for decades. A relatively new packed bed system is the Pebble Bed Reactor, a type of generation-IV nuclear reactor. Unlike many of the packed beds encountered in chemical and process engineering applications, Pebble Bed Reactors are larger and operate at significantly higher Reynolds numbers. As a result of these differences, there is a very limited amount of information on the detailed flow physics that exist in these complex geometries. This work seeks to contribute to a growing database of flow data for Pebble Bed Reactor systems by performing Direct Numerical Simulations of the flow in an experimental bed of 67 pebbles for a range of conditions. Simulations are performed at a Prandtl number of 0.66 and Reynolds numbers from 300–600. These Reynolds numbers are chosen to gain additional knowledge on the spatial development of turbulence in these systems. Analysis of the Turbulent Kinetic Energy, turbulence anisotropy, and Turbulent Heat Flux is performed. Results demonstrate significant development of the TKE across the tested range of Reynolds numbers. Examination of both the TKE and THF reveal that development first occurs near the center of the bed and propagates radially as the flow moves further into the bed. Notable regions of negative production of turbulent kinetic energy are observed in regions where flow accelerates around pebble contact points. Furthermore, these regions are found to coincide with regions of 1-component turbulence.Kindly check and confirm, all authors email id is correctly identified.These are correct
Direct numberical simulation↗
Accelerating transients with NekRS: GPU overlapping domain implementation and multi-rate timestepping
The simulation of nuclear transients using Computational Fluid Dynamics (CFD) presents significant computational challenges due to the inherent complexity and the wide separation in temporal scales between various flow physical phenomena. These disparities lead to high computational costs, often making the simulation of transients impractical without advanced techniques. Consequently, multiple research initiatives are being pursued by the NEAMS thermal-hydraulic area, some driven by academic institutions and some by national laboratories. Overall, they are exploring novel methods to make transient simulations more feasible and efficient. This report delves into recent advancements within the CFD code NekRS, specifically those achieved in Fiscal Year 2024 under the CONNECT effort, aimed at improving the performance and feasibility of transient simulations. The first major advancement involves the porting of NekRS to Aurora, one of the Department of Energy’s (DOE) most powerful supercomputers. Additionally, the report discusses the implementation of an overlapping domain capability within NekRS. This novel GPU-accelerated capability allows different spatial regions of the domain to be solved independently, enhancing the code’s efficiency, particularly when running large-scale simulations in complex domains. The scalability of this approach is demonstrated, highlighting its potential to transform how transients are approached in CFD simulations. Lastly, the report focuses on how this overlapping domain capability specifically accelerates transient simulations through multi-rate timestepping. By decoupling different regions and facilitating faster computations, this method offers a promising pathway to making nuclear transient simulations more computationally feasible, addressing one of the critical bottlenecks in the field. Together, these advancements represent a significant leap forward in transient simulation technology, bringing closer the possibility of handling highly complex nuclear scenarios with greater efficiency and accuracy.
73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗
Towards exascale for wind energy simulations
We examine large-eddy-simulation modeling approaches and computational performance of two open-source computational fluid dynamics codes for the simulation of atmospheric boundary layer flows that are of direct relevance to wind energy production. The first code, NekRS, is a high-order, unstructured-grid, spectral element code. The second code, AMR-Wind, is a second-order, block-structured, finite-volume code with adaptive mesh refinement capabilities. The objective of this study is to co-develop these codes in order to improve model fidelity and performance for each. These features will be critical for running ABL-based applications such as wind farm analysis on advanced computing architectures. To this end, we investigate the performance of NekRS and AMR-Wind on the Oak Ridge Leadership Facility supercomputers Summit, using 4 to 800 nodes (24 to 4,800 NVIDIA V100 GPUs), and Crusher, the testbed for the Frontier exascale system, using 18 to 384 Graphics Compute Dies on AMD MI250X GPUs. We compare strong- and weak-scaling capabilities, linear solver performance, and time to solution. We also identify leading inhibitors to parallel scaling.
17 WIND ENERGY↗
Energy Exascale Computational Fluid Dynamics Simulations With the Spectral Element Method
Development and application of the open-source GPU-based fluid-thermal simulation code, NekRS, are described. Time advancement is based on an efficient kth-order accurate timesplit formulation coupled with scalable iterative solvers. Spatial discretization is based on the high-order spectral element method (SEM), which affords the use of fast, low-memory, matrix-free operator evaluation. Further, recent developments include support for nonconforming meshes using overset grids and for GPU-based Lagrangian particle tracking. Results of large-eddy simulations of atmospheric boundary layers for wind-energy applications as well as extensive nuclear energy applications are presented.
42 ENGINEERING↗
Evaluation of IDDES in Nek5000 for predicting the flow over periodic hills
We demonstrate the adoption possibilities of the IDDES model in Nek5000 based on the spectral element method. The defined filter width for the LES mode, which considers the length of the element and the polynomial order, is implemented in Nek5000.
97 MATHEMATICS AND COMPUTING↗
Exascale Multiphysics Nuclear Reactor Simulations for Advanced Designs
ENRICO is a coupled application developed under the U.S. Department of Energy's Exascale Computing Project (ECP) targeting the modeling of advanced nuclear reactors. It couples radiation transport with heat and fluid simulation, including the high-fidelity, highresolution Monte-Carlo code Shift and the Computational fluid dynamics code NekRS. NekRS is a highly-performant open-source code for simulation of incompressible and low-Mach fluid flow, heat transfer, and combustion with a particular focus on turbulent flows in complex domains. It is based on rapidly convergent high-order spectral element discretizations that feature minimal numerical dissipation and dispersion. State-of-the-art multilevel preconditioners, efficient high-order time-splitting methods, and runtime-adaptive communication strategies are built on a fast OCCA-based kernel library, libParanumal, to provide scalability and portability across the spectrum of current and future high-performance computing platforms. On Frontier, Nek5000/RS has recently achieved an unprecedented milestone in breaching over 1 billion spectral elements and 350 billion degrees of freedom. Shift has demonstrated the capability to transport upwards of 1 billion particles per second in full core nuclear reactor simulations featuring complete temperature-dependent, continuous-energy physics on Frontier. Shift achieved a weak-scaling efficiency of 97.8% on 8192 nodes of Frontier and calculated 6 reactions in 214,896 fuel pin regions below 1% statistical error yielding first-of-a-kind resolution for a Monte Carlo transport application.
Hamilton, Steven P.↗
Development of MHD for NekRS
A spectral-element-based formulation of incompressible MHD is presented in the context of the open-source fluid-thermal code, Nek5000/RS. The formulation supports magnetic fields in a solid domain that surrounds the fluid domain. Several steady-state and time-transient model problems are presented as part of the code verification process. Nek5000/RS is designed for large-scale turbulence simulations, which will be the next step with this new MHD capability.
97 MATHEMATICS AND COMPUTING↗
Advanced Turbulence Models for Large-Scale Atmospheric Boundary Layer Flows
We present high-fidelity large-eddy-simulation (LES) modeling approaches for the turbulent atmospheric boundary layer (ABL) flows. Wind energy is a prime example of an application driven by ABL. Generation of electrical energy from farms of wind turbines at night in the stable ABL is a particularly interesting situation. In this report, we consider the well-known GEWEX (Global Energy and Water Cycle Experiment) Atmospheric Boundary Layer Study (GABLS) stably stratified benchmark LES case. We use a high-order spectral element code Nek5000/RS, which is supported under the DOE's Exascale Computing Project (ECP) Center for Efficient Exascale Discretizations (CEED) project, targeting application simulations on various acceleration-device based exascale computing platforms. In our earlier ANL report, we demonstrated our newly developed subgrid-scale (SGS) models based on high-pass filter (HPF), mean-field eddy viscosity (MFEV), and Smagorinsky (SMG) with no-slip and traction boundary conditions, provided with low-order statistics, convergence and turbulent structure analysis. In this report, we extend the range of our SGS modeling approaches in the context of the mean-field eddy viscosity (MFEV), to include the solution of an SGS turbulent kinetic energy equation (TKE). We demonstrate the model fidelity of Nek5000/RS in comparison to that of AMR-Wind, a block-structured second-order finite-volume code with adaptive-mesh-refinement capabilities, with which we studied scaling performance for both codes in comparison on DOE's leadership computing platforms.
17 WIND ENERGY↗
Turbulence Modeling with Nek5000/RS, SOD2D and Alya
We present Validation and Verification (V & V) study of two high-order spectral element method (SEM) based Computational Fluid Dyanimcs (CFD) codes that we will utilize for work on turbulence modeling: Nek5000 and SOD2D. While the former solves the incompressible form of Navier-Stokes equation, the latter works with compressible set of equations and uses an entropy-viscosity formulation to account for the discontinuities for high Mach number flows. We demonstrate the accuracy of these codes for two benchmark sub-sonic turbulent flows: periodic channel and pipe flow, by carrying out first and second order statistical analysis, grid convergence and turbulent structure analysis using wall-resolved large eddy simulations (WRLES). Later, we report on the implementation and testing of various wall-modeling strategies for large eddy simulation of turbulent flows in Nek5000. These include both classical log-law based and decision-tree based machine-learning models. Accuracy of these closure strategies are analyzed and necessary future work is outlined.
97 MATHEMATICS AND COMPUTING↗
Accurate and Efficient Parametric Model-Order Reduction for Turbulent Thermal Transport
This project produced new algorithms and software for low-cost exploration of turbulent thermal-fluids behavior under parametric variation by using reduced-order models (ROMs). The ROMs numerically solve the governing equations for fluid motion by using a small set of basis functions (typically, N=20-200 modes) to represent the solution. The base modes are computed as optimal combinations of solutions from expensive high-fidelity "anchor-point" solutions involving millions of unknowns, which are typically generated by solving the full Navier-Stokes equations on a supercomputer. The ROM solution is itself a combination of the base modes, where the basis coefficients are determined by evolving an NxN system of nonlinear equations. The overarching idea is to use the inexpensive ROM to predict solutions under conditions where the parameters differ from the anchor-point conditions. Several ingredients are required to make ROMs useful for thermal hydraulics analysis. These include: a stable and accurate ROM that is capable of reproducing the large-scale dynamics of turbulent flow, error indicators than can guide the choice of anchor points, and low-cost mechanisms for evaluating nonlinear terms in the reduced equations.
22 GENERAL STUDIES OF NUCLEAR REACTORS↗
NekRS, a GPU-accelerated spectral element Navier–Stokes solver
The development of NekRS, a GPU-oriented thermal-fluids simulation code based on the spectral element method (SEM) is described. For performance portability, the code is based on the open concurrent compute abstraction and leverages scalable developments in the SEM code Nek5000 and in libParanumal, which is a library of high-performance kernels for high-order discretizations and PDE-based miniapps. Critical performance sections of the Navier–Stokes time advancement are addressed. Performance results on several platforms are presented here, including scaling to 27,648 V100s on OLCF Summit, for calculations of up to 60B gridpoints.
97 MATHEMATICS AND COMPUTING↗
Nek5000/RS Performance on Advanced GPU Architectures
We demonstrate NekRS performance results on various advanced GPU architectures. NekRS is a GPU-accelerated version of Nek5000 that is targeting high performance on forthcoming exascale platforms. It is being developed in DOE’s Center of Efficient Exascale Discretizations (CEED), which is one of the co-design centers under the Exascale Computing Project (ECP). In this report, we consider Frontier, Crusher, Spock, Polaris, Perlmutter, ThetaGPU and Summit. Simulations are performed using ExaSMR’s 17x17 rod-bundle geometries with different problem sizes. The report focuses on strong scaling performance and analysis. Many of the results shown in this report are the outcome from participation in the ALCF GPU Hackathon on Polaris, which was held on 7/19/22, and 7/26–7/28/22. Members of the Nek5000/RS team included Misun Min, Yu-Hsiang Lan, Paul Fischer and Thilina Rathnayake. Mentors were Kris Rowe (ALCF) and Peng Wang (NVIDIA). Also presented in this report are results on Frontier, which were obtained in collaboration with John Holmen at OLCF.
97 MATHEMATICS AND COMPUTING↗
Augmented reduced order models for turbulence
The authors introduce an augmented-basis method (ABM) to stabilize reduced-order models (ROMs) of turbulent incompressible flows. The method begins with standard basis functions derived from proper orthogonal decomposition (POD) of snapshot sets taken from a full-order model. These are then augmented with divergence-free projections of a subset of the nonlinear interaction terms that constitute a significant fraction of the time-derivative of the solution. The augmenting bases, which are rich in localized high wavenumber content, are better able to dissipate turbulent kinetic energy than the standard POD bases. Several examples illustrate that the ABM significantly out-performs L 2 -, H 1 - and Leray-stabilized POD ROM approaches. The ABM yields accuracy that is comparable to constraint-based stabilization approaches yet is suitable for parametric model-order reduction in which one uses the ROM to evaluate quantities of interests at parameter values that differ from those used to generate the full-order model snapshots. Several numerical experiments point to the importance of localized high wavenumber content in the generation of stable, accurate, and efficient ROMs for turbulent flows.
Kaneko, Kento↗
Pressure Drop Correlation Improvement for the Near-Wall Region of Pebble-Bed Reactors
Packed beds play an important role in several engineering fields, with their applications in nuclear energy being driven by the development of next-generation reactors utilizing pebble fuel. The random nature of a packed pebble bed creates a flow field that is complex and difficult to predict. Porous media models are an attractive option for modeling pebble-bed reactors (PBRs), as they provide intermediate fidelity results and are computationally efficient. Porous media models, however, rely on the use of correlations to estimate the effect of complicated flow features on the pressure drop and heat transfer in the system. Existing correlations were developed to predict the average behavior of the bed, but they are inaccurate in the near-wall region where the presence of the wall affects the pebble packing. This work aims to investigate the accuracy of a porous media model using the Kerntechnischer Ausschuss (KTA) correlation, the most common pressure drop correlation for PBRs compared to the high-fidelity large eddy simulation (LES). A bed of 1568 pebbles is investigated at Reynolds numbers from 625 to 10 000. The bed is divided into five concentric subdomains to compare the average velocity, friction losses, and form losses between the porous media and LES codes. The comparison between the LES simulation and the KTA correlation revealed that the KTA correlation largely underpredicts the form losses in the near-wall region, leading to an overprediction of the velocity near the wall by nearly 30%. An investigation of the form losses across the range of Reynolds numbers in the LES results provided additional insight into how the KTA correlation may be improved to better predict these spatial effects in a pebble bed. These data suggest that the form coefficient near the wall must be increased by 48% while decreasing the form coefficient of the inner bulk region of the bed by 15%. The implementation of these improvements to the KTA correlation in a porous media model produced a radial velocity profile that saw significantly improved agreement with the LES results.
22 GENERAL STUDIES OF NUCLEAR REACTORS↗
Parametric model-order-reduction development for unsteady convection
A time-averaged error indicator with POD- h Greedy is developed to drive parametric model order reduction (pMOR) for 2D unsteady natural convection in a high-aspect ratio slot parameterized with the Prandtl number, Rayleigh number, and slot angle with respect to the gravity. The error indicator is extended to accommodate the energy equation and Leray regularization. Despite being two-dimensional and laminar, the target flow regime presents several challenges: 1) there is a bifurcation in the angle parameter space; 2) the solution can be multivalued, even at steady state; and 3) the solution exhibits spatio-temporal chaos at several points in the parameter space. The authors explore several reduced-order models (ROMs) and demonstrate that Leray-regularized Galerkin ROMs provide a robust solution approach for this class of flows. They further demonstrate that error-indicated pMOR can efficiently predict several QOIs, such as mean flow, mean Nusselt number and mean turbulent kinetic energy, even in the presence of a bifurcation. Finally, they show that spatio-temporal chaos can lead to lack of reproducibility in both the full-order model and the reduced-order model and that the variance in the full-order model provides a lower bound on the pMOR error in these cases.
leray regularization↗
Scalable Interpolation on GPUs for Thermal Fluids Applications
The need for efficient interpolation arises in many use cases for spectral element simulations. Towards this end, we present a fast and robust GPU-enable interpolation utility for NekRS, a spectral-element, Navier-Stokes solver. This utility is based on the findpts and findpts eval routines from the gslib library. To demonstrate the functionality, we added support for particle tracking. The GPU implementation of the interpolation allows for tracking 100 times more particles for the same computational cost.
71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
Port and optimize the CEED software stack to Aurora/Frontier EA (ECP Milestone Report)
The goal of this milestone was to port the CEED software stack, including Nek, MFEM and libCEED to the Frontier and Aurora early access hardware, work on optimizing the performance on AMD and Intel GPUs, and demonstrate impact in CEED-enabled ECP applications. As part of this milestone, we also performed a number of other activities, including continued optimizations for CEED applications at scale on Summit, performance evaluation on non-ECP hardware, including the Fugaku’s A64FX chip and the NVIDIA A100 GPU architecture, exploring the use of just-in-time compilations in applications at scale and potential of mixed precision optimizations in the CEED discretization and solver algorithms, and more. During the milestone period, the CEED team released new versions of 6 of its packages, including major releases of MFEM, NekRS and OCCA. We also organized the fifth CEED Annual meeting (CEED5AM) which included nearly 100 researchers from national labs, universities and industry.
97 MATHEMATICS AND COMPUTING↗