SEARCH · Engineering Papers
Results for “spectral elements”
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
Inviscid simulations of expansion waves propagating into structured particle beds at low volume fractions
Expansion waves propagating into particle beds are found in many manmade and natural applications. Rapid decompression of a highly pressurized powder bed or a volcanic eruption are two such examples. In this investigation, we perform fully resolved simulations of expansion waves propagating into particle beds of three different volume fractions using the discontinuous Galerkin spectral element method code, cmt-nek. We validate state-of-the-art drag models for a particle in an unsteady compressible flow and show good agreement, particularly at lower volume fractions. We model the particle bed as a nozzle or area reduction and, using isentropic flow relations, predict the final pressure, temperature, and Mach number extremely well in each case.
Large Eddy Simulation of the Flow Behavior in a Simplified Helical Coil Steam Generator
Large eddy simulation (LES) is conducted for the flow over the shell side of a helical coil steam generator (HCSG) heat exchanger. Simulations are conducted on a simplified experimental test section that represents a one-column region of the helical coils using half-rods. Although the rods are wall-bounded, the flow still exhibits the turbulent characteristics and fluctuations from vortex shedding that one would expect from crossflow around a cylinder. The spectral element, computational fluid dynamics (CFD) code Nek5000, is used to capture the physics, and the results are compared with particle image velocimetry (PIV) measurements. In order to ensure that the turbulence is resolved, analysis is conducted by using the Taylor length scales and normalized wall distance. Sensitivity to the inlet boundary conditions (BCs) and the spatial discretization for different polynomial order solutions are also studied, finding only minor differences between each case. Pressure drop and velocity statistics show reasonable agreement with PIV. Proper orthogonal decomposition (POD) analysis reveals that the primary modes are similar between experiment and simulation, although the LES predicts higher turbulent kinetic energy than does PIV. Overall, the study establishes the resolution and resources required in order to conduct a high-fidelity simulation over 12 helical rods.
Calculation of Friction Factors and Nusselt Numbers for Twisted Elliptical Tube Heat Exchangers Using Nek5000
Twisted elliptical tube geometries for heat exchanger design are expected to offer significant enhancement in heat transfer with a marginal increase in frictional losses. The presented work focuses on computational fluid dynamics (CFD) simulations of heated molten salt flows through twisted elliptical tube geometries at low modified Froude numbers. In this work, the objectives are to evaluate the available correlations at low Froude numbers and to determine the impact of using a small gap to resolve contact points. The spectral element CFD code Nek5000 was used for all simulations, which were performed in periodic domains of triangular and square unit cells surrounding a single tube through a complete twist using an explicit filtering large eddy simulation (LES) method. The simulation methodology was validated by comparing to the experimental data of Tan et al. Excellent agreement was observed for this comparison. The Dzyubenko correlation for transitional flow was evaluated for a range of Froude numbers. This correlation demonstrated good agreement with LES results at high modified Froude number. As Froude number was decreased toward the bounds of the correlation, the agreement worsened. Cases were then simulated at low Froude number, testing the effects of tube spacing. It was determined that the laminar case for the square unit cell is the most affected by increasing gap size, which should be minimized to mitigate the effect. In the triangular unit cell, the laminar flow regime is also more significantly impacted by increasing gap size compared to the turbulent flow regime which was only marginally impacted.
Assessment of Machine Learning Wall Modeling Approaches for Large Eddy Simulation of Gas Turbine Film Cooling Flows: An a Priori Study
Here, in this work, a priori analysis of machine learning (ML) strategies is carried out with the goal of data-driven wall modeling for large eddy simulation (LES) of gas turbine film cooling flows. High-fidelity flow datasets are extracted from wall-resolved LES (WRLES) of flow over a flat plate interacting with the coolant flow supplied by a single row of 7-7-7 shaped cooling holes inclined at 30 degrees with the flat plate at different blowing ratios (BR). The WRLES are performed using the high-order Nek5000 spectral element computational fluid dynamics (CFD) solver. Light gradient boosting machine (LightGBM) is employed as the ML algorithm for the data-driven wall model. Parametric tests are conducted to systematically assess the influence of a wide range of input flow features (velocity components, velocity gradients, pressure gradients, and fluid properties) on the accuracy of ML wall model with respect to prediction of wall shear stress. In addition, the use of spatial stencil and time delay is also explored within the ML wall modeling framework. It is shown that features associated with gradients of the streamwise and spanwise velocity components have a major impact on the prediction fidelity of wall model, while the effect of gradients of wall-normal velocity component is found to be negligible. Moreover, adding flow feature information from an x-y-z spatial stencil significantly improves the ML model accuracy and generalizability compared to just using local flow features from the matching location. Overall, highest prediction accuracy is achieved when both spatial stencil and time delay features are incorporated within the data-driven wall modeling paradigm.
Scalable direct Vlasov solver with discontinuous Galerkin method on unstructured mesh.
This paper presents the development of parallel direct Vlasov solvers with discontinuous Galerkin (DG) method for beam and plasma simulations in four dimensions. Both physical and velocity spaces are in two dimesions (2P2V) with unstructured mesh. Contrary to the standard particle-in-cell (PIC) approach for kinetic space plasma simulations, i.e., solving Vlasov-Maxwell equations, direct method has been used in this paper. There are several benefits to solving a Vlasov equation directly, such as avoiding noise associated with a finite number of particles and the capability to capture fine structure in the plasma. The most challanging part of a direct Vlasov solver comes from higher dimensions, as the computational cost increases as N{sup 2d}, where d is the dimension of the physical space. Recently, due to the fast development of supercomputers, the possibility has become more realistic. Many efforts have been made to solve Vlasov equations in low dimensions before; now more interest has focused on higher dimensions. Different numerical methods have been tried so far, such as the finite difference method, Fourier Spectral method, finite volume method, and spectral element method. This paper is based on our previous efforts to use the DG method. The DG method has been proven to be very successful in solving Maxwell equations, and this paper is our first effort in applying the DG method to Vlasov equations. DG has shown several advantages, such as local mass matrix, strong stability, and easy parallelization. These are particularly suitable for Vlasov equations. Domain decomposition in high dimensions has been used for parallelization; these include a highly scalable parallel two-dimensional Poisson solver. Benchmark results have been shown and simulation results will be reported.
Conservative Multimoment Transport along Characteristics for Discontinuous Galerkin Methods
A set of algorithms based on characteristic discontinuous Galerkin methods is presented for tracer transport on the sphere. The algorithms are designed to reduce message passing interface communication volume per unit of simulated time relative to current methods generally, and to the spectral element scheme employed by the U.S. Department of Energy's Exascale Earth System Model (E3SM) specifically. Two methods are developed to enforce discrete mass conservation when the transport schemes are coupled to a separate dynamics solver; constrained transport and Jacobian-combined transport. A communication-efficient method is introduced to enforce tracer consistency between the transport scheme and dynamics solver; this method also provides the transport scheme's shape preservation capability. A subset of the algorithms derived here is implemented in E3SM and shown to improve transport performance by a factor of 2.2 for the model's standard configuration with 40 tracers at the strong scaling limit of one element per core. Read More: https://epubs.siam.org/doi/10.1137/18M1165943
Pillars of the Mantle: Imaging the Interior of the Earth with Adjoint Tomography. In: PEARC17 Proceedings of the Practice and Experience in Advanced Research Computing 2017 on Sustainability, Success and Impact, Article No. 75
In this work, we investigate global seismic tomographic models obtained by spectral-element simulations of seismic wave propagation and adjoint methods. Global crustal and mantle models are obtained based on an iterative conjugate-gradient type of optimization scheme. Forward and adjoint seismic wave propagation simulations, which result in synthetic seismic data to make measurements and data sensitivity kernels to compute gradient for model updates, respectively, are performed by the SPECFEM3D_GLOBE package [1] [2] at the Oak Ridge Leadership Computing Facility (OLCF) to study the structure of the Earth at unprecedented levels. Using advances in solver techniques that run on the GPUs on Titan at the OLCF, scientists are able to perform large-scale seismic inverse modeling and imaging. Using seismic data from global and regional networks from global CMT earthquakes, scientists are using SPECFEM3D_GLOBE to understand the structure of the mantle layer of the Earth. Visualization of the generated data sets provide an effective way to understand the computed wave perturbations which define the structure of mantle in the Earth.
mgsflib
Code for my PhD thesis that creates meshes using gmsh and code I wrote to create high-order hybrid meshes to use for testing a hybrid method between spectral element method and generalized finite differences. It also uses a tetrahedral mesh improvement method called "stellar" to improve mesh quality near a curved boundary.
An MPI/OpenACC implementation of a high-order electromagnetics solver with GPUDirect communication
We present performance results and an analysis of a message passing interface (MPI)/OpenACC implementation of an electromagnetic solver based on a spectral-element discontinuous Galerkin discretization of the time-dependent Maxwell equations. The OpenACC implementation covers all solution routines, including a highly tuned element-by-element operator evaluation and a GPUDirect gather–scatter kernel to effect nearest neighbor flux exchanges. Modifications are designed to make effective use of vectorization, streaming, and data management. Performance results using up to 16,384 graphics processing units of the Cray XK7 supercomputer Titan show more than 2.5× speedup over central processing unit-only performance on the same number of nodes (262,144 MPI ranks) for problem sizes of up to 6.9 billion grid points. We discuss performance-enhancement strategies and the overall potential of GPU-based computing for this class of problems.
Towards exascale for wind energy simulations
We examine large-eddy-simulation modeling approaches and computational performance of two open-source computational fluid dynamics codes for the simulation of atmospheric boundary layer flows that are of direct relevance to wind energy production. The first code, NekRS, is a high-order, unstructured-grid, spectral element code. The second code, AMR-Wind, is a second-order, block-structured, finite-volume code with adaptive mesh refinement capabilities. The objective of this study is to co-develop these codes in order to improve model fidelity and performance for each. These features will be critical for running ABL-based applications such as wind farm analysis on advanced computing architectures. To this end, we investigate the performance of NekRS and AMR-Wind on the Oak Ridge Leadership Facility supercomputers Summit, using 4 to 800 nodes (24 to 4,800 NVIDIA V100 GPUs), and Crusher, the testbed for the Frontier exascale system, using 18 to 384 Graphics Compute Dies on AMD MI250X GPUs. We compare strong- and weak-scaling capabilities, linear solver performance, and time to solution. We also identify leading inhibitors to parallel scaling.
Memory-Efficient Simulation of Frequency-Dependent Q
Memory-variable methods have been widely applied to approximate frequency-independent quality factor Q in numerical simulation of wave propagation. The frequency-independent model is often appropriate for frequencies up to about 1 Hz but at higher frequencies is inconsistent with some regional studies of seismic attenuation. We apply the memory-variable approach to frequency-dependent Q models that are constant below, and follow a power-law above, a chosen transition frequency. We present numerical results for the corresponding memory-variable relaxation times and weights, obtained by nonnegative least-squares fitting of the Q(f) function, for a range of exponent values; these times and weights can be scaled to an arbitrary transition frequency and a power-law prefactor, respectively. The resulting memory-variable formulation can be used with numerical wave-propagation solvers based on methods such as finite differences (FDs) or spectral elements and may be implemented in either conventional or coarse-grained form. In the coarse-grained approach, we fit effective Q for low-Q values (<200) using a nonlinear inversion technique and use an interpolation formula to find the corresponding weighting coefficients for arbitrary Q. A 3D staggered-grid FD implementation closely approximates the frequency–wavenumber solution to both a half-space and a layered model with a shallow dislocation source for Q as low as 20 over a bandwidth of two decades. We compare the effects of different power-law exponents using a finite-fault source model of the 2008 Mw 5.4 Chino Hills, California, earthquake and find that Q(f) models generally better fit the strong-motion data than the constant Q models for frequencies above 1 Hz.
Scalability under a Power Bound using the GREMLINs Framework
With the move towards exascale, system and software developers will have to deal with issues of extreme parallelism. The system properties affected most by the increase in node and core count are the shared resources on node and across the system. The increase in parallelism leads to reduced memory and bandwidth when regarding individual cores. Since power is a limiting factor for supercomputers, and power is not fully utilized in current systems, overprovisioning compute resources is a viable approach to maximized power utilization. To maximize system performance in regard to these changing conditions, it is necessary to understand how resource restrictions impact performance and system behavior. For the purpose of understanding anticipated system properties the GREMLINs Framework was developed. The framework gives the opportunity to add power restrictions, hinder memory properties and introduce faults to study resilience, among others. These features give the opportunity to use current petascale technology to study problems system designers and software developers will have to face when moving towards exascale and beyond. This work describes the initial release of the GREMLINs Framework, developed for this work, and shows how it can be used to study the scaling behavior of proxy applications. These proxy applications represent a selection of HPC workloads important to the scientific community. The proxy applications studied are AMG2013, an algebraic multi-grid linear system solver, CoMD, a classical molecular dynamics proxy application and NEKBONE, an application that uses a high order spectral element method to solve the Navier-Stokes equations. The main interest of these studies lies in analysis regarding their power behavior at scale under a power bound. These findings show how the GREMLINs Framework can help systems and software designers to attain better application performance and can also be used as basis for CPU power balancing tools to use power more efficiently.
Collaborative project. Ocean-atmosphere interaction from meso-to planetary-scale. Mechanisms, parameterization, and variability
This project aims to improve long term global climate simulations by resolving ocean mesoscale activity and the corresponding response in the atmosphere. The main computational objectives are; i) to perform and assess Community Earth System Model (CESM) simulations with the new Community Atmospheric Model (CAM) spectral element dynamical core; ii) use static mesh refinement to focus on oceanic fronts; iii) develop a new Earth System Modeling tool to investigate the atmospheric response to fronts by selectively filtering surface flux fields in the CESM coupler. The climate research objectives are 1) to improve the coupling of ocean fronts and the atmospheric boundary layer via investigations of dependency on model resolution and stability functions: 2) to understand and simulate the ensuing tropospheric response that has recently been documented in observations: and 3) to investigate the relationship of ocean frontal variability to low frequency climate variability and the accompanying storm tracks and extremes in high resolution simulations. This is a collaborative multi-institution project consisting of computational scientists, climate scientists and climate model developers. It specifically aims at DOE objectives of advancing simulation and predictive capability of climate models through improvements in resolution and physical process representation.
Final Project Report
Most climate models are currently run with grid spacings of around 100km, which, with today’s computing power, allows for long (up to 1000 year) simulations, or ensembles of simulations to explore climate change and variability. However this grid spacing does not resolve important components of the weather/climate system such as atmospheric fronts and mesoscale systems, and ocean boundary currents and eddies. The overall aim of this project has been to look at the effect of these small-scale features on the weather/climate system using a suite of high and low resolution climate models, idealized models and observations. This project was only possible due to the highly scalable aspect of the CAM Spectral Element dynamical core, and the significant resources allocated at Yellowstone and NERSC for which we are grateful.
Chemistry in CESM-SE: Evaluation, Performance and Optimization
The purpose of the proposed work focused on development of chemistry representation within the Spectral Element (SE) dynamical core as implemented in the Community Earth System Model (CESM). More specifically, a main focus was on the ability of SE to accurately represent tracer transport. The proposed approach was to incrementally increase the complexity of the problem, starting from specified two-dimensional flow and tracers to simulations using specified dynamics and full chemistry. As demonstrated below, we have successfully studied all aspects of the proposed work, although only part of the work has been published in the refereed literature so far. Furthermore, because the SE dynamical core has been found to have several deficiencies that are still being investigated for solution, not all proposed tasks were finalized. In addition to the tests for SE performance, in an effort to decrease the computational burden of interactive chemistry, especially in the case of a large number of chemical species and chemical reactions, development on a faster chemical solver and implementation on GPUs has been implemented in CESM under the leadership of John Drake (U. Tennessee).
Toward a Non-Hydrostatic HOMME
CSLAM integration into HOMME: The finite-volume based CSLAM algorithm is designed for a uniform-resolution equiangular central projection on the cubed-sphere for the control volumes (cells). However, the HOMME model uses spectral elements that contain highly irregular cells within. Thus, to implement CSLAM in the HOMME framework presents two major challenges: (1) extend CSLAM to general quadrilateral meshes and (2) widen the halo regions in HOMME to incorporate monotonicity constraints. Since the HOMME communication package gathers only the points on the surface of its nearest-neighbor elements, widening the halo region is a significant data structure change in HOMME that required substantial software engineering support