Engineering Papers⌕ Search

Engineering topics

Sjogreen, B.

Publications and source records attributed to Sjogreen, B..

At least 19 records

Porting a 3D seismic modeling code (SW4) to CORAL machines

Seismic waves fourth order (SW4) solves the seismic wave equations on Cartesian and curvilinear grids using large compute clusters with O (100,000) cores. In this article we discuss the porting of SW4 to run on the CORAL architecture using the RAJA performance portability abstraction layer. The performances of key kernels using RAJA and CUDA are compared to estimate the performance penalty of using the portability abstraction layer. Code changes required for efficiency on GPUs and minimizing time spent in Message Passing Interface (MPI) are discussed. This article describes a path for efficiently porting large code bases to GPU-based machines while avoiding the pitfalls of a new architecture in the early stages of its deployment. Current bottlenecks in the code are discussed along with possible architectural or software mitigations. SW4 runs 28× faster on one 4-GPU CORAL node than on a CTS-1 node (Dual Intel Xeon E5-2695 v4). SW4 is now in routine use on problems of unprecedented resolution (203 billion grid points) and scale on 1,200 nodes of Summit.

97 MATHEMATICS AND COMPUTING↗

High Order Numerical Methods for LES of Turbulent Flows with Shocks

Simulation of turbulent flows with shocks employing explicit subgrid-scale (SGS) filtering may encounter a loss of accuracy in the vicinity of a shock. In this work we perform a comparative study of different approaches to reduce this loss of accuracy within the framework of the dynamic Germano SGS model. One of the possible approaches is to apply Harten’s subcell resolution procedure to locate and sharpen the shock, and to use a one-sided test filter at the grid points adjacent to the exact shock location. The other considered approach is local disabling of the SGS terms in the vicinity of the shock location. In this study we use a canonical shock-turbulence interaction problem for comparison of the considered modifications of the SGS filtering procedure. For the considered test case both approaches show a similar improvement in the accuracy near the shock.

LES↗

Comparitive Study of High-Order Positivity-Preserving WENO Schemes

In gas dynamics and magnetohydrodynamics flows, physically, the density ρ and the pressure p should both be positive. In a standard conservative numerical scheme, however, the computed internal energy is The ideas of Zhang & Shu (2012) and Hu et al. (2012) precisely address the aforementioned issue. Zhang & Shu constructed a new conservative positivity-preserving procedure to preserve positive density and pressure for high-order Weighted Essentially Non-Oscillatory (WENO) schemes by the Lax-Friedrichs flux (WENO/LLF). In general, WENO/LLF is obtained by subtracting the kinetic energy from the total energy, resulting in a computed p that may be negative. Examples are problems in which the dominant energy is kinetic. Negative ρ may often emerge in computing blast waves. In such situations the computed eigenvalues of the Jacobian will become imaginary. Consequently, the initial value problem for the linearized system will be ill posed. This explains why failure of preserving positivity of density or pressure may cause blow-ups of the numerical algorithm. The adhoc methods in numerical strategy which modify the computed negative density and/or the computed negative pressure to be positive are neither a conservative cure nor a stable solution. Conservative positivity-preserving schemes are more appropriate for such flow problems. too dissipative for flows such as turbulence with strong shocks computed in direct numerical simulations (DNS) and large eddy simulations (LES). The new conservative positivity-preserving procedure proposed in Hu et al. (2012) can be used with any high-order shock-capturing scheme, including high-order WENO schemes using the Roe's flux (WENO/Roe). The goal of this study is to compare the results obtained by non-positivity-preserving methods with the recently developed positivity-preserving schemes for representative test cases. In particular the more di cult 3D Noh and Sedov problems are considered. These test cases are chosen because of the negative pressure/density most often exhibited by standard high-order shock-capturing schemes. The simulation of a hypersonic nonequilibrium viscous shock tube that is related to the NASA Electric Arc Shock Tube (EAST) is also included. EAST is a high-temperature and high Mach number viscous nonequilibrium ow consisting of 13 species. In addition, as most common shock-capturing schemes have been developed for problems without source terms, when applied to problems with nonlinear and/or sti source terms these methods can result in spurious solutions, even when solving a conservative system of equations with a conservative scheme. This kind of behavior can be observed even for a scalar case as well as for the case consisting of two species and one reaction.. This EAST example indicated that standard high-order shock-capturing methods exhibit instability of density/pressure in addition to grid-dependent discontinuity locations with insufficient grid points. The evaluation of these test cases is based on the stability of the numerical schemes together with the accuracy of the obtained solutions.

Schemes↗

Comparative Study on High-Order Positivity-preserving WENO Schemes

In gas dynamics and magnetohydrodynamics flows, physically, the density and the pressure p should both be positive. In a standard conservative numerical scheme, however, the computed internal energy is obtained by subtracting the kinetic energy from the total energy, resulting in a computed p that may be negative. Examples are problems in which the dominant energy is kinetic. Negative may often emerge in computing blast waves. In such situations the computed eigenvalues of the Jacobian will become imaginary. Consequently, the initial value problem for the linearized system will be ill posed. This explains why failure of preserving positivity of density or pressure may cause blow-ups of the numerical algorithm. The adhoc methods in numerical strategy which modify the computed negative density and/or the computed negative pressure to be positive are neither a conservative cure nor a stable solution. Conservative positivity-preserving schemes are more appropriate for such flow problems. The ideas of Zhang & Shu (2012) and Hu et al. (2012) precisely address the aforementioned issue. Zhang & Shu constructed a new conservative positivity-preserving procedure to preserve positive density and pressure for high-order WENO schemes by the Lax-Friedrichs flux (WENO/LLF). In general, WENO/LLF is too dissipative for flows such as turbulence with strong shocks computed in direct numerical simulations (DNS) and large eddy simulations (LES). The new conservative positivity-preserving procedure proposed in Hu et al. (2012) can be used with any high-order shock-capturing scheme, including high-order WENO schemes using the Roe's flux (WENO/Roe). The goal of this study is to compare the results obtained by non-positivity-preserving methods with the recently developed positivity-preserving schemes for representative test cases. In particular the more difficult 3D Noh and Sedov problems are considered. These test cases are chosen because of the negative pressure/density most often exhibited by standard high-order shock-capturing schemes. The simulation of a hypersonic nonequilibrium viscous shock tube that is related to the NASA Electric Arc Shock Tube (EAST) is also included. EAST is a high-temperature and high Mach number viscous nonequilibrium flow consisting of 13 species. In addition, as most common shock-capturing schemes have been developed for problems without source terms, when applied to problems with nonlinear and/or sti source terms these methods can result in spurious solutions, even when solving a conservative system of equations with a conservative scheme. This kind of behavior can be observed even for a scalar case (LeVeque & Yee 1990) as well as for the case consisting of two species and one reaction (Wang et al. 2012). For further information concerning this issue see (LeVeque & Yee 1990; Griffiths et al. 1992; Lafon & Yee 1996; Yee et al. 2012). This EAST example indicated that standard high-order shock-capturing methods exhibit instability of density/pressure in addition to grid-dependent discontinuity locations with insufficient grid points. The evaluation of these test cases is based on the stability of the numerical schemes together with the accuracy of the obtained solutions.

WENO↗

High Order Finite Difference Methods with Subcell Resolution for 2D Detonation Waves

In simulating hyperbolic conservation laws in conjunction with an inhomogeneous stiff source term, if the solution is discontinuous, spurious numerical results may be produced due to different time scales of the transport part and the source term. This numerical issue often arises in combustion and high speed chemical reacting flows.

Wang, W.↗

Pre-Shot Simulations of Near-Field and Far-Field Ground Motion for the Source Physics Experiment (SPE) Explosions at the Climax Stock, Nevada National Security Site: SPE3

The Source Physics Experiment (SPE) is designed to improve ground motion prediction capabilities for underground nuclear testing through the development of new physics-based models together with validation via new chemical explosion data. The SPE shots consist of different size and depth chemical explosions in a granite borehole at the Climax Stock on the Nevada National Security Site (NNSS). The third shot in the series (SPE3) will be the same size (1000 kg TNT equivalent) and depth (45 m canister centroid) as the previous shot (SPE2). The main purpose of the SPE3 is to investigate the effects of the damage zone caused by the SPE2 chemical explosion on the SPE3 explosion, keeping all other parameters (e.g. size, depth, explosive type) the same. This SPE3 pre-shot report addresses progress made in simulation of the expected wavefield in the both the near- and the far-field. In this report we consider the far-field to be where seismic wave motion can be characterized by elastic or visco-elastic approximations, in contrast to the near-field where other, often non-linear effects, play a major role. The current simulations build on previous work from the SPE1 and SPE2 shots. The simulations are intended to highlight differences between SPE2 and SPE3 using a variety to approximations intended to mimic the expected effect of the damage zone. The simplest physics of damage zone effects suggest changes in the source region material properties (density, velocity), and in the yield strength (coupling) of the rock, should both impact the far-field signal. We can illustrate this in a simplistic way for P-waves in Figure A using the empirical Mueller-Muphy (1971) far-field explosion model (MM). First if the SPE2 explosion-induced damage reduces velocities and densities without affecting the yield strength or coupling, then by energy conservation the amplitudes for SPE 3 would increase as shown in the top of Figure A. Similarly if the explosion induced damage increased density (say by compacting pore space) the amplitude would decrease. This impedance effect, where the product of the density, wave-velocity, and ground-velocity squared is conserved, is well known. The second and harder effect to quantify, are changes in the energy coupling or yield strength. In the simple MM model this is represented by the coupling factor between cavity radius and yield. If we assume the granite was slightly weakened by SPE2 such that the cavity radius increases slightly then the overall amplitudes of SPE3 decrease relative to SPE2 as shown in the bottom of Figure A. In reality both effects, may exist due to the damage zone, and furthermore we know they are interrelated in complex ways. For S-waves we expect even more complex effects due to motion on joints and faults. For this reason we are developing physics-based near field simulation models coupled to far-field propagation codes to predict the overall effects of changes in the source emplacement region on observed signals. Near-field simulations have been performed using the Eulerian code (GEODYN). An ensemble model, calibrated to match records from past large-scale nuclear events is used. Both 2D axisymmetric (no faults) and fully three-dimensional GEODYN calculations (with /without faults) were conducted in which the damage zones sustained by SPE2 are retained as the initial condition for the SPE3 calculations. The results indicate that the waveform peak amplitudes are slightly decayed since the ensemble model includes no softening mechanism. The pulse widths are reduced, indicating the energy is dissipated more by the damage zone as expected. The dominant motions calculated in all the scenarios are spherically symmetric and non-spherical components are small as seen for the SPE2 calculations. The arrival time of the SPE3 waveform at the boreholes is shorter than SPE2 due to compaction during the previous explosion. As shown in the SPE 1 and 2 observations shear waves are generated, indicating the potential contributions from the induced motion on distributed joints. The effects of joints are evaluated using the Lagrangian hydrocode (GEODYN-L), which was benchmarked against the GEODYN calculations for the ensemble model. A new granite model calibrated for samples from the SPE site is used. The new model includes effects of poroelasticity and water saturation and may be scaled allowing application to both small and large scales. Parametric tests were performed to evaluate the effect of joints within the granite and the results demonstrate that in situ joints introduce anisotropy and significant tangential motions.

58 GEOSCIENCES↗

Numerical Dissipation and Wrong Propagation Speed of Discontinuities for Stiff Source Terms

In compressible turbulent combustion/nonequilibrium flows, the constructions of numerical schemes for (a) stable and accurate simulation of turbulence with strong shocks, and (b) obtaining correct propagation speed of discontinuities for stiff reacting terms on coarse grids share one important ingredient - minimization of numerical dissipation while maintaining numerical stability. Here coarse grids means standard mesh density requirement for accurate simulation of typical non-reacting flows. This dual requirement to achieve both numerical stability and accuracy with zero or minimal use of numerical dissipation is most often conflicting for existing schemes that were designed for non-reacting flows. The goal of this paper is to relate numerical dissipations that are inherited in a selected set of high order shock-capturing schemes with the onset of wrong propagation speed of discontinuities for two representative stiff detonation wave problems.

Yee, H. C.↗

LES of Temporally Evolving Mixing Layers by an Eighth-Order Filter Scheme

An eighth-order filter method for a wide range of compressible flow speeds (H.C. Yee and B. Sjogreen, Proceedings of ICOSAHOM09, June 22-26, 2009, Trondheim, Norway) are employed for large eddy simulations (LES) of temporally evolving mixing layers (TML) for different convective Mach numbers (Mc) and Reynolds numbers. The high order filter method is designed for accurate and efficient simulations of shock-free compressible turbulence, turbulence with shocklets and turbulence with strong shocks with minimum tuning of scheme parameters. The value of Mc considered is for the TML range from the quasi-incompressible regime to the highly compressible supersonic regime. The three main characteristics of compressible TML (the self similarity property, compressibility effects and the presence of large-scale structure with shocklets for high Mc) are considered for the LES study. The LES results using the same scheme parameters for all studied cases agree well with experimental results of Barone et al. (2006), and published direct numerical simulations (DNS) work of Rogers & Moser (1994) and Pantano & Sarkar (2002).

Hadjadj, A↗

Pre-shot simulations of far-field ground motion for the Source Physics Experiment (SPE) Explosions at the Climax Stock, Nevada National Security Site: SPE2

The Source Physics Experiment (SPE) is planning a 1000 kg (TNT equivalent) shot (SPE2) at the Nevada National Security Site (NNSS) in a granite borehole at a depth (canister centroid) of 45 meters. This shot follows an earlier shot of 100 kg in the same borehole at a depth 60 m. Surrounding the shotpoint is an extensive array of seismic sensors arrayed in 5 radial lines extending out 2 km to the north and east and approximately 10-15 to the south and west. Prior to SPE1, simulations using a finite difference code and a 3D numerical model based on the geologic setting were conducted, which predicted higher amplitudes to the south and east in the alluvium of Yucca Flat along with significant energy on the transverse components caused by scattering within the 3D volume along with some contribution by topographic scattering. Observations from the SPE1 shot largely confirmed these predictions although the ratio of transverse energy relative to the vertical and radial components was in general larger than predicted. A new set of simulations has been conducted for the upcoming SPE2 shot. These include improvements to the velocity model based on SPE1 observations as well as new capabilities added to the simulation code. The most significant is the addition of a new source model within the finite difference code by using the predicted ground velocities from a hydrodynamic code (GEODYN) as driving condition on the boundaries of a cube embedded within WPP which provides a more sophisticated source modeling capability linked directly to source site materials (e.g. granite) and type and size of source. Two sets of SPE2 simulations are conducted, one with a GEODYN source and 3D complex media (no topography node spacing of 5 m) and one with a standard isotropic pre-defined time function (3D complex media with topography, node spacing of 5 m). Results were provided as time series at specific points corresponding to sensor locations for both translational (x,y,z) and rotational components. Estimates of spectral scaling for SPE2 are provided using a modified version of the Mueller-Murphy model. An estimate of expected aftershock probabilities were also provided, based on the methodology of Ford and Walter , [2010].

58 GEOSCIENCES↗

SPE 1 Data Quicklook report

The SPE 1 shot was recorded by an extensive set of instrumentation and data return was good. Here we review the far-­field data with emphasis on improving data return for the upcoming SPE shots. Initially, the data was checked for instrumentation and metadata problems. Seven sensors appear to malfunctioning, based on data appearance and amplitude. An additional four digitizers may have incorrect channel assignments for the stations. Although not visible on the shot itself, spikes appear periodically in the data and may be related to the solar panel power set-­up, although this has not been confirmed. An assessment of potential clipping for the next shot was carried out with the existing sensor emplacement and rough calculations suggest that clipping appears unlikely. Preliminary comparisons of amplitude versus yield and comparisons with synthetics are shown. A brief discussion of the planned far-field seismic analysis is given.

58 GEOSCIENCES↗

A Numerical Study of Resistivity and Hall Effects for a Compressible MHD Model

The effect of resistive, Hall, and viscous terms on the flow structure compared with compressible ideal MHD is studied numerically for a one-fluid non-ideal MHD model. The goal of the present study is to shed some light on the emerging area of non-ideal MHD modeling and simulation. Numerical experiments are performed on a hypersonic blunt body flow with future application to plasma aerodynamics flow control in reentry vehicles. Numerical experiments are also performed on a magnetized time-developing mixing layer with possible application to magnetic/turbulence mixing.

Yee, H. C.↗

Nonlinear filtering and limiting in high order methods for ideal and non-ideal MHD

The various filtering mechanisms and base scheme options of the newly developed adaptive numerical dissipation control in spatially high order filter schemes for the ideal and non-ideal magnetohydrodynamics (MHD) equations are investigated. These filter schemes are applicable to complex unsteady MHD high-speed shock/shear/turbulence problems. They also provide a natural and efficient way for the minimization of Div(B) numerical error. The type of spatial base scheme to be used in conjunction with our filter idea is very general. For example, spectral, compact and non-compact spatially central finite difference schemes are possible candidates. The adaptive numerical dissipation mechanism consists of automatic detection of different flow features as distinct sensors to signal the appropriate type and amount of numerical dissipation/filter where needed and to leave the rest of the region free from numerical dissipation contamination. The numerical dissipation considered consists of high order linear dissipation for the suppression of high frequency oscillation and the nonlinear dissipative portion of high-resolution shock-capturing methods for discontinuity capturing. The applicable nonlinear dissipative portion of high-resolution shock-capturing methods is also very general. The objective of this paper is to investigate the performance of using compact and non-compact central base schemes in conjunction with three commonly used types of nonlinear numerical dissipation for both the ideal and non-ideal MHD. This extended abstract shows the performance of three nonlinear filters in conjunction with a sixth-order non-compact spatial central base scheme. In the final paper, the high order compact spatial central base scheme will be illustrated and compared with the non-compact base scheme. The reason for the investigation of the high order compact spatial central base scheme over the non-compact base scheme is to evaluate if additional accuracy can be gained in regions of fine scale turbulence that are away from shocks/shears.

Yee,H. C.↗