Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Grid Logging”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 55 records · Page 3

Using a Canopy Model Framework to Improve Large-Eddy Simulations of the Neutral Atmospheric Boundary Layer in the Weather Research and Forecasting Model

A canopy model framework is implemented in the Weather Research and Forecasting Model to improve the accuracy of large-eddy simulations (LES) of the atmospheric boundary layer (ABL). The model includes two options that depend on the scale of surface roughness elements. A resolved canopy model, typically used to model flow through vegetation canopies, is employed when roughness elements are resolved by the vertical LES grid. In the case of unresolved roughness, a modified “pseudocanopy model” is developed to distribute drag over a shallow layer above the surface. Both canopy model options are validated against idealized test cases in neutral stability conditions and are shown to improve surface layer velocity profiles relative to simulations employing Monin–Obukhov similarity theory (MOST), which is commonly used as a surface boundary condition in ABL models. Use of the canopy model framework also leads to increased levels of resolved turbulence kinetic energy and turbulent stresses. Because LES of the ABL has a well-known difficulty recovering the expected logarithmic velocity profile (log law) in the surface layer, particular focus is placed on using the pseudocanopy model to alleviate this issue over a range of model configurations. Tests with varying surface roughness values, LES closures, and grid aspect ratios confirm that the pseudocanopy model generally improves log-law agreement relative to simulations that employ a standard MOST boundary condition. The canopy model framework thus represents a low-cost, easy-to-implement method for improving LES of the ABL.

17 WIND ENERGY↗

Solving Graph Laplacian Systems Through Recursive Bisections and Two-Grid Preconditioning

We present a parallelizable direct method for computing the solution to graph Laplacian-based linear systems derived from graphs that can be hierarchically bipartitioned with small edge cuts. For a graph of size n with constant-size edge cuts, our method decomposes a graph Laplacian in time O(n log n), and then uses that decomposition to perform a linear solve in time O(n log n). We then use the developed technique to design a preconditioner for graph Laplacians that do not have this property. Finally, we augment this preconditioner with a two-grid method that accounts for much of the preconditioner's weaknesses. We present an analysis of this method, as well as a general theorem for the condition number of a general class of two-grid support graph-based preconditioners. Numerical experiments illustrate the performance of the studied methods.

97 MATHEMATICS AND COMPUTING↗

SU-F-T-190: Homogeneity Study of Proton and Heavy Ion Scanning Beams Based On Different Compositions of Spot Sizes (FWHM) and Grid Sizes

Purpose: Different particle scanning beam delivery systems have different delivery accuracies. This study was performed to determine, for our particle treatment system, an appropriate composition (n=FWHM/GS) of spot size(FWHM) and grid size (GS), which can provide homogenous delivered dose distributions for both proton and heavy ion scanning beam radiotherapy. Methods: We analyzed the delivery errors of our beam delivery system using log files from the treatment of 28 patients. We used a homemade program to simulate square fields for different n values with and without considering the delivery errors and analyzed the homogeneity. All spots were located on a rectilinear grid with equal spacing in the × and y directions. After that, we selected 7 energy levels for both proton and carbon ions. For each energy level, we made 6 square field plans with different n values (1, 1.5, 2, 2.5, 3, 3.5). Then we delivered those plans and used films to measure the homogeneity of each field. Results: For program simulation without delivery errors, when n≥1.1 the homogeneity can be within ±3%. For both proton and carbon program simulations with delivery errors and film measurements, the homogeneity can be within ±3% when n≥2.5. Conclusion: For our facility with system errors, the n≥2.5 is appropriate for maintaining homogeneity within ±3%.

60 APPLIED LIFE SCIENCES↗

NEW ATLAS9 AND MARCS MODEL ATMOSPHERE GRIDS FOR THE APACHE POINT OBSERVATORY GALACTIC EVOLUTION EXPERIMENT (APOGEE)

We present a new grid of model photospheres for the SDSS-III/APOGEE survey of stellar populations of the Galaxy, calculated using the ATLAS9 and MARCS codes. New opacity distribution functions were generated to calculate ATLAS9 model photospheres. MARCS models were calculated based on opacity sampling techniques. The metallicity ([M/H]) spans from -5 to 1.5 for ATLAS and -2.5 to 0.5 for MARCS models. There are three main differences with respect to previous ATLAS9 model grids: a new corrected H{sub 2}O line list, a wide range of carbon ([C/M]) and {alpha} element [{alpha}/M] variations, and solar reference abundances from Asplund et al. The added range of varying carbon and {alpha}-element abundances also extends the previously calculated MARCS model grids. Altogether, 1980 chemical compositions were used for the ATLAS9 grid and 175 for the MARCS grid. Over 808,000 ATLAS9 models were computed spanning temperatures from 3500 K to 30,000 K and log g from 0 to 5, where larger temperatures only have high gravities. The MARCS models span from 3500 K to 5500 K, and log g from 0 to 5. All model atmospheres are publicly available online.

79 ASTRONOMY AND ASTROPHYSICS↗

HUBBLE SPACE TELESCOPE SPECTROPHOTOMETRY AND MODELS FOR SOLAR ANALOGS

Absolute flux distributions for seven solar analog stars are measured from 0.3 to 2.5 {mu}m by Hubble Space Telescope (HST) spectrophotometry. In order to predict the longer wavelength mid-IR fluxes that are required for James Webb Space Telescope calibration, the HST spectral energy distributions are fit with Castelli and Kurucz model atmospheres; and the results are compared with fits from the MARCS model grid. The rms residuals in 10 broadband bins are all <0.5% for the best fits from both model grids. However, the fits differ systematically: the MARCS fits are 40-100 K hotter in T {sub eff}, 0.25-0.80 higher in log g, 0.01-0.10 higher in log z, and 0.008-0.021 higher in the reddening E(B - V), probably because their specifications include different metal abundances. Despite these differences in the parameters of the fits, the predicted mid-IR fluxes differ by only {approx}1%; and the modeled flux distributions of these G stars have an estimated ensemble accuracy of 2% out to 30 {mu}m.

79 ASTRONOMY AND ASTROPHYSICS↗

A NEW SYNTHETIC LIBRARY OF THE NEAR-INFRARED Ca II TRIPLET INDICES. I. INDEX DEFINITION, CALIBRATION, AND RELATIONS WITH STELLAR ATMOSPHERIC PARAMETERS

Adopting the SPECTRUM package, which is a stellar spectral synthesis program, we have synthesized a comprehensive set of 2890 near-infrared (NIR) synthetic spectra with a resolution and wavelength sampling similar to the Sloan Digital Sky Survey (SDSS) and the forthcoming Large Sky Area Multi-Object Fiber Spectroscopic Telescope (LAMOST) spectra. During the synthesis, we applied the 'New grids of ATLAS9 Model Atmosphere' to develop a grid of local thermodynamic equilibrium model atmospheres for effective temperatures (T{sub eff}) ranging from 3500 to 7500 K, for surface gravities (log g) from 0.5 to 5.0 dex, for metallicities ([Fe/H]) from -4.0 to 0.5 dex, and for solar ([{alpha}/Fe] = 0.0 dex) and non-solar ([{alpha}/Fe] = +0.4 dex) abundances. This synthetic stellar library is composed of 1350 solar scaled abundance (SSA) and 1530 non-solar scaled abundance (NSSA) spectra, grounding on which we have defined a new set of NIR Ca II triplet indices and an index CaT as the sum of the three. These defined indices were automatically measured on every spectrum of the synthetic stellar library and calibrated with the indices computed on the observational spectra from the INDO-U.S. stellar library. In order to check the effect of {alpha}-element enhancement on the so-defined Ca II indices, we compared indices measured on the SSA spectra with those on the NSSA ones at the same trine of stellar parameters (T{sub eff}, log g, [Fe/H]); luckily, little influences of {alpha}-element enhancement were found. Furthermore, comparisons of our synthetic indices with the observational ones from measurements on the INDO-U.S. stellar library, the SDSS-DR7 and SDSS-DR8 spectroscopic survey are presented, respectively, for dwarfs and giants in specific. For dwarfs, our synthetic indices could well reproduce the behaviors of the observational indices versus stellar parameters, which verifies the validity of our index definitions for dwarfs. For giants, the consistency between our synthetic indices and the observational ones does not appear to be as good. However, a new synthetic library of NIR Ca II indices has been founded for deeper studies on the NIR wave band of stellar spectra, and this library is particularly appropriate for the SDSS and the forthcoming LAMOST stellar spectra. We have regressed the strength of the CaT index as a function of stellar parameters for both dwarfs and giants after a series of experimental investigations into relations of the indices with stellar parameters. For dwarfs, log g has little effect on the indices, while [Fe/H] and T{sub eff} play a role together. The leading factor is probably [Fe/H], which changes the strength of the indices by a positive trend. For giants, log g starts to influence the strength of the indices by a negative trend for the metal-poor, and even impact deeply for the metal-rich; besides, [Fe/H] and T{sub eff} still matter. In addition, we briefly discussed the major differences between our Ca II triplet indices and the CaHK doublet indices. Ultimately, a supplemental experiment was carried out to show that spectral noises do have effects on our set of NIR Ca II indices. However, the influence is not weak enough to be ignored if the signal-to-noise ratio falls below 20.

79 ASTRONOMY AND ASTROPHYSICS↗

A Communication-Avoiding 3D LU Factorization Algorithm for Sparse Matrices

We propose a new algorithm to improve the strong scalability of right-looking sparse LU factorization on distributed memory systems. Our 3D sparse LU algorithm uses a three-dimensional MPI process grid, aggressively exploits elimination tree parallelism and trades off increased memory for reduced per-process communication. We also analyze the asymptotic improvements for planar graphs (e.g., from 2D grid or mesh domains) and certain non-planar graphs (specifically for 3D grids and meshes). For planar graphs with n vertices, our algorithm reduces communication volume asymptotically in n by a factor of O{log n} and latency by a factor of O{log n}. For non-planar cases, our algorithm can reduce the per-process communication volume by 3× and latency by O{n^1/3} times. In all cases, the memory needed to achieve these gains is a constant factor. We implemented our algorithm by extending the 2D data structure used in superLU. Our new 3D code achieves speedups up to 27× for planar graphs and up to 3.3× for non-planar graphs over the baseline 2D superLU when run on 24,000 cores of a Cray XC30.

Sao, Piyush↗

Geologic Modeling and Simulation Report for the Aquistore Project

In this update, a regional-scale model extends beyond the 34-square-kilometer PTRC 3-D seismic survey area. A local grid refinement (LGR) system near both the injection and observation wells was introduced for the history-matching and uncertainty analysis. Spinner log survey and pressure test data provided by partners were used and evaluated to adjust the nearwellbore local permeability in order to history-match the field pressure data. As of January 9, 2016, approximately 24,000 tonnes of CO 2 has been injected during a series of relatively short operating periods. However, the injection rate during these short operating periods increased near the end of year 2015 to a level of 300 to 500 tonnes per day, based on the quantity of CO 2 made available from the pipeline.

54 ENVIRONMENTAL SCIENCES↗

Alabama Carbon Storage: Data Sharing and Engagement (Final Report)

This report is the final technical report on Alabama Carbon Storage: Data Sharing Engagement (ACS:DSE) project activities. The goals of the ACS:DSE project are to compile geologic, geophysical, infrastructure, and other relevant CCUS datasets for the study area and develop a geologic model of the study area; develop an online platform to serve data to stakeholders; engage with the public, students, and industry to educate them about CCUS and the data platform; and ensure energy and environmental justice is central to all aspects of the project. Datasets compiled and expanded include formation depths and elevations, digital geophysical well logs, reservoir properties, geologic structures, and geologic models. The geologic data were used to create a three-dimensional geologic model, structure grids, structure contour maps, and fault trace maps. In addition to downloadable datasets, links to CCUS relevant regulatory agencies (e.g., OGB, U.S. Environmental Protection Agency) and sources for infrastructure and educational information were included on the website Educational materials on CCUS for use by K-12 teachers were produced as part of the ACS:DSE project.

01 COAL, LIGNITE, AND PEAT↗

Fast mix table construction for material discretization

An effective hybrid Monte Carlo-deterministic implementation typically requires the approximation of a continuous geometry description with a discretized piecewise-constant material field. The inherent geometry discretization error can be reduced somewhat by using material mixing, where multiple materials inside a discrete mesh voxel are homogenized. Material mixing requires the construction of a 'mix table,' which stores the volume fractions in every mixture so that multiple voxels with similar compositions can reference the same mixture. Mix table construction is a potentially expensive serial operation for large problems with many materials and voxels. We formulate an efficient algorithm to construct a sparse mix table in O(number of voxels x log number of mixtures) time. The new algorithm is implemented in ADVANTG and used to discretize continuous geometries onto a structured Cartesian grid. When applied to an end-of-life MCNP model of the High Flux Isotope Reactor with 270 distinct materials, the new method improves the material mixing time by a factor of 100 compared to a naive mix table implementation. (authors)

97 MATHEMATICS AND COMPUTING↗

Fast Mix Table Construction for Material Discretization

An effective hybrid Monte Carlo--deterministic implementation typically requires the approximation of a continuous geometry description with a discretized piecewise-constant material field. The inherent geometry discretization error can be reduced somewhat by using material mixing, where multiple materials inside a discrete mesh voxel are homogenized. Material mixing requires the construction of a ``mix table,'' which stores the volume fractions in every mixture so that multiple voxels with similar compositions can reference the same mixture. Mix table construction is a potentially expensive serial operation for large problems with many materials and voxels. We formulate an efficient algorithm to construct a sparse mix table in $O(\text{number of voxels}\times \log \text{number of mixtures})$ time. The new algorithm is implemented in ADVANTG and used to discretize continuous geometries onto a structured Cartesian grid. When applied to an end-of-life MCNP model of the High Flux Isotope Reactor with 270 distinct materials, the new method improves the material mixing time by a factor of 100 compared to a naive mix table implementation.

Johnson, Seth R↗

THE MASS MIXING LENGTH IN CONVECTIVE STELLAR ENVELOPES

The scale length over which convection mixes mass in a star can be calculated as the inverse of the vertical derivative of the unidirectional (up or down) mass flux. This is related to the mixing length in the mixing length theory of stellar convection. We give the ratio of mass mixing length to pressure scale height for a grid of three-dimensional surface convection simulations, covering from 4300 K to 6900 K on the main sequence, and up to giants at log g = 2.2, all for solar composition. These simulations also confirm what is already known from solar simulations that convection does not proceed by discrete convective elements, but rather as a continuous, slow, smooth, warm upflow and turbulent, entropy deficient, fast down drafts. This convective topology also results in mixing on a scale comparable to the classic mixing length formulation, and is simply a consequence of mass conservation on flows in a stratified atmosphere.

79 ASTRONOMY AND ASTROPHYSICS↗

Integrated Modeling Report for CO 2 Storage with Enhanced Oil Recovery in Northern Michigan

The Midwest Regional Carbon Sequestration Partnership (MRCSP) was founded in 2003 as part of the U.S. Department of Energy’s (DOE’s) Regional Carbon Sequestration Partnership initiative. Since its founding, MRCSP has made significant strides toward making CCUS a viable option for states in the region. The public/private consortium, funded through the DOE Regional Carbon Sequestration Initiative, brings together nearly 40 industry partners and 10 states. Battelle, as the project lead, oversees research, development and operations and coordinates activities among the partners. The incremental, phased approach has built a valuable knowledge base for the industry and paved the way for commercial-scale adoption of CCUS technologies. From 2008 to 2020, MRCSP Phase III focused on the development of large-scale injection projects. This report is part of a series of reports prepared under the Midwestern Regional Carbon Sequestration Partnership (MRCSP) Phase III (Development Phase). These reports summarize and detail the findings of the work conducted under the Phase III project. MRCSP has been investigating various reservoir characterization and modeling technologies related to Carbon Capture, Utilization, and Storage (CCUS) in conjunction with carbon dioxide enhanced oil recovery (CO 2 -EOR) operations in multiple Silurian-age (Niagaran), oil-bearing, carbonate pinnacle reefs in northern Michigan, USA. This report provides a comprehensive discussion of reservoir modeling studies that were conducted for tracking oil production, forecasting CO 2 plume migration, and estimating associated storage in a number of reefs that were at different stages of their CO 2 -EOR life cycle. The modeling process for simulating oil production, CO 2 injection, and associated storage in these reefs entails two phases. The first phase, geologic framework modeling, integrates all pertinent geological and geophysical data (from logs, cores and seismic surveys) about reservoir structure, geometry, rock types, and property distributions (porosity, permeability, water saturation) into a 3-D distributed grid-based static earth model (SEM). The second phase, dynamic reservoir modeling, uses the SEM as a platform to simulate the movement of oil, gas, water, and CO 2 within the reservoir during primary hydrocarbon production, as well as during subsequent phases such as CO 2 -injection assisted EOR, plume migration, and associated storage. In addition, an assessment of coupled process effects is also carried out, where the impacts of geochemical and geo-mechanical processed induced by CO 2 injection are studied.

01 COAL, LIGNITE, AND PEAT↗

Modeling the near-ultraviolet band of GK stars. III. Dependence on abundance pattern

We extend the grid of non-LTE (NLTE) models presented in Paper II to explore variations in abundance pattern in two ways: (1) the adoption of the Asplund et al. (GASS10) abundances, (2) for stars of metallicity, [M/H], of –0.5, the adoption of a non-solar enhancement of α-elements by +0.3 dex. Moreover, our grid of synthetic spectral energy distributions (SEDs) is interpolated to a finer numerical resolution in both T {sub eff} (ΔT {sub eff} = 25 K) and log g (Δlog g = 0.25). We compare the values of T {sub eff} and log g inferred from fitting LTE and NLTE SEDs to observed SEDs throughout the entire visible band, and in an ad hoc 'blue' band. We compare our spectrophotometrically derived T {sub eff} values to a variety of T {sub eff} calibrations, including more empirical ones, drawn from the literature. For stars of solar metallicity, we find that the adoption of the GASS10 abundances lowers the inferred T {sub eff} value by 25-50 K for late-type giants, and NLTE models computed with the GASS10 abundances give T {sub eff} results that are marginally in better agreement with other T {sub eff} calibrations. For stars of [M/H] = –0.5 there is marginal evidence that adoption of α-enhancement further lowers the derived T {sub eff} value by 50 K. Stellar parameters inferred from fitting NLTE models to SEDs are more dependent than LTE models on the wavelength region being fitted, and we find that the effect depends on how heavily line blanketed the fitting region is, whether the fitting region is to the blue of the Wien peak of the star's SED, or both.

79 ASTRONOMY AND ASTROPHYSICS↗

Assessment of CO 2 Storage Resources in Depleted Oil and Gas Fields in the Ship Shoal Area, Gulf of Mexico

The Gulf of Mexico is one of the most important regions in the United States for energy resources and infrastructure. Gulf of Mexico federal offshore oil and gas production accounts for 17% of total U.S. crude oil production and 5% of total U.S. gas production (EIA Gulf of Mexico Fact Sheet). This region presents an excellent combination of high need and significant opportunity for large scale geologic storage of CO 2 . The Ship Shoal Area is located about 20 miles offshore Louisiana within the Gulf Coast federal waters. Miocene and Pliocene sediments in the Ship Shoal Area are proven to provide excellent and secure traps for oil and gas. The Ship Shoal Area contains a large number of depleted oil and gas fields either currently abandoned, or planned for abandonment by 2025, which may provide very significant potential CO 2 storage capacity. GeoMechanics Technologies conducted a comprehensive research project to better characterize Neogene sediments in the Ship Shoal Area for high volume CO 2 storage. The data generated from literature, well data, well log, and formation evaluation were input into Rockwork 16 geologic software to create a detailed geologic model for the Ship Shoal study area. The geologic grids were then fed into TOUGH2, the gas migration model software and FLAC 3D, the geomechanical model software. The fluid flow modeling for both the Miocene and Pliocene simulation from both Block 107 and Block 84 indicated that Pliocene and Miocene are a good reservoir for the CO 2 sequestration. Thirty years of CO 2 injection (at a rate of 1 million metric tons CO 2 per year) and 30 years of observation simulations were run with the fluid flow modeling. The 3D geomechanical modeling for Ship Shoal Block 107 and Block 84 were developed to evaluate induced surface deformation and potential fault reactivation after 30 years CO 2 injection at base of Pliocene and at upper Miocene injection locations. Evaluation results indicate low to no risks for fault slips or fault reactivation. Twelve well bores and 76 well bores for Ship Shoal Blocks 84 and 107 respectively were reviewed for their cement history. Most wells (57 out of 76 and 7 out of 12 wells in Ship Shoal Block 107 and 84 respectively) have good integrity. Nineteen and 5 wells (in Ship Shoal Block 107 and 84 respectively) with no top plug, incomplete cement or Plug and Abandonment information are given yellow cautionary indicators. These cautionary wells may provide leakage paths of CO 2 through the well bores to the USDWs. Using our Quantitative Risk & Decision Analysis Tool (QRDAT) for caprock integrity evaluation, we compared Ship Shoal’s risk to that of In Salah, Sleipner, Kevin Dome, Loudon, Illinois Industrial CCS and Wilmington Graben. We found the risk at the Ship Shoal Blocks 84 and 107 fields are similar to the known CO 2 active sequestration sites, but lower than the Wilmington Graben turbidities offshore California studied site. The risk of natural seismicity in the Gulf of Mexico is relatively low. Geomechanics Technologies has documented the top 25 CO 2 emission sources within the close proximity of the Ship Shoal Block 84 and 107 fields. All the offshore and onshore pipelines have been digitized and can be viewed in an interactive website (http://www.geomechanicstech.com/shipshoal.html). We also performed a feasibility study on the potential for converting existing oil or gas pipelines for CO 2 transport. There are abandoned, idled and retired onshore and offshore pipelines. However, there is no standard specification for maximum pipeline pressures needed for CO 2 transport. It is a function of design, materials, and testing provided; thus it will be the responsibility of the pipeline operator to correctly determine, maintain and operate within the limits of the pipelines. There are a few transit corridors extending from onshore Louisiana (Cailou Bay) to offshore trunk-lines. The cost for constructing a pipeline has increased about 46% from 2015 to an average cost of $5,064,046 per mile in 2016. We use the NETL CO 2 storage resource mass estimate to calculate the potential resources for Ship Shoal Block 84 and 107 fields using the depleted oil and gas reservoir volume and sand volume generated from our geologic modeling. The estimated storage resource results are greater when using the sand volumes derived through geologic modeling than the oil and gas reservoir data. The difference is due to the depleted oil and gas reservoir information not accounting for the water-flooded sand located below the oil/gas-water contact or unproductive sand units. Resource calculation using the sand volume obtained through geologic modeling overestimate the storage capacity as the model accounts for all sand within the formation, not just the interconnected sand.

02 PETROLEUM↗

An efficient, high-order probabilistic collocation method on sparse grids for three-dimensional flow and solute transport in randomly heterogeneous porous media

In this study, a probabilistic collocation method (PCM) on sparse grids was used to solve stochastic equations describing flow and transport in three-dimensional in saturated, randomly heterogeneous porous media. Karhunen-Lo\`{e}ve (KL) decomposition was used to represent the three-dimensional log hydraulic conductivity $Y=\ln K_s$. The hydraulic head $h$ and average pore-velocity $\bf v$ were obtained by solving the three-dimensional continuity equation coupled with Darcy's law with random hydraulic conductivity field. The concentration was computed by solving a three-dimensional stochastic advection-dispersion equation with stochastic average pore-velocity $\bf v$ computed from Darcy's law. PCM is an extension of the generalized polynomial chaos (gPC) that couples gPC with probabilistic collocation. By using the sparse grid points, PCM can handle a random process with large number of random dimensions, with relatively lower computational cost, compared to full tensor products. Monte Carlo (MC) simulations have also been conducted to verify accuracy of the PCM. By comparing the MC and PCM results for mean and standard deviation of concentration, it is evident that the PCM approach is computational more efficient than Monte Carlo simulations. Unlike the conventional moment-equation approach, there is no limitation on the amplitude of random perturbation in PCM. Furthermore, PCM on sparse grids can efficiently simulate solute transport in randomly heterogeneous porous media with large variances.

54 ENVIRONMENTAL SCIENCES↗

Stochastic multiscale flux basis for Stokes-Darcy flows

Three algorithms are developed for uncertainty quantification in modeling coupled Stokes and Darcy flows. The porous media may consist of multiple regions with different properties. The permeability is modeled as a non-stationary stochastic variable, with its log represented as a sum of local Karhunen-Loève (KL) expansions. The problem is approximated by stochastic collocation on either tensor-product or sparse grids, coupled with a multiscale mortar mixed finite element method for the spatial discretization. A non-overlapping domain decomposition algorithm reduces the global problem to a coarse scale mortar interface problem, which is solved by an iterative solver, for each stochastic realization. In the traditional domain decomposition implementation, each subdomain solves a local Dirichlet or Neumann problem in every interface iteration. To reduce this cost, two additional algorithms based on deterministic or stochastic multiscale flux basis are introduced. The basis consists of the local flux (or velocity trace) responses from each mortar degree of freedom. It is computed for each subdomain independently before the interface iteration begins. The use of the multiscale flux basis avoids the need for subdomain solves on each iteration. The deterministic basis is computed at each stochastic collocation and used only at this realization. The stochastic basis is formed by further looping over all local realizations of a subdomain's KL region before the stochastic collocation begins. It is used over multiple realizations. Numerical tests are presented to illustrate the performance of the three algorithms, with the stochastic multiscale flux basis showing significant savings in computational cost compared to the other two algorithms.

97 MATHEMATICS AND COMPUTING↗

A communication-avoiding 3D algorithm for sparse LU factorization on heterogeneous systems

We propose a new algorithm to improve the strong scalability of right-looking sparse LU factorization on distributed memory systems. Our 3D algorithm for sparse LU uses a three-dimensional MPI process grid, exploits elimination tree parallelism, and trades off increased memory for reduced per-process communication. We also analyze the asymptotic improvements for planar graphs (e.g., those arising from 2D grid or mesh discretizations) and certain non-planar graphs (specifically for 3D grids and meshes). For a planar graph with $n$ vertices, our algorithm reduces communication volume asymptotically in $n$ by a factor of $\mathscr{O}$ $\Big(\sqrt{log \ n}\Big)$ and latency by a factor of $\mathscr{O}$ $(log \ n)$. For nonplanar cases, our algorithm can reduce the per-process communication volume by 3× and latency by $\mathscr{O}$ $\Big(n^\frac13\Big)$ times. In all cases, the memory needed to achieve these gains is a constant factor. We implemented our algorithm by extending the 2D data structure used in SuperLU_DIST. Our new 3D code achieves empirical speedups up to 27× for planar graphs and up to 3.3× for non-planar graphs over the baseline 2D SuperLU_DIST when run on 24,000 cores of a Cray XC30. We extend the 3D algorithm for heterogeneous architectures by adding the Highly Asynchronous Lazy Offload (Halo) algorithm for co-processor offload. On 4096 nodes of a Cray XK7 with 32,768 CPU cores and 4096 Nvidia K20x GPUs, the 3D algorithm achieves empirical speedups up to 24× for planar graphs and 3.5× for non-planar graphs over the baseline 2D SuperLU_DIST with co-processor acceleration.

97 MATHEMATICS AND COMPUTING↗