Engineering Papers⌕ Search

Engineering topics

Berrill, Mark A.

Publications and source records attributed to Berrill, Mark A..

Geometric state function for two-fluid flow in porous media

Models that describe two-fluid flow in porous media suffer from a widely recognized problem that the constitutive relationships used to predict capillary pressure as a function of the fluid saturation are nonunique, thus requiring a hysteretic description. As an alternative to the traditional perspective, we consider a geometric description of the capillary pressure, which relates the average mean curvature, the fluid saturation, the interfacial area between fluids, and the Euler characteristic. The state equation is formulated using notions from algebraic topology and cast in terms of measures of the macroscale state. Synchrotron-based x-ray microcomputed tomography and high-resolution pore-scale simulation is applied to examine the uniqueness of the proposed relationship for six different porous media. We show that the geometric state function is able to characterize the microscopic fluid configurations that result from a wide range of simulated flow conditions in an averaged sense. The geometric state function can serve as a closure relationship within macroscale models to effectively remove hysteretic behavior attributed to the arrangement of fluids within a porous medium. As a result, this provides a critical missing component needed to enable a new generation of higher fidelity models to describe two-fluid flow in porous media.

42 ENGINEERING↗

QMCPACK : an open source ab initio quantum Monte Carlo package for the electronic structure of atoms, molecules and solids

QMCPACK is an open source quantum Monte Carlo package for ab-initio electronic structure calculations. It supports calculations of metallic and insulating solids, molecules, atoms, and some model Hamiltonians. Implemented real space quantum Monte Carlo algorithms include variational, diffusion, and reptation Monte Carlo. QMCPACK uses Slater-Jastrow type trial wave functions in conjunction with a sophisticated optimizer capable of optimizing tens of thousands of parameters. The orbital space auxiliary field quantum Monte Carlo method is also implemented, enabling cross validation between different highly accurate methods. The code is specifically optimized for calculations with large numbers of electrons on the latest high performance computing architectures, including multicore central processing unit (CPU) and graphical processing unit (GPU) systems. We detail the program’s capabilities, outline its structure, and give examples of its use in current research calculations. The package is available at http://www.qmcpack.org.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

Flow regimes during immiscible displacement

Fractional ow of immiscible phases occurs at the pore scale where grain surfaces and phases interfaces obstruct phase mobility. However, the larger scale behavior is described by a saturation-dependent phenomenological relationship called relative permeability. As a consequence, pore-scale parameters, such as phase topology and/ or geometry, and details of the flow regime cannot be directly related to Darcy-scale flow parameters. It is well understood that relative permeability is not a unique relationship of wetting-phase saturation and rather depends on the experimental conditions at which it is measured. Herein we use fast X-ray microcomputed tomography to image pore-scale phase arrangements during fractional flow and then forward simulate the flow regimes using the lattice-Boltzmann method to better understand the underlying pore-scale flow regimes and their influence on Darcy-scale parameters. We find that relative permeability is highly dependent on capillary number and that the Corey model fits the observed trends. At the pore scale, while phase topologies are continuously changing on the scale of individual pores, the Euler characteristic of the nonwetting phase (NWP) averaged over a sufficiently large field of view can describe the bulk topological characteristics; the Euler characteristic decreases with increasing capillary number resulting in an increase in relative permeability. Lastly, we quantify the fraction of NWP that flows through disconnected ganglion dynamics and demonstrate that this can be a significant fraction of the NWP flux for intermediate wetting-phase saturation. Furthermore, rate dependencies occur in our homogenous sample (without capillary end effect) and the underlying cause is attributed to ganglion flow that can significantly influence phase topology during the fractional flow of immiscible phases.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

Beyond Darcy's law: The role of phase topology and ganglion dynamics for two-fluid flow

Relative permeability quantifies the ease at which immiscible phases flow through porous rock and is one of the most well known constitutive relationships for petroleum engineers. It however exhibits troubling dependencies on experimental conditions and is not a unique function of phase saturation as commonly accepted in industry practices. The problem lies in the multi-scale nature of the problem where underlying disequilibrium processes create anomalous macroscopic behavior. Here we show that relative permeability rate dependencies are explained by ganglion dynamic flow. We utilize fast X-ray micro-tomography and pore-scale simulations to identify unique flow regimes during the fractional flow of immiscible phases and quantify the contribution of ganglion flux to the overall flux of non-wetting phase. We anticipate our approach to be the starting point for the development of sophisticated multi-scale flow models that directly link pore-scale parameters to macro-scale behavior. Such models will have a major impact on how we recover hydrocarbons from the subsurface, store sequestered CO 2 in geological formations, and remove non-aqueous environmental hazards from the vadose zone.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

MiniApps derived from production HPC applications using multiple programing models

Here, we have developed a set of reduced, proxy applications (“MiniApps”) based on large-scale application codes supported at the Oak Ridge Leadership Computing Facility (OLCF). The MiniApps are designed to encapsulate the details of the most important (i.e. the most time-consuming and/or unique) facets of the applications that run in production mode on the OLCF. In each case, we have produced or plan to produce individual versions of the MiniApps using different specific programing models (e.g., OpenACC, CUDA, OpenMP). We describe some of our initial observations regarding these different implementations along with estimates of how closely the MiniApps track the actual performance characteristics (in particular, the overall scalability) of the large-scale applications from which they are derived.

97 MATHEMATICS AND COMPUTING↗

Influence of phase connectivity on the relationship among capillary pressure, fluid saturation, and interfacial area in two-fluid-phase porous medium systems

Here, multiphase flow in porous medium systems is typically modeled using continuum mechanical representations at the macroscale in terms of averaged quantities. These models require closure relations to produce solvable forms. One of these required closure relations is an expression relating fluid pressures, fluid saturations, and, in some cases, the interfacial area between the fluid phases, and the Euler characteristic. An unresolved question is whether the inclusion of these additional morphological and topological measures can lead to a non-hysteretic closure relation compared to the hysteretic forms that are used in traditional models, which typically do not include interfacial areas, or the Euler characteristic. We develop a lattice-Boltzmann (LB) simulation approach to investigate the equilibrium states of a two-fluid-phase porous medium system, which include disconnected now- wetting phase features. The proposed approach is applied to a synthetic medium consisting of 1,964 spheres arranged in a random, non-overlapping, close-packed manner, yielding a total of 42,908 different equilibrium points. This information is evaluated using a generalized additive modeling approach to determine if a unique function from this family exists, which can explain the data. The variance of various model estimates is computed, and we conclude that, except for the limiting behavior close to a single fluid regime, capillary pressure can be expressed as a deterministic and non-hysteretic function of fluid saturation, interfacial area between the fluid phases, and the Euler characteristic. This work is unique in the methods employed, the size of the data set, the resolution in space and time, the true equilibrium nature of the data, the parameterizations investigated, and the broad set of functions examined. The conclusion of essentially non-hysteretic behavior provides support for an evolving class of two-fluid-phase flow in porous medium systems models.

36 MATERIALS SCIENCE↗

VERA 3.5 Installation Guide

This guide describes the structure and setup of the standard VERA development environment (VERA Dev Env) and standard VERA Third Party Libraries (TPLs) that need to be in place before installing many of the VERA simulation components. It describes everything from the initial setup on a new machine to the final build, testing, and installation of VERA components. The goal of this document is to describe how to create the directories and contents outlined in Standard VERA Dev Env Directory Structure and then obtain the remaining VERA source and build, test, and install any of the necessary VERA components on a given system. This document describes the process both for a development version of VERA and for a released tarball of the VERA sources.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗

Tracking interface and common curve dynamics for two-fluid flow in porous media

Pore-scale studies of multiphase flow in porous medium systems can be used to understand transport mechanisms and quantitatively determine closure relations that better incorporate microscale physics into macroscale models. Multiphase flow simulators constructed using the lattice Boltzmann method provide a means to conduct such studies, including both the equilibrium and dynamic aspects. Moving, storing, and analyzing the large state space presents a computational challenge when highly-resolved models are applied. We present an approach to simulate multiphase flow processes in which in-situ analysis is applied to track multiphase flow dynamics at high temporal resolution. We compute a comprehensive set of measures of the phase distributions and the system dynamics, which can be used to aid fundamental understanding and inform closure relations for macroscale models. The measures computed include microscale point representations and macroscale averages of fluid saturations, the pressure and velocity of the fluid phases, interfacial areas, interfacial curvatures, interface and common curve velocities, interfacial orientation tensors, phase velocities and the contact angle between the fluid-fluid interface and the solid surface. Test cases are studied to validate the approach and illustrate how measures of system state can be obtained and used to inform macroscopic theory.

97 MATHEMATICS AND COMPUTING↗

Asynchronous In Situ Connected-Components Analysis for Complex Fluid flows, In: ISAV '16 Proceedings of the 2nd Workshop on In Situ Infrastructures for Enabling Extreme-scale Analysis and Visualization

The simulation of multiscale physics is an important challenge for scientific computing. For this class of problem, large three-dimensional simulations are performed to advance scientific inquiry. On massively parallel computing systems, the volume of data generated by such approaches can become a productivity bottleneck if the raw data generated from the simulation is analyzed in a post-processing step. To address this, we present a physics-based framework for in situ data reduction that is theoretically grounded in multiscale averaging theory. We show how task parallelism can be exploited to concurrently perform a variety of analysis tasks with data-dependent costs, including the generation of iso-surfaces, morphological analyses, and connected components analysis. All analyses are performed in parallel using distributed memory and use the same domain decomposition as the simulation. A task management framework is constructed to leverage available parallelism within a node for analysis. The capabilities of the framework are to launch asynchronous analysis threads, manage dependencies between different tasks, promote data locality and minimize the impact of data transfers. The framework is applied to analyze GPU-based simulations of two-fluid-phase flow in porous media, generating a set of averaged measures that represents the overall system behavior. We demonstrate how the approach can be applied to perform physically-consistent analysis over fluid sub-regions determined from connected components analysis. Simulations performed on Oak Ridge National Lab's Titan supercomputer are profiled to demonstrate the performance of the associated multi-threaded in situ analysis approach for typical production simulation of two-fluid-phase flow.

Computer Science↗

Accelerated application development: The ORNL Titan experience

The use of computational accelerators such as NVIDIA GPUs and Intel Xeon Phi processors is now widespread in the high performance computing community, with many applications delivering impressive performance gains. However, programming these systems for high performance, performance portability and software maintainability has been a challenge. In this paper we discuss experiences porting applications to the Titan system. Titan, which began planning in 2009 and was deployed for general use in 2013, was the first multi-petaflop system based on accelerator hardware. To ready applications for accelerated computing, a preparedness effort was undertaken prior to delivery of Titan. In this paper we report experiences and lessons learned from this process and describe how users are currently making use of computational accelerators on Titan.

97 MATHEMATICS AND COMPUTING↗

A parallel multi-domain solution methodology applied to nonlinear thermal transport problems in nuclear fuel pins

This paper describes an efficient and nonlinearly consistent parallel solution methodology for solving coupled nonlinear thermal transport problems that occur in nuclear reactor applications over hundreds of individual 3D physical subdomains. Efficiency is obtained by leveraging knowledge of the physical domains, the physics on individual domains, and the couplings between them for preconditioning within a Jacobian Free Newton Krylov method. Details of the computational infrastructure that enabled this work, namely the open source Advanced Multi-Physics (AMP) package developed by the authors is described. Details of verification and validation experiments, and parallel performance analysis in weak and strong scaling studies demonstrating the achieved efficiency of the algorithm are presented. Furthermore, numerical experiments demonstrate that the preconditioner developed is independent of the number of fuel subdomains in a fuel rod, which is particularly important when simulating different types of fuel rods. Finally, we demonstrate the power of the coupling methodology by considering problems with couplings between surface and volume physics and coupling of nonlinear thermal transport in fuel rods to an external radiation transport code.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

A Parallel Multi-Domain Solution Methodology Applied to Nonlinear Thermal Transport Problems in Nuclear Fuel Pins

This paper describes an efficient and nonlinearly consistent parallel solution methodology for solving coupled nonlinear thermal transport problems that occur in nuclear reactor applications over hundreds of individual 3D physical subdomains. Efficiency is obtained by leveraging knowledge of the physical domains, the physics on individual domains, and the couplings between them for preconditioning within a Jacobian Free Newton Krylov method. Details of the computational infrastructure that enabled this work, namely the open source Advanced Multi-Physics (AMP) package developed by the authors is described. Details of verification and validation experiments, and parallel performance analysis in weak and strong scaling studies demonstrating the achieved efficiency of the algorithm are presented. Furthermore, numerical experiments demonstrate that the preconditioner developed is independent of the number of fuel subdomains in a fuel rod, which is particularly important when simulating different types of fuel rods. Finally, we demonstrate the power of the coupling methodology by considering problems with couplings between surface and volume physics and coupling of nonlinear thermal transport in fuel rods to an external radiation transport code.

11 NUCLEAR FUEL CYCLE AND FUEL MATERIALS↗

Block-Iterative Methods for 3D Constant-Coefficient Stencils on GPUs and Multicore CPUs

Block iterative methods are extremely important as smoothers for multigrid methods, as preconditioners for Krylov methods, and as solvers for diagonally dominant linear systems. Developing robust and efficient smoother algorithms suitable for current and evolving GPU and multicore CPU systems is a significant challenge. We address this issue in the case of constant-coefficient stencils arising in the solution of elliptic partial differential equations on structured 3D uniform and adaptively refined block structured grids. Robust, highly parallel implementations of block Jacobi and chaotic block Gauss-Seidel algorithms with exact inversion of the blocks are developed using different parallelization techniques. Experimental results for NVIDIA Fermi/Kepler GPUs and AMD multicore systems are presented.

97 MATHEMATICS AND COMPUTING↗

Integrated Radiation Transport and Nuclear Fuel Performance for Assembly-Level Simulations

The Advanced Multi-Physics (AMP) Nuclear Fuel Performance code (AMPFuel) is focused on predicting the temperature and strain within a nuclear fuel assembly to evaluate the performance and safety of existing and advanced nuclear fuel bundles within existing and advanced nuclear reactors. AMPFuel was extended to include an integrated nuclear fuel assembly capability for (one-way) coupled radiation transport and nuclear fuel assembly thermo-mechanics.

11 NUCLEAR FUEL CYCLE AND FUEL MATERIALS↗

Fiscal Year 2011 Infrastructure Refactorizations in AMP

In Fiscal Year 2011 (FY11), the AMP (Advanced MultiPhysics) Nuclear Fuel Performance code went through a thorough review and refactorization based on the lessons-learned from the previous year, in which the version 0.9 of the software was released as a prototype. This report describes the refactorization work that has occurred or is in progress during FY11.

11 NUCLEAR FUEL CYCLE AND FUEL MATERIALS↗