Engineering Papers⌕ Search

Engineering topics

Philip, Bobby

Publications and source records attributed to Philip, Bobby.

Fault Diagnosis of Hybrid Computing Systems Using Chaotic-Map Method

Computing systems are becoming increasingly complex with nodes consisting of a combination of multi-core central processing units (CPUs), many integrated core (MIC) and graphics processing unit (GPU) accelerators. These computing units and their interconnections are subject to different classes of hardware and software faults, which should be detected to support mitigation measures. We present the chaotic-map method that uses the exponential divergence and wide Fourier properties of the trajectories, combined with memory allocations and assignments to diagnose component-level faults in these hybrid computing systems. We propose lightweight codes that utilize highly parallel chaotic-map computations tailored to isolate faults in arithmetic units, memory elements and interconnects. The diagnosis module on a node utilizes pthreads to place chaotic-map threads on CPU and MIC cores, and CUDA C and OpenCL kernels on GPU blocks. We present experimental diagnosis results on five multi-core CPUs; one MIC; and, seven GPUs with typical diagnosis run-times under a minute.

Rao, Nageswara↗

An assessment of coupling algorithms for nuclear reactor core physics simulations

This paper evaluates the performance of multiphysics coupling algorithms applied to a light water nuclear reactor core simulation. The simulation couples the k-eigenvalue form of the neutron transport equation with heat conduction and subchannel flow equations. We compare Picard iteration (block Gauss–Seidel) to Anderson acceleration and multiple variants of preconditioned Jacobian-free Newton–Krylov (JFNK). The performance of the methods are evaluated over a range of energy group structures and core power levels. A novel physics-based approximation to a Jacobian-vector product has been developed to mitigate the impact of expensive on-line cross section processing steps. Numerical simulations demonstrating the efficiency of JFNK and Anderson acceleration relative to standard Picard iteration are performed on a 3D model of a nuclear fuel assembly. Both criticality (k-eigenvalue) and critical boron search problems are considered.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

Communication Characterization and Optimization of Applications Using Topology-Aware Task Mapping on Large Supercomputers

On large supercomputers, the job scheduling systems may assign a non-contiguous node allocation for user applications depending on available resources. With parallel applications using MPI (Message Passing Interface), the default process ordering does not take into account the actual physical node layout available to the application. This contributes to non-locality in terms of physical network topology and impacts communication performance of the application. In order to mitigate such performance penalties, this work describes techniques to identify suitable task mapping that takes the layout of the allocated nodes as well as the application's communication behavior into account. During the first phase of this research, we instrumented and collected performance data to characterize communication behavior of critical US DOE (United States - Department of Energy) applications using an augmented version of the mpiP tool. Subsequently, we developed several reordering methods (spectral bisection, neighbor join tree etc.) to combine node layout and application communication data for optimized task placement. We developed a tool called mpiAproxy to facilitate detailed evaluation of the various reordering algorithms without requiring full application executions. This work presents a comprehensive performance evaluation (14,000 experiments) of the various task mapping techniques in lowering communication costs on Titan, the leadership class supercomputer at Oak Ridge National Laboratory.

97 MATHEMATICS AND COMPUTING↗

Communication Characterization and Optimization of Applications Using Topology-Aware Task Mapping on Large Supercomputers

On large supercomputers, the job scheduling systems may assign a non-contiguous node allocation for user applications depending on available resources. With parallel applications using MPI (Message Passing Interface), the default process ordering does not take into account the actual physical node layout available to the application. This contributes to non-locality in terms of physical network topology and impacts communication performance of the application. In order to mitigate such performance penalties, this work describes techniques to identify suitable task mapping that takes the layout of the allocated nodes as well as the application's communication behavior into account. During the first phase of this research, we instrumented and collected performance data to characterize communication behavior of critical US DOE (United States - Department of Energy) applications using an augmented version of the mpiP tool. Subsequently, we developed several reordering methods (spectral bisection, neighbor join tree etc.) to combine node layout and application communication data for optimized task placement. We developed a tool called mpiAproxy to facilitate detailed evaluation of the various reordering algorithms without requiring full application executions. This work presents a comprehensive performance evaluation (14,000 experiments) of the various task mapping techniques in lowering communication costs on Titan, the leadership class supercomputer at Oak Ridge National Laboratory.

Sreepathi, Sarat↗

Accelerated application development: The ORNL Titan experience

The use of computational accelerators such as NVIDIA GPUs and Intel Xeon Phi processors is now widespread in the high performance computing community, with many applications delivering impressive performance gains. However, programming these systems for high performance, performance portability and software maintainability has been a challenge. In this paper we discuss experiences porting applications to the Titan system. Titan, which began planning in 2009 and was deployed for general use in 2013, was the first multi-petaflop system based on accelerator hardware. To ready applications for accelerated computing, a preparedness effort was undertaken prior to delivery of Titan. In this paper we report experiences and lessons learned from this process and describe how users are currently making use of computational accelerators on Titan.

97 MATHEMATICS AND COMPUTING↗

A parallel multi-domain solution methodology applied to nonlinear thermal transport problems in nuclear fuel pins

This paper describes an efficient and nonlinearly consistent parallel solution methodology for solving coupled nonlinear thermal transport problems that occur in nuclear reactor applications over hundreds of individual 3D physical subdomains. Efficiency is obtained by leveraging knowledge of the physical domains, the physics on individual domains, and the couplings between them for preconditioning within a Jacobian Free Newton Krylov method. Details of the computational infrastructure that enabled this work, namely the open source Advanced Multi-Physics (AMP) package developed by the authors is described. Details of verification and validation experiments, and parallel performance analysis in weak and strong scaling studies demonstrating the achieved efficiency of the algorithm are presented. Furthermore, numerical experiments demonstrate that the preconditioner developed is independent of the number of fuel subdomains in a fuel rod, which is particularly important when simulating different types of fuel rods. Finally, we demonstrate the power of the coupling methodology by considering problems with couplings between surface and volume physics and coupling of nonlinear thermal transport in fuel rods to an external radiation transport code.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

A projection preconditioner for solving the implicit immersed boundary equations

This study presents a method for solving the linear semi-implicit immersed boundary equations which avoids the severe time step restriction presented by explicit-time methods. The Lagrangian variables are eliminated via a Schur complement to form a purely Eulerian saddle point system, which is preconditioned by a projection operator and then solved by a Krylov subspace method. From the viewpoint of projection methods, we derive an ideal preconditioner for the saddle point problem and compare the efficiency of a number of simpler preconditioners that approximate this perfect one. For low Reynolds number and high stiffness, one particular projection preconditioner yields an efficiency improvement of the explicit IB method by a factor around thirty. Substantial speed-ups over explicit-time method are achieved for Reynolds number below 100. In conclusion, this speedup increases as the Eulerian grid size and/or the Reynolds number are further reduced.

42 ENGINEERING↗

Geometric multigrid for an implicit-time immersed boundary method

The immersed boundary (IB) method is an approach to fluid-structure interaction that uses Lagrangian variables to describe the deformations and resulting forces of the structure and Eulerian variables to describe the motion and forces of the fluid. Explicit time stepping schemes for the IB method require solvers only for Eulerian equations, for which fast Cartesian grid solution methods are available. Such methods are relatively straightforward to develop and are widely used in practice but often require very small time steps to maintain stability. Implicit-time IB methods permit the stable use of large time steps, but efficient implementations of such methods require significantly more complex solvers that effectively treat both Lagrangian and Eulerian variables simultaneously. Moreover, several different approaches to solving the coupled Lagrangian-Eulerian equations have been proposed, but a complete understanding of this problem is still emerging. This paper presents a geometric multigrid method for an implicit-time discretization of the IB equations. This multigrid scheme uses a generalization of box relaxation that is shown to handle problems in which the physical stiffness of the structure is very large. Numerical examples are provided to illustrate the effectiveness and efficiency of the algorithms described herein. Finally, these tests show that using multigrid as a preconditioner for a Krylov method yields improvements in both robustness and efficiency as compared to using multigrid as a solver. They also demonstrate that with a time step 100–1000 times larger than that permitted by an explicit IB method, the multigrid-preconditioned implicit IB method is approximately 50–200 times more efficient than the explicit method.

97 MATHEMATICS AND COMPUTING↗

A Parallel Multi-Domain Solution Methodology Applied to Nonlinear Thermal Transport Problems in Nuclear Fuel Pins

This paper describes an efficient and nonlinearly consistent parallel solution methodology for solving coupled nonlinear thermal transport problems that occur in nuclear reactor applications over hundreds of individual 3D physical subdomains. Efficiency is obtained by leveraging knowledge of the physical domains, the physics on individual domains, and the couplings between them for preconditioning within a Jacobian Free Newton Krylov method. Details of the computational infrastructure that enabled this work, namely the open source Advanced Multi-Physics (AMP) package developed by the authors is described. Details of verification and validation experiments, and parallel performance analysis in weak and strong scaling studies demonstrating the achieved efficiency of the algorithm are presented. Furthermore, numerical experiments demonstrate that the preconditioner developed is independent of the number of fuel subdomains in a fuel rod, which is particularly important when simulating different types of fuel rods. Finally, we demonstrate the power of the coupling methodology by considering problems with couplings between surface and volume physics and coupling of nonlinear thermal transport in fuel rods to an external radiation transport code.

11 NUCLEAR FUEL CYCLE AND FUEL MATERIALS↗

Block-Iterative Methods for 3D Constant-Coefficient Stencils on GPUs and Multicore CPUs

Block iterative methods are extremely important as smoothers for multigrid methods, as preconditioners for Krylov methods, and as solvers for diagonally dominant linear systems. Developing robust and efficient smoother algorithms suitable for current and evolving GPU and multicore CPU systems is a significant challenge. We address this issue in the case of constant-coefficient stencils arising in the solution of elliptic partial differential equations on structured 3D uniform and adaptively refined block structured grids. Robust, highly parallel implementations of block Jacobi and chaotic block Gauss-Seidel algorithms with exact inversion of the blocks are developed using different parallelization techniques. Experimental results for NVIDIA Fermi/Kepler GPUs and AMD multicore systems are presented.

97 MATHEMATICS AND COMPUTING↗

Dynamic implicit 3D adaptive mesh refinement for non-equilibrium radiation diffusion

The time dependent non-equilibrium radiation diffusion equations are important for solving the transport of energy through radiation in optically thick regimes and find applications in several fields including astrophysics and inertial confinement fusion. The associated initial boundary value problems that are encountered often exhibit a wide range of scales in space and time and are extremely challenging to solve. To efficiently and accurately simulate these systems we describe our research on combining techniques that will also find use more broadly for long term time integration of nonlinear multi-physics systems: implicit time integration for efficient long term time integration of stiff multi-physics systems, local control theory based step size control to minimize the required global number of time steps while controlling accuracy, dynamic 3D adaptive mesh refinement (AMR) to minimize memory and computational costs, Jacobian Free Newton–Krylov methods on AMR grids for efficient nonlinear solution, and optimal multilevel preconditioner components that provide level independent solver convergence.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

A generalized multi-dimensional mathematical model for charging and discharging processes in a supercapacitor

A generalized three dimensional computational model based on unied formulation of electrode- electrolyte-electrode system of a electric double layer supercapacitor has been developed. The model accounts for charge transport across the solid-liquid system. This formulation based on volume averaging process is a widely used concept for the multiphase ow equations ([28] [36]) and is analogous to porous media theory typically employed for electrochemical systems [22] [39] [12]. This formulation is extended to the electrochemical equations for a supercapacitor in a consistent fashion, which allows for a single-domain approach with no need for explicit interfacial boundary conditions as previously employed ([38]). In this model it is easy to introduce the spatio-temporal variations, anisotropies of physical properties and it is also conducive for introducing any upscaled parameters from lower length{scale simulations and experiments. Due to the irregular geometric congurations including porous electrode, the charge transport and subsequent performance characteristics of the super-capacitor can be easily captured in higher dimensions. A generalized model of this nature also provides insight into the applicability of 1D models ([38]) and where multidimensional eects need to be considered. In addition, simple sensitivity analysis on key input parameters is performed in order to ascertain the dependence of the charge and discharge processes on these parameters. Finally, we demonstarted how this new formulation can be applied to non-planar supercapacitors

Single Domain↗

A Validation Study of Pin Heat Transfer for UO2 Fuel Based on the IFA-432 Experiments

The IFA-432 (Integrated Fuel Assessment) experiments from the International Fuel Performance Experiments (IFPE) database were designed to study the effects of gap size, fuel density, and fuel densification on fuel centerline temperature in light-water-reactor fuel. An evaluation of nuclear fuel pin heat transfer in the FRAPCON-3.4 and Exnihilo codes for uranium dioxide (UO$_2$) fuel systems was performed, with a focus on the densification stage (2.2 \unitfrac{GWd}{MT(UO$_{2}$)}). In addition, sensitivity studies were performed to evaluate the effect of the radial power shape and approximations to the geometry to account for the thermocouple hole. The analysis demonstrated excellent agreement for rods 1, 2, 3, and 5 (varying gap thicknesses and density with traditional fuel), demonstrating the accuracy of the codes and their underlying material models for traditional fuel. For rod 6, which contained unstable fuel that densified an order of magnitude more than traditional, stable fuel, the magnitude of densification was over-predicted and the temperatures were outside of the experimental uncertainty. The radial power shape within the fuel was shown to significantly impact the predicted centerline temperatures, whereas modeling the fuel at the thermocouple location as either annular or solid was relatively negligible. This has provided an initial benchmarking of the pin heat transfer capability of Exnihilo for UO$_2$ fuel with respect to a well-validated nuclear fuel performance code.

Phillippe, Aaron M↗

A Validation Study of Pin Heat Transfer for MOX Fuel Based on the IFA-597 Experiments

Abstract The IFA-597 (Integrated Fuel Assessment) experiments from the International Fuel Performance Experiments (IFPE) database were designed to study the thermal behavior of mixed oxide (MOX) fuel and the effects of an annulus on fission gas release in light-water-reactor fuel. An evaluation of nuclear fuel pin heat transfer in the FRAPCON-3.4 and Exnihilo codes for MOX fuel systems was performed, with a focus on the first 20 time steps ( 6 GWd/MT(iHM)) for explicit comparison between the codes. In addition, sensitivity studies were performed to evaluate the effect of the radial power shape and approximations to the geometry to account for the thermocouple hole, dish, and chamfer. The analysis demonstrated relative agreement for both solid (rod 1) and annular (rod 2) fuel in the experiment, demonstrating the accuracy of the codes and their underlying material models for MOX fuel, while also revealing a small energy loss artifact in how gap conductance is currently handled in Exnihilo for chamfered fuel pellets. The within-pellet power shape was shown to significantly impact the predicted centerline temperatures. This has provided an initial benchmarking of the pin heat transfer capability of Exnihilo for MOX fuel with respect to a well-validated nuclear fuel performance code.

Phillippe, Aaron M↗

Adaptive Implicit Non-Equilibrium Radiation Diffusion

We describe methods for accurate and efficient long term time integra- tion of non-equilibrium radiation diffusion systems: implicit time integration for effi- cient long term time integration of stiff multiphysics systems, local control theory based step size control to minimize the required global number of time steps while control- ling accuracy, dynamic 3D adaptive mesh refinement (AMR) to minimize memory and computational costs, Jacobian Free Newton-Krylov methods on AMR grids for efficient nonlinear solution, and optimal multilevel preconditioner components that provide level independent solver convergence.

Philip, Bobby↗