Employing Multiple Levels of Parallelism for CFD at Large Scales on Next Generation High-Performance Computing Platforms.
Abstract not provided.
SEARCH · Engineering Papers
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
Abstract not provided.
This article presents our work toward correct and efficient automatic differentiation of OpenMP parallel worksharing loops in forward and reverse mode. Automatic differentiation is a method to obtain gradients of numerical programs, which are crucial in optimization, uncertainty quantification, and machine learning. The computational cost to compute gradients is a common bottleneck in practice. For applications that are parallelized for multicore CPUs or GPUs using OpenMP, one also wishes to compute the gradients in parallel. Here, we propose a framework to reason about the correctness of the generated derivative code, from which we justify our OpenMP extension to the differentiation model. We implement this model in the automatic differentiation tool Tapenade and present test cases that are differentiated following our extended differentiation procedure. Performance of the generated derivative programs in forward and reverse mode is better than sequential, although our reverse mode often scales worse than the input programs.
Explore the source record for details and available documents.
Inspired by the in-memory computing architectures of biological systems, neuromorphic computing using crossbar arrays of artificial synapses based on non-volatile memory (NVM) devices with variable conductances has emerged as a new paradigm to enable massively parallel and ultra-low power computing hardware for data centric applications. Although inference has been demonstrated successfully using crossbars based on a variety of NMV technologies, efficient learning and scaling to large arrays (>10 6 elements) remains a challenge due to the synaptic elements' non-ideal electrical characteristics which degrades ANN accuracy. A further challenge is that in the conductive state memristors draw large currents >μA resulting in significant voltage drops in the interconnect wires and increased probability of failure in scaled arrays. We suggest the organic polymer redox transistor (RT) is an alternate approach that could solve many of these challenges, enabling both inference and parallel outer product updates, as recently demonstrated by Fuller et al. An RT consists of redox-active channel and gate electrodes in contact with a liquid or solid electrolyte. lon insertion through the electrolyte controls the channel electronic conductivity, while electron transfer through an external circuit maintains overall charge neutrality. Unlike a rechargeable battery, in the RT the voltage built-up across the electrolyte is kept to a minimum (typically <100 mV) by using the same material for the gate and channel. Elimination of the voltage offset simplifies integration of the RT into programmable arrays by enabling the use of various selectors. RTs based on inorganic and organic materials have been recently demonstrated with conductance tuning occurring at potentials of just a few mV and hundreds to thousands of linearly and symmetrically programmable conductance states, enabling near ideal accuracy in neural network simulations. Introduced in the 1980's, redox transistors with metallic gate electrodes and organic channel materials, also known as organic electrochemical transistors (OECTs), have been explored for a variety of applications such as chem- and bio-sensing, neural interfaces, and low cost printed circuits. A typical channel material for OECTs is the conducting polymer poly(3,4-ethylenedioxythiophene) doped with poly(styrene sulfonate) (PEDOT:PSS). PEDOT is a p-type semiconducting polymer with mobile positively charged polarons that hop chain-to-chain.
We propose explicitly incorporating large-scale load siting into a stochastic nodal power system capacity expansion planning model that concurrently co-optimizes generation, transmission, and storage expansion. The potential operational flexibility of some of these large loads is also taken into account by considering them as consisting of a set of tranches with different reliability requirements, which are modeled as a constraint on expected served energy across operational scenarios. We implement our model as a two-stage stochastic mixed-integer optimization problem with cross-scenario expectation constraints. To overcome the challenge of scalability, we build upon existing work to implement this model on a high performance computing platform and exploit scenario parallelization using an augmented Progressive Hedging Algorithm. The algorithm is implemented using the bounding features of mpisppy, which have shown to provide satisfactory provable optimality gaps despite the absence of theoretical guarantees of convergence. We test our approach and assess the value of this proactive planning framework on total system cost and reliability metrics using realistic testcases geographically assigned to San Diego and South Carolina, with datacenter and direct air capture facilities as large loads.
The surge in availability of genomic data holds promise for enabling determination of genetic causes of observed individual traits, with applications to problems such as discovery of the genetic roots of phenotypes, be they molecular phenotypes such as gene expression or metabolite concentrations, or complex phenotypes such as diseases. However, the growing sizes of these datasets and the quadratic, cubic or higher scaling characteristics of the relevant algorithms pose a serious computational challenge necessitating use of leadership scale computing. In this study we describe a new approach to performing vector similarity metrics calculations, suitable for parallel systems equipped with graphics processing units (GPUs) or Intel Xeon Phi processors. Our primary focus is the Proportional Similarity metric applied to Genome Wide Association Studies (GWAS) and Phenome Wide Association Studies (PheWAS). We describe the implementation of the algorithms on accelerated processors, methods used for eliminating redundant calculations due to symmetries, and techniques for efficient mapping of the calculations to many-node parallel systems. Finally, results are presented demonstrating high per-node performance and parallel scalability with rates of more than five quadrillion (5 × 10 15 ) elementwise comparisons achieved per second on the ORNL Titan system. In a companion paper we describe corresponding techniques applied to calculations of the Custom Correlation Coefficient for comparative genomics applications.
Explore the source record for details and available documents.
Solving sparse problems is at the core of many DOE computational science applications. We focus on the challenge of developing sparse algorithms that can fully exploit the parallelism in extreme-scale computing systems, in particular systems with massive numbers of cores per node. Our approach is to express a sparse matrix factorization as a large number of bilinear constraint equations, and then solving these equations via an asynchronous iterative method. The unknowns in these equations are the matrix entries of the factorization that is desired.
Explore the source record for details and available documents.
We present an optimized Floyd-Warshall (Floyd-Warshall) algorithm that computes the All-pairs shortest path (APSP) for GPU accelerated clusters. The Floyd-Warshall algorithm due to its structural similarities to matrix-multiplication is well suited for highly parallel GPU architectures. To achieve high parallel efficiency, we address two key algorithmic challenges: reducing high communication overhead and addressing limited GPU memory. To reduce high communication costs, we redesign the parallel (a) to expose more parallelism, (b) aggressively overlap communication and computation with pipelined and asynchronous scheduling of operations, and (c) tailored MPI-collective. To cope with limited GPU memory, we employ an offload model, where the data resides on the host and is transferred to GPU on-demand. The proposed optimizations are supported with detailed performance models for tuning. Our optimized parallel Floyd-Warshall implementation is up to 5x faster than a strong baseline and achieves 8.1 PetaFLOPS/sec on 256~nodes of the Summit supercomputer at Oak Ridge National Laboratory. This performance represents 70% of the theoretical peak and 80% parallel efficiency. The offload algorithm can handle 2.5x larger graphs with a 20% increase in overall running time.
As the penetration level of power electronics-interfaced renewables such as photovoltaics (PV) and wind has surged in modern electric grids, new operational risks caused by the dynamics of those inverter-based resources (IBRs) are emerging in parallel. Lessons learned from various grid events include that the impact of IBRs on system-level grid stability will become prominent along with the increase of renewables and that the short-timescale dynamic impacts of IBRs on grid stability are not fully captured by current commercial dynamic simulation tools [1] [2]. For example, IBRs can be controlled to mitigate those destabilizing interactions, but conventional phasor-domain tools (e.g. PSS/E, PSLF) often cannot capture that; likewise, the existing electromagnetic transient (EMT) simulation tools (e.g. PSCAD, EMTP) can simulate detailed IBR controls, but for large power systems with many IBRs, slow simulation speeds severely impede the ability to study dynamic events [3] [4]. Massively paralleling simulations using high-performance computing (HPC) can help address this, especially now that cloud-based HPC capability is widely available, but today s EMT tools are not HPC-compatible, and parallelization of dynamic simulation solvers is not trivial because each region can dynamically affect the others. Thus, dynamic simulation of grids with very large numbers of IBRs potentially poses a barrier to the ongoing energy transition.
In this study, quantifying simulation uncertainties is a critical component of rigorous predictive simulation. A key component of this is forward propagation of uncertainties in simulation input data to output quantities of interest. Typical approaches involve repeated sampling of the simulation over the uncertain input data, and can require numerous samples when accurately propagating uncertainties from large numbers of sources. Often simulation processes from sample to sample are similar and much of the data generated from each sample evaluation could be reused. We explore a new method for implementing sampling methods that simultaneously propagates groups of samples together in an embedded fashion, which we call embedded ensemble propagation. We show how this approach takes advantage of properties of modern computer architectures to improve performance by enabling reuse between samples, reducing memory bandwidth requirements, improving memory access patterns, improving opportunities for fine-grained parallelization, and reducing communication costs. We describe a software technique for implementing embedded ensemble propagation based on the use of C++ templates and describe its integration with various scientific computing libraries within Trilinos. We demonstrate improved performance, portability and scalability for the approach applied to the simulation of partial differential equations on a variety of CPU, GPU, and accelerator architectures, including up to 131,072 cores on a Cray XK7 (Titan).
In this report we study the percolation critical surface of the kagome lattice in which each triangle is allowed an arbitrary connectivity. Using the method of critical polynomials, we find points along this critical surface to high precision. This kagome hypergraph contains many unsolved problems as special cases, including bond percolation on the kagome and (3, 12 2 ) lattices, and site percolation on the hexagonal, or honeycomb, lattice, as well as a single point for which there is an exact solution. We are able to compute enough points along the critical surface to find a very accurate fit, essentially a Taylor series about the exact point, that allows estimations of the critical point of any system that lies on the surface to precision rivaling Monte Carlo and traditional techniques of similar accuracy. We find also that this system sheds light on some of the surprising aspects of the method of critical polynomials, such as why it is so accurate for certain problems, like the kagome and (3, 12 2 ) lattices. The bond percolation critical points of these lattices can be found to 17 and 18 digits, respectively, because they are in close proximity, in a sense that can be made quantitative, to the exact point on the critical surface. We also discuss in detail a parallel implementation of the method which we use here for a few calculations.
We present a new computer program, SRTC++, to solve spatial problems associated with explorations of Saturn’s moon Titan. The program implements a three-dimensional structure well-suited to addressing shortcomings arising from plane-parallel radiative transfer approaches. SRTC++'s design uses parallel processing in an object-oriented, compiled computer language (C++) leading to a flexible and fast architecture. We validate SRTC++ using analytical results, semianalytical radiative transfer expressions, and an existing Titan plane-parallel model. SRTC++ complements existing approaches, addressing spatial problems like near-limb and near-terminator geometries, non-Lambertian surface phase functions (including specular reflections), and surface albedo nonuniformity.
Crystallite orientation identification is invaluable, but is often limited to small area identification or requires a large area sample. Nondestructive optical methods such as polarized Raman spectroscopy, in contrast, can be used to completely map a variety of sample sizes, but their potential is not yet fully realized. Here, we report a systematic study of polarized Raman scattering of high-quality, hydrothermally grown, single crystals of urania and thoria. The peak intensity variations for as-grown major crystal planes, post-growth polished crystal planes, and a post-growth polished non-crystallographic plane are directly linked to crystallographic orientation and crystal rotation, and agree with computed models. In particular, the parallel polarized peak intensity results are directly correlated with metal–oxygen–metal chains in the fluorite structure and can be used to determine both orientation and rotational alignment of a given crystal face if sufficiently small rotational steps are applied. These results are structure based, being applicable to the larger fluorite phase space, which is useful for optical, semiconductor, nuclear, and solid oxide fuel cell industries. Further, these results suggest that Raman spectroscopy can identify non-crystallographic orientations that are not discernable by traditional means.
Considering the increase of disruptive variable renewable energy penetration into the power grid, this article focuses on the investigation of a multiobjective and dynamic real-time optimization framework to address the cycling of large-scale power plants under renewable penetration. In this framework, a parallelized particle swarm optimization step is first performed to generate feasible initial points. Then, a multiobjective and dynamic real-time optimization formulation generates optimal trajectories. Further, the benefit of predictive capability is investigated for the dynamic component, which introduces the novel nonlinear multiobjective and dynamic real-time predictive optimization approach. Two multiobjective formulations to obtain Pareto front optimal in real time are explored: the modified Tchebycheff-based weighted metric and ϵ-constraint methods. Economic and environmental objectives are considered in this study. A novel topical discussion on the intersection of dynamic real-time optimization with model predictive control is also presented. The developed framework is successfully applied to a baseload coal-fired power plant with postcombustion CO 2 capture. Results indicate that the approach can be deployed for a large-scale system if automatic differentiation, model reduction, and parallelization are adopted to improve computational tractability, with computational improvement up to 120-folds after performing these steps. Finally, market and carbon policies showed an impact on the optimal compromise between the objectives with an additional 63 ton of CO 2 captured under favorable market conditions.
The semi-implicit leapfrog time-discretization is a workhorse algorithm for initial-value MHD codes to bridge between vastly separated time scales. Inclusion of atomic interactions with neutrals breaks the functional structure of the MHD equations that exploited by the leapfrog. In this work, we address how to best integrate atomic physics into the semi-implicit leapfrog. Following the Crank-Nicolson method, one approach is to time-center the atomic interactions in the linear solver and use a Newton method to include the nonlinear contributions. Alternatively, another family of methods are based on operator-splitting the terms associated with the atomic interactions using a Strang-splitting technique. These methods naturally break equations into constituent ODE and PDE parts and preserve the structure exploited by the semi-implicit leapfrog. We study the accuracy and efficiency of these methods through a battery of 0D and 1D cases and show that a second-order-in-time Douglas-Rachford inspired coupling between the ODE and PDE advances is effective in reducing the time-discretization error to be comparable to that of Crank-Nicolson with Newton iteration of the nonlinear terms. Splitting ODE and PDE parts results in independent matrix solves for each field which reduces the computational cost considerably and provides parallelization over species relative to Crank-Nicolson.
In this paper we develop a numerical method based on the quantum circuit theory to approach the coherent electronic transport in a network of quantum dots connected with arbitrary topology. The algorithm was employed in a circuit formed by quantum dots connected each other in a shape of a linear chain (associations in series), and of a ring (associations in series, and in parallel). For both systems we compute two current observables: conductance and shot noise power. We find an excellent agreement between our numerical results and the ones found in the literature. Moreover, we analyze the algorithm efficiency for a chain of quantum dots, where the mean processing time exhibits a linear dependence with the number of quantum dots in the array.