Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “CRAY COMPUTERS”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 397 records · Page 22

Application of Cray XMT for Power Grid Contingency Selection

Contingency analysis is a key function to assess the impact of various combinations of power system component failures. It involves combinatorial numbers of contingencies which exceed the capability of even very large supercomputing platforms. Therefore, it is critical to select contingency cases within the constraint of computing power. This paper presents a contingency selection method employing graph theory (edge betweenness centrality) to power grid weighted graphs to remove low-impact components. The parallel implementation of the method was successfully carried out on the Cray XMT machine. The implementation takes advantage of the superior capabilities of the Cray XMT for graph-based problems and its programming features. This paper presents the performance scalability of Cray XMT and comparison with a cache-based, shared-memory machine.

24 POWER TRANSMISSION AND DISTRIBUTION↗

Performance Tuning of Fock Matrix and Two-Electron Integral Calculations for NWChem on Leading HPC Platforms

Attaining performance in the evaluation of two-electron repulsion integrals and constructing the Fock matrix is of considerable importance to the computational chemistry community. Due to its numerical complexity improving the performance behavior across a variety of leading supercomputing platforms is an increasing challenge due to the significant diversity in high-performance computing architectures. In this paper, we present our successful tuning methodology for these important numerical methods on the Cray XE6, the Cray XC30, the IBM BG/Q, as well as the Intel Xeon Phi. Our optimization schemes leverage key architectural features including vectorization and simultaneous multithreading, and results in speedups of up to 2.5x compared with the original implementation.

Shan, Hongzhan↗

A high-order discontinuous Galerkin method for wave propagation through coupled elastic-acoustic media

We introduce a high-order discontinuous Galerkin (dG) scheme for the numerical solution of three-dimensional (3D) wave propagation problems in coupled elastic-acoustic media. A velocity-strain formulation is used, which allows for the solution of the acoustic and elastic wave equations within the same unified framework. Careful attention is directed at the derivation of a numerical flux that preserves high-order accuracy in the presence of material discontinuities, including elastic-acoustic interfaces. Explicit expressions for the 3D upwind numerical flux, derived as an exact solution for the relevant Riemann problem, are provided. The method supports h-non-conforming meshes, which are particularly effective at allowing local adaptation of the mesh size to resolve strong contrasts in the local wavelength, as well as dynamic adaptivity to track solution features. The use of high-order elements controls numerical dispersion, enabling propagation over many wave periods. We prove consistency and stability of the proposed dG scheme. To study the numerical accuracy and convergence of the proposed method, we compare against analytical solutions for wave propagation problems with interfaces, including Rayleigh, Lamb, Scholte, and Stoneley waves as well as plane waves impinging on an elastic-acoustic interface. Spectral rates of convergence are demonstrated for these problems, which include a non-conforming mesh case. Finally, we present scalability results for a parallel implementation of the proposed high-order dG scheme for large-scale seismic wave propagation in a simplified earth model, demonstrating high parallel efficiency for strong scaling to the full size of the Jaguar Cray XT5 supercomputer.

97 MATHEMATICS AND COMPUTING↗

Energy dataset of Frontier supercomputer for waste heat recovery

The Hewlett Packard Enterprise–Cray EX Frontier is the world’s first and fastest exascale supercomputer, hosted at the Oak Ridge Leadership Computing Facility in Tennessee, United States. Frontier is a significant electricity consumer, drawing 8–30 MW; this massive energy demand produces significant waste heat, requiring extensive cooling measures. Although harnessing this waste heat for campus heating is a sustainability goal at Oak Ridge National Laboratory (ORNL), the 30 °C–38 °C waste heat temperature poses compatibility issues with standard HVAC systems. Heat pump systems, prevalent in residential settings and some industries, can efficiently upgrade low-quality heat to usable energy for buildings. Thus, heat pump technology powered by renewable electricity offers an efficient, cost-effective solution for substantial waste heat recovery. However, a major challenge is the absence of benchmark data on high-performance computing (HPC) heat generation and waste heat profiles. This paper reports power demand and waste heat measurements from an ORNL HPC data centre, aiming to guide future research on optimizing waste heat recovery in large-scale data centres, especially those of HPC calibre.

97 MATHEMATICS AND COMPUTING↗

ServiceNow/CrayPort Integration v2.1.5

The National Energy Research Scientific Computing Center (NERSC) is the primary scientific computing facility for the Office of Science in the U.S. Department of Energy. NERSC houses top-ranked Cray supercomputers, and our staff works very closely with on-site Cray engineers, submitting 75 Cray cases per month on average. Until recently, submitting a Cray case was only possible by phone or email, and all subsequent updates were delivered manually. This process led to delays, errors during manual data entry, and greatly increased incident processing and resolution time. In 2018, NERSC deployed an API-based bidirectional integration that allowed to submit and update Cray cases directly from our Incident Management platform, thus streamlining our 24x7 operations and enhancing communication between our engineering teams. This solution is applicable to any Incident Management platform.

Gens, Daniel↗

A parallel algorithm for generation and assembly of finite element stiffness and mass matrices

A new algorithm is proposed for parallel generation and assembly of the finite element stiffness and mass matrices. The proposed assembly algorithm is based on a node-by-node approach rather than the more conventional element-by-element approach. The new algorithm's generality and computation speed-up when using multiple processors are demonstrated for several practical applications on multi-processor Cray Y-MP and Cray 2 supercomputers.

Storaasli, O. O.↗

A parallel vectorized implementation of triple excitations in CCSD(T) - Application to the binding energies of the AlH3, AlH2F, AlHF2 and AlF3 dimers

An efficient method for various noniterative estimates of connected triple excitations in coupled-cluster theory is outlined and related to a similar expression occurring in Moller-Plesset perturbation theory. The method is highly vectorized and capable of utilizing multiple processors on a shared-memory machine, leading to computational rates in excess of one billion floating-point operations per second on four processors of a CRAY Y-MP. Using the new procedure, the binding energies of the D(2h) diborane-type dimers of AlH3, AlH2F, AlHF2, and AlF3 have been determined to be 32, 40, 20, and 47 kcal/mol, respectively. For Al2F6, the correlation procedure includes 232 molecular orbitals and over 1.5 x 10 to the 6th single and double coupled-cluster amplitudes, effectively accounting for over 2 x 10 to the 9th connected triple excitations.

Rendell, Alistair P.↗

Using High Performance Computing to Understand Roles of Labile and Nonlabile U(VI) on Hanford 300 Area Plume Longevity

Evolution of a hexavalent uranium [U(VI)] plume at the Hanford 300 Area bordering the Columbia River is investigated to evaluate the roles of labile and nonlabile forms of U(VI) on the longevity of the plume. A high fidelity, three-dimensional, field-scale, reactive flow and transport model is used to represent the system. Richards equation coupled to multicomponent reactive transport equations are solved for times up to 100 years taking into account rapid fluctuations in the Columbia River stage resulting in pulse releases of U(VI) into the river. The peta-scale computer code PFLOTRAN developed under a DOE SciDAC-2 project is employed in the simulations and executed on ORNL's Cray XT5 supercomputer Jaguar. Labile U(VI) is represented in the model through surface complexation reactions and its nonlabile form through dissolution of metatorbernite used as a surrogate mineral. Initial conditions are constructed corresponding to the U(VI) plume already in place to avoid uncertainties associated with the lack of historical data for the waste stream. The cumulative U(VI) flux into the river is compared for cases of equilibrium and multirate sorption models and for no sorption. The sensitivity of the U(VI) flux into the river on the initial plume configuration is investigated. The presence of nonlabile U(VI) was found to be essential in explaining the longevity of the U(VI) plume and the prolonged high U(VI) concentrations at the site exceeding the EPA MCL for uranium.

99 GENERAL AND MISCELLANEOUS↗

Embedded ensemble propagation for improving performance, portability, and scalability of uncertainty quantification on emerging computational architectures

In this study, quantifying simulation uncertainties is a critical component of rigorous predictive simulation. A key component of this is forward propagation of uncertainties in simulation input data to output quantities of interest. Typical approaches involve repeated sampling of the simulation over the uncertain input data, and can require numerous samples when accurately propagating uncertainties from large numbers of sources. Often simulation processes from sample to sample are similar and much of the data generated from each sample evaluation could be reused. We explore a new method for implementing sampling methods that simultaneously propagates groups of samples together in an embedded fashion, which we call embedded ensemble propagation. We show how this approach takes advantage of properties of modern computer architectures to improve performance by enabling reuse between samples, reducing memory bandwidth requirements, improving memory access patterns, improving opportunities for fine-grained parallelization, and reducing communication costs. We describe a software technique for implementing embedded ensemble propagation based on the use of C++ templates and describe its integration with various scientific computing libraries within Trilinos. We demonstrate improved performance, portability and scalability for the approach applied to the simulation of partial differential equations on a variety of CPU, GPU, and accelerator architectures, including up to 131,072 cores on a Cray XK7 (Titan).

97 MATHEMATICS AND COMPUTING↗

Optimal Full Information Synthesis for Flexible Structures Implemented on Cray Supercomputers

This paper considers an algorithm for synthesis of optimal controllers for full information feedback. The synthesis procedure reduces to a single linear matrix inequality which may be solved via established convex optimization algorithms. The computational cost of the optimization is investigated. It is demonstrated the problem dimension and corresponding matrices can become large for practical engineering problems. This algorithm represents a process that is impractical for standard workstations for large order systems. A flexible structure is presented as a design example. Control synthesis requires several days on a workstation but may be solved in a reasonable amount of time using a Cray supercomputer.

Lind, Rick↗

Optimizing Nuclear Physics Codes on the XT5

Scientists studying the structure and behavior of the atomic nucleus require immense high-performance computing resources to gain scientific insights. Several nuclear physics codes are capable of scaling to more than 100,000 cores on Oak Ridge National Laboratory's petaflop Cray XT5 system, Jaguar. In this paper, we present our work on optimizing codes in the nuclear physics domain.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗

Avoiding Communication in Primal and Dual Block Coordinate Descent Methods

Primal and dual block coordinate descent methods are iterative methods for solving regularized and unregularized optimization problems. Distributed-memory parallel implementations of these methods have become popular in analyzing large machine learning datasets. However, existing implementations communicate at every iteration, which, on modern data center and supercomputing architectures, often dominates the cost of floating-point computation. Recent results on communication-avoiding Krylov subspace methods suggest that large speedups are possible by re-organizing iterative algorithms to avoid communication. We show how applying similar algorithmic transformations can lead to primal and dual block coordinate descent methods that only communicate every $s$ iterations---where $s$ is a tuning parameter---instead of every iteration for the regularized least-squares problem. We show that the communication-avoiding variants reduce the number of synchronizations by a factor of $s$ on distributed-memory parallel machines without altering the convergence rate and attain strong scaling speedups of up to $6.1\times$ over the “standard algorithm" on a Cray XC30 supercomputer.

97 MATHEMATICS AND COMPUTING↗

Highly Parallel Solver for Multi-scale Parquet Quantum Modeling of Strongly Correlated Materials

The parquet formalism to calculate the two-particle Green s functions of large systems requires the solution of a large, sparse, complex system of quadratic equations. If Nf Matsubara frequencies are used for a system of size Nc , and Newton s method is used to solve the nonlinear system, the Jacobian system has O(8Nt^3 ) variables and O(40Nt^4 ) complex entries where Nt = Nc Nf . For Nt = 1024, the nonlinear system has over 8.5 billion degrees of freedom and the sparse Jacobian will require over 351 TBytes of memory. The Jacobian is very expensive to store but the matrix-vector products can be computed directly. We are developing a highly scalable parallel solver that uses both OpenMP and MPI to exploit the multicore nodes. We present initial scalability results on the Cray XT5 that suggests the code can be scaled to solve larger problems with Nt 1024.

97 MATHEMATICS AND COMPUTING↗

Scaling of data communications for an advanced supercomputer network

The goal of NASA's Numerical Aerodynamic Simulation (NAS) Program is to provide a powerful computational environment for advanced research and development in aeronautics and related disciplines. The present NAS system consists of a Cray 2 supercomputer connected by a data network to a large mass storage system, to sophisticated local graphics workstations and by remote communication to researchers throughout the United States. The program plan is to continue acquiring the most powerful supercomputers as they become available. The implications of a projected 20-fold increase in processing power on the data communications requirements are described.

Levin, E.↗

Application and implementation of transient algorithms in computer programs

A brief introduction is given to the nonlinear finite element programs developed at Lawrence Livermore National Laboratory. The four programs are DYNA3D and DYNA2D, which are explicit hydrocodes, and NIKE3D and NIKE2D, which are implicit programs. The main emphasis is on DYNA3D with asides about the other programs. During the past year several new features were added to DYNA3D, and major improvements were made in the computational efficiency of the shell and beam elements. Most of these new features and improvements will eventually make their way into the other programs. The emphasis in the computational mechanics effort was always, and continues to be, efficiency. To get the most out of the supercomputers, all Crays, the programs were vectorized where possible. Several of the more interesting capabilities of DYNA3D will be described and the impact on efficiency will be discussed. Some of the recent work on NIKE3D and NIKE2D will also be presented. In the belief that a single example is worth a thousand equations, the theory is skipped entirely and the examples presented.

Benson, David J.↗

Investigation of forced unsteady separated flows using velocity-vorticity form of Navier-Stokes equations

The phenomenon of forced unsteady separation and eruption of boundary-layer vorticity is a highly-complex, high-Reynolds number flow phenomenon, which abruptly leads to the formation of a dynamic stall vortex as demonstrated earlier by the authors for a NACA 0015 airfoil undergoing constant rate pitch-up motion. This, as well as the results of other researchers, have convincingly demonstrated a complex vortical structure within the state of unsteady separation prior to the evolution of dynamic stall. This phenomenon of vortex eruption, although observed in studying dynamic stall phenomena, is also associated with transition from laminar to turbulence flow and its generic nature has been stressed by many researchers including the present investigators. An unsteady Navier-Stokes (NS) analysis is developed for arbitrarily maneuvering bodies using velocity-vorticity variables; this formulation is nearly form-invariant under a generalized non-inertial coordinate transformation. A fully-implicit uniformly second-order accurate method is used, with the nonlinear convective terms approximated using a biased third-order upwind differencing scheme to be able to simulate higher-Re flows. No explicit artificial dissipation is added. The numerical method is fully vectorized and currently achieves a computational index of 7 micro-seconds per time step per mesh point, using a single processor on a CRAY Y-MP. The simulation results show that the energetic free shear from the leading edge is responsible for the wall viscous layer to abruptly erupt near the center of the counterclockwise rotating eddy in the unsteady boundary layer. Primary, secondary, tertiary and quaternary vortices have been observed before the dynamic stall vortex evolves and gathers its maximum strength. This study will discuss the simulation results of Reynolds number up to Re = 45,000 and will also discuss the efforts of initial acceleration in a specific maneuver, on the evolution of the stall vortex.

Ghia, K. N.↗

GVT Algorithms and Discrete Event Dynamics on 128K+ Processor Cores

Parallel discrete event simulation (PDES) represents a class of codes that are challenging to scale to large number of processors due to tight global timestamp-ordering and fine-grained event execution. One of the critical factors in scaling PDES is the efficiency of the underlying global virtual time (GVT) algorithm needed for correctness of parallel execution and speed of progress. Although many GVT algorithms have been proposed previously, few have been proposed for scalable asynchronous execution and none customized to exploit one-sided communication. Moreover, the detailed performance effects of actual GVT algorithm implementations on large platforms are unknown. Here, three major GVT algorithms intended for scalable execution on high-performance systems are studied: (1) a synchronous GVT algorithm that affords ease of implementation, (2) an asynchronous GVT algorithm that is more complex to implement but can relieve blocking latencies, and (3) a variant of the asynchronous GVT algorithm, proposed and studied for the first time here, to exploit one-sided communication in extant supercomputing platforms. Performance results are presented of implementations of these algorithms on over 64,000 cores of a Cray XT5 system, exercised on a range of parameters: optimistic and conservative synchronization, fine- to medium-grained event computation, synthetic and non-synthetic applications, and different lookahead values. Performance of tens of billions of events executed per second are registered, exceeding the speeds of any known PDES engine, and showing asynchronous GVT algorithms to outperform state-of-the-art synchronous GVT algorithms. Detailed PDES-specific runtime metrics are presented to further the understanding of tightly-coupled discrete event execution dynamics on massively parallel platforms.

99 GENERAL AND MISCELLANEOUS↗

A Parallel Prefix Algorithm for Almost Toeplitz Tridiagonal Systems

A compact scheme is a discretization scheme that is advantageous in obtaining highly accurate solutions. However, the resulting systems from compact schemes are tridiagonal systems that are difficult to solve efficiently on parallel computers. Considering the almost symmetric Toeplitz structure, a parallel algorithm, simple parallel prefix (SPP), is proposed. The SPP algorithm requires less memory than the conventional LU decomposition and is efficient on parallel machines. It consists of a prefix communication pattern and AXPY operations. Both the computation and the communication can be truncated without degrading the accuracy when the system is diagonally dominant. A formal accuracy study has been conducted to provide a simple truncation formula. Experimental results have been measured on a MasPar MP-1 SIMD machine and on a Cray 2 vector machine. Experimental results show that the simple parallel prefix algorithm is a good algorithm for symmetric, almost symmetric Toeplitz tridiagonal systems and for the compact scheme on high-performance computers.

Sun, Xian-He↗