Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “parallel, scaling”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 505 records · Page 28

Negative viscosity from negative compressibility and axial flow shear stiffness in a straight magnetic field

Here, negative compressibility ITG turbulence in a linear plasma device (CSDX) can induce a negative viscosity increment. However, even with this negative increment, we show that the total axial viscosity remains positive definite, i.e. no intrinsic axial flow can be generated by pure ITG turbulence in a straight magnetic field. This differs from the case of electron drift wave (EDW) turbulence, where the total viscosity can turn negative, at least transiently. When the flow gradient is steepened by any drive mechanism, so that the parallel shear flow instability (PSFI) exceeds the ITG drive, the flow profile saturates at a level close to the value above which PSFI becomes dominant. This saturated flow gradient exceeds the PSFI linear threshold, and grows with $\nabla T_{i0}$ as $|\nabla V_\parallel| / |k_\parallel c_s| \sim|\nabla T_{i0}|^{2/3} / (k_\parallel T_{i0})^{2/3}$. This scaling trend characterizes the effective stiffness of the parallel flow gradient.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗

Coupling LAMMPS and the vl3 Framework for Co-Visualization of Atomistic Simulations

In this work we perform in-situ visualization of molecular dynamics simulations, which can help scientists to visualize simulation output on-the-fly, without incurring storage overheads. We present a case study to couple LAMMPS, the large-scale molecular dynamics simulation code with vl3, our parallel framework for large-scale visualization and analysis. Our motivation is to identify effective approaches for covisualization and exploration of large-scale atomistic simulations at interactive frame rates.We propose a system of coupled libraries and describe its architecture, with an implementation that runs on GPU-based clusters. We present the results of strong and weak scalability experiments, as well as future research avenues based on our results.

co-visualization↗

A Scalable Parallel Wideband MLFMA for Efficient Electromagnetic Simulations on Large Scale Clusters

The development of the multilevel fast multipole algorithm (MLFMA) and its multiscale variants have enabled the use of integral equation (IE) based solvers to compute scattering from complicated structures. Development of scalable parallel algorithms, to extend the reach of these solvers, has been a topic of intense research for about a decade. In this paper, we present a new algorithm for parallel implementation of IE solver that is augmented with a wideband MLFMA and scalable on large number of processors. The wideband MLFMA employed here, to handle multiscale problems, is a hybrid combination of the accelerated Cartesian expansion (ACE) and the classical MLFMA. The salient feature of the presented parallel algorithm is that it is implicitly load balanced and exhibits higher performance. This is achieved by developing a strategy to partition the MLFMA tree, and hence the associated computations, in a self-similar fashion among the parallel processors. As detailed in the paper, the algorithm employs both spatial and direction partitioning approaches in a flexible manner to ensure scalable performance. Plethora of results are presented here to exhibit the scalability of this algorithm on 512 and more processors.

99 GENERAL AND MISCELLANEOUS↗

Scalable Incremental Checkpointing using GPU-Accelerated De-Duplication

Writing large amounts of data concurrently to stable storage is a typical I/O pattern of many HPC workflows. This pattern introduces high I/O overheads and results in increased storage space utilization especially for workflows that need to capture the evolution of data structures with high frequency as checkpoints. In this context, many applications, such as graph pattern matching, perform sparse updates to large data structures between checkpoints. For these applications, incremental checkpointing techniques that save only the differences from one checkpoint to another can dramatically reduce the checkpoint sizes, I/O bottlenecks, and storage space utilization. However, such techniques are not without challenges: it is non-trivial to transparently determine what data has changed since a previous checkpoint and assemble the differences in a compact fashion that does not result in excessive metadata. State-of-art data reduction techniques (e.g., compression and de-duplication) have significant limitations when applied to modern HPC applications that leverage GPUs: slow at detecting the differences, generate a large amount of metadata to keep track of the differences, and ignore crucial spatiotemporal checkpoint data redundancy. This paper addresses these challenges by proposing a Merkle tree-based incremental checkpointing method to exploit GPUs' high memory bandwidth and massive parallelism. Experimental results at scale show a significant reduction of the I/O overhead and space utilization of checkpointing compared with state-of-the-art incremental checkpointing and compression techniques.

Tan, Nigel↗

Large-scale anisotropy in stably stratified rotating flows

We present results from direct numerical simulations of the Boussinesq equations in the presence of rotation and/or stratification, both in the vertical direction. The runs are forced isotropically and randomly at small scales and have spatial resolutions of up to $1024^3$ grid points and Reynolds numbers of $\approx 1000$. We first show that solutions with negative energy flux and inverse cascades develop in rotating turbulence, whether or not stratification is present. However, the purely stratified case is characterized instead by an early-time, highly anisotropic transfer to large scales with almost zero net isotropic energy flux. This is consistent with previous studies that observed the development of vertically sheared horizontal winds, although only at substantially later times. However, and unlike previous works, when sufficient scale separation is allowed between the forcing scale and the domain size, the total energy displays a perpendicular (horizontal) spectrum with power law behavior compatible with $\sim k_\perp^{-5/3}$, including in the absence of rotation. In this latter purely stratified case, such a spectrum is the result of a direct cascade of the energy contained in the large-scale horizontal wind, as is evidenced by a strong positive flux of energy in the parallel direction at all scales including the largest resolved scales.

17 WIND ENERGY↗

Linear complexity

We present factorization and solution phases for a new linear complexity direct solver designed for concurrent batch operations on fine-grained parallel architectures, for matrices amenable to hierarchical representation. We focus on the strong-admissibility-based $\mathscr{H}^{2}$ format, where strong recursive skeletonization factorization compresses remote interactions. We build upon previous implementations of $\mathscr{H}^{2}$ matrix construction for efficient factorization and solution algorithm design, which are illustrated graphically in stepwise detail. The algorithms are ‘blackbox’ in the sense that the only inputs are the matrix and right-hand side, without analytical or geometrical information about the origin of the system. We demonstrate linear complexity scaling in both time and memory on four representative families of dense matrices up to one million in size. Parallel scaling up to 16 threads is enabled by a multi-level matrix graph coloring and avoidance of dynamic memory allocations thanks to prefix-sum memory management. An experimental backward error analysis is included. We break down the timings of different phases, identify phases that are memory-bandwidth limited, and discuss alternatives for phases that may be sensitive to the trend to employ lower precisions for performance.

Boukaram, Wajih↗

Linear vs. nonlinear acceleration in plasma turbulence. II. Hall–finite-Larmor-radius magnetohydrodynamics

The local k-space ratio of linear and nonlinear accelerations associated with a variety of initial conditions undergoing steady relaxation is investigated for the Hall–finite-Larmor-radius magnetohydrodynamics (MHD) system in the presence of a mean magnetic field. Building on a related study (Paper I) where it was shown that discrepancies exist between describing the global and local characterizations of the pure MHD system with mean magnetic field, we find regions of the Fourier space that are consistently dominated by linear acceleration and other regions that are consistently dominated by nonlinear acceleration, independent of the overall system's description as linear, weakly nonlinear, or turbulent. In general, dynamics within a certain angular range of the mean magnetic field direction are predominantly linear, while dynamics adjacent the Hall scales along the field-parallel direction and dynamics adjacent the finite Larmor radius scales in the field-perpendicular direction can become strongly nonlinear. The nonlinear influences are particularly significant as the plasma beta increases from unity to higher values.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗

Spectral quadrature for the first principles study of crystal defects: Application to magnesium

In this work, we present an accurate and efficient finite-difference formulation and parallel implementation of Kohn-Sham Density (Operator) Functional Theory (DFT) for non periodic systems embedded in a bulk environment. Specifically, employing non-local pseudopotentials, local reformulation of electrostatics, and truncation of the spatial Kohn-Sham Hamiltonian, and the Linear Scaling Spectral Quadrature method to solve for the pointwise electronic fields in real-space and the non-local component of the atomic force, we develop a parallel finite difference framework suitable for distributed memory computing architectures to simulate non-periodic systems embedded in a bulk environment. Choosing examples from magnesium-aluminum alloys, we first demonstrate the convergence of energies and forces with respect to spectral quadrature polynomial order, and the width of the spatially truncated Hamiltonian. Next, we demonstrate the parallel scaling of our framework, and show that the computation time and memory scale linearly with respect to the number of atoms. Next, we use the developed framework to simulate isolated point defects and their interactions in magnesium-aluminum alloys. Our findings conclude that the binding energies of divacancies, Al solute-vacancy and two Al solute atoms are anisotropic and are dependent on cell size. Furthermore, the binding is favorable in all three cases.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

Advanced Distillation Final Report

The Advanced Distillation project was concluded on December 31, 2009. This U.S. Department of Energy (DOE) funded project was completed successfully and within budget during a timeline approved by DOE project managers, which included a one year extension to the initial ending date. The subject technology, Microchannel Process Technology (MPT) distillation, was expected to provide both capital and operating cost savings compared to conventional distillation technology. With efforts from Velocys and its project partners, MPT distillation was successfully demonstrated at a laboratory scale and its energy savings potential was calculated. While many objectives established at the beginning of the project were met, the project was only partially successful. At the conclusion, it appears that MPT distillation is not a good fit for the targeted separation of ethane and ethylene in large-scale ethylene production facilities, as greater advantages were seen for smaller scale distillations. Early in the project, work involved flowsheet analyses to discern the economic viability of ethane-ethylene MPT distillation and develop strategies for maximizing its impact on the economics of the process. This study confirmed that through modification to standard operating processes, MPT can enable net energy savings in excess of 20%. This advantage was used by ABB Lumus to determine the potential impact of MPT distillation on the ethane-ethylene market. The study indicated that a substantial market exists if the energy saving could be realized and if installed capital cost of MPT distillation was on par or less than conventional technology. Unfortunately, it was determined that the large number of MPT distillation units needed to perform ethane-ethylene separation for world-scale ethylene facilities, makes the targeted separation a poor fit for the technology in this application at the current state of manufacturing costs. Over the course of the project, distillation experiments were performed with the targeted mixture, ethane-ethylene, as well as with analogous low relative volatility systems: cyclohexane-hexane and cyclopentane-pentane. Devices and test stands were specifically designed for these efforts. Development progressed from experiments and models considering sections of a full scale device to the design, fabrication, and operation of a single-channel distillation unit with integrated heat transfer. Throughout the project, analytical and numerical models and Computational Fluid Dynamics (CFD) simulations were validated with experiments in the process of developing this platform technology. Experimental trials demonstrated steady and controllable distillation for a variety of process conditions. Values of Height-to-an-Equivalent Theoretical Plate (HETP) ranging from less than 0.5 inch to a few inches were experimentally proven, demonstrating a ten-fold performance enhancement relative to conventional distillation. This improvement, while substantial, is not sufficient for MPT distillation to displace very large scale distillation trains. Fortunately, parallel efforts in the area of business development have yielded other applications for MPT distillation, including smaller scale separations that benefit from the flowsheet flexibility offered by the technology. Talks with multiple potential partners are underway. Their outcome will also help determine the path ahead for MPT distillation.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗

NWChem: scalable parallel computational chemistry

NWChem is a general purpose computational chemistry code specifically designed to run on distributed memory parallel computers. The core functionality of the code focuses on molecular dynamics, Hartree-Fock and density functional theory methods for both plane-wave basis sets as well as Gaussian basis sets, tensor contraction engine based coupled cluster capabilities and combined quantum mechanics/molecular mechanics descriptions. It was realized from the beginning that scalable implementations of these methods required a programming paradigm inherently different from what message passing approaches could offer. In response a global address space library, the Global Array Toolkit, was developed. The programming model it offers is based on using predominantly one-sided communication. This model underpins most of the functionality in NWChem and the power of it is exemplified by the fact that the code scales to tens of thousands of processors. In this paper the core capabilities of NWChem are described as well as their implementation to achieve an efficient computational chemistry code with high parallel scalability. NWChem is a modern, open source, computational chemistry code1 specifically designed for large scale parallel applications2. To meet the challenges of developing efficient, scalable and portable programs of this nature a particular code design was adopted. This code design involved two main features. First of all, the code is build up in a modular fashion so that a large variety of functionality can be integrated easily. Secondly, to facilitate writing complex parallel algorithms the Global Array toolkit was developed. This toolkit allows one to write parallel applications in a shared memory like approach, but offers additional mechanisms to exploit data locality to lower communication overheads. This framework has proven to be very successful in computational chemistry but is applicable to any engineering domain. Within the context created by the features above NWChem has grown into a general purpose computational chemistry code that supports a wide variety of energy expressions and capabilities to calculate properties based there upon. The main energy expressions are classical mechanics force fields, Hartree-Fock and DFT both for finite systems and condensed phase systems, coupled cluster, as well as QM/MM. For most energy expressions single point calculations, geometry optimizations, excited states, and other properties are available. Below we briefly discuss each of the main energy expressions and the critical points involved in scalable implementations thereof.

99 GENERAL AND MISCELLANEOUS↗

A derivation and scalable implementation of the synchronous parallel kinetic Monte Carlo method for simulating long-time dynamics

Kinetic Monte Carlo (KMC) simulations are used to study long-time dynamics of a wide variety of systems. Unfortunately, the conventional KMC algorithm is not scalable to larger systems, since its time scale is inversely proportional to the simulated system size. A promising approach to resolving this issue is the synchronous parallel KMC (SPKMC) algorithm, which makes the time scale size-independent. This paper introduces a formal derivation of the SPKMC algorithm based on local transition-state and time-dependent Hartree approximations, as well as its scalable parallel implementation based on a dual linked-list cell method. The resulting algorithm has achieved a weak-scaling parallel efficiency of 0.935 on 1024 Intel Xeon processors for simulating biological electron transfer dynamics in a 4.2 billion-heme system, as well as decent strong-scaling parallel efficiency. The parallel code has been used to simulate a lattice of cytochrome complexes on a bacterial-membrane nanowire, and it is broadly applicable to other problems such as computational synthesis of new materials.

97 MATHEMATICS AND COMPUTING↗

Steps toward fault-tolerant quantum chemistry.

Developing quantum chemistry programs on the coming generation of exascale computers will be a difficult task. The programs will need to be fault-tolerant and minimize the use of global operations. This work explores the use a task-based model that uses a data-centric approach to allocate work to different processes as it applies to quantum chemistry. After introducing the key problems that appear when trying to parallelize a complicated quantum chemistry method such as coupled-cluster theory, we discuss the implications of that model as it pertains to the computational kernel of a coupled-cluster program - matrix multiplication. Also, we discuss the extensions that would required to build a full coupled-cluster program using the task-based model. Current programming models for high-performance computing are fault-intolerant and use global operations. Those properties are unsustainable as computers scale to millions of CPUs; instead one must recognize that these systems will be hierarchical in structure, prone to constant faults, and global operations will be infeasible. The FAST-OS HARE project is introducing a scale-free computing model to address these issues. This model is hierarchical and fault-tolerant by design, allows for the clean overlap of computation and communication, reducing the network load, does not require checkpointing, and avoids the complexity of many HPC runtimes. Development of an algorithm within this model requires a change in focus from imperative programming to a data-centric approach. Quantum chemistry (QC) algorithms, in particular electronic structure methods, are an ideal test bed for this computing model. These methods describe the distribution of electrons in a molecule, which determine the properties of the molecule. The computational cost of these methods is high, scaling quartically or higher in the size of the molecule, which is why QC applications are major users of HPC resources. The complexity of these algorithms means that MPI alone is insufficient to achieve parallel scaling; QC developers have been forced to use alternative approaches to achieve scalability and would be receptive to radical shifts in the programming paradigm. Initial work in adapting the simplest QC method, Hartree-Fock, to this the new programming model indicates that the approach is beneficial for QC applications. However, the advantages to being able to scale to exascale computers are greatest for the computationally most expensive algorithms; within QC these are the high-accuracy coupled-cluster (CC) methods. Parallel coupledcluster programs are available, however they are based on the conventional MPI paradigm. Much of the effort is spent handling the complicated data dependencies between the various processors, especially as the size of the problem becomes large. The current paradigm will not survive the move to exascale computers. Here we discuss the initial steps toward designing and implementing a CC method within this model. First, we introduce the general concepts behind a CC method, focusing on the aspects that make these methods difficult to parallelize with conventional techniques. Then we outline what is the computational core of the CC method - a matrix multiply - within the task-based approach that the FAST-OS project is designed to take advantage of. Finally we outline the general setup to implement the simplest CC method in this model, linearized CC doubles (LinCC).

99 GENERAL AND MISCELLANEOUS↗

RELATIVISTIC JETS IN THE RADIO REFERENCE FRAME IMAGE DATABASE. II. BLAZAR JET ACCELERATIONS FROM THE FIRST 10 YEARS OF DATA (1994-2003)

We analyze blazar jet apparent speeds and accelerations from the RDV series of astrometric and geodetic very long baseline interferometry (VLBI) experiments. From these experiments, we have produced and analyzed 2753 global VLBI images of 68 sources at 8 GHz with a median beam size of 0.9 milliarcseconds (mas) and a median of 43 epochs per source. From this sample, we analyze the motions of 225 jet components in 66 sources. The distribution of the fastest measured apparent speed in each source has a median of 8.3c and a maximum of 44c. Sources in the 2FGL Fermi LAT catalog display higher apparent speeds than those that have not been detected. On average, components farther from the core in a given source have significantly higher apparent speeds than components closer to the core; for example, for a typical source, components at {approx}3 mas from the core ({approx}15 pc projected at z {approx} 0.5) have apparent speeds about 50% higher than those of components at {approx}1 mas from the core ({approx}5 pc projected at z {approx} 0.5). We measure accelerations of components in orthogonal directions parallel and perpendicular to their average velocity vector. Parallel accelerations have significantly larger magnitudes than perpendicular accelerations, implying that observed accelerations are predominantly due to changes in the Lorentz factor (bulk or pattern) rather than projection effects from jet bending. Positive parallel accelerations are significantly more common than negative ones, so the Lorentz factor (bulk or pattern) tends to increase on the scales observed here. Observed parallel accelerations correspond to modest source frame increases in the bulk or pattern Lorentz factor.

79 ASTRONOMY AND ASTROPHYSICS↗