Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “parallel, scaling”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 631 records · Page 35

Extreme-scale stochastic optimization and simulation via learning-enhanced decomposition and parallelization (Final Technical Report)

Stochastic optimization and simulation models ubiquitously arise in designing and operating complex service/engineering systems. They can be extreme in scale due to high-dimensional data and decisions, and can also involve decisions made sequentially in response to newly revealed data, both causing significant computational challenge. The objective of this research is to explore a unified framework that integrates machine learning with discrete optimization and risk-averse modeling, to improve the efficiency of decomposition paradigms for stochastic optimization and simulations at extreme scale. The models we consider represent a broad class of complex decision-making problems, where 0-1 or continuous decisions are made before and/or after knowing multiple sources of uncertainties that could be correlated. We will employ machine learning methods to dynamically decide and prioritize computational procedures, including cut generation, branching, and bounding of the optimal objective. Furthermore, the research will shed new lights on the traditional decomposition algorithms for extreme-scale computing. Deliverables of the research include new modeling and computational methods for advancing the state-of-the-art research in optimization and simulation, bringing many relevant risk-averse, data-driven optimization problems in practice within the range of tractability. Examples include distributed computing server scheduling and sensor deployment for monitoring critical infrastructures. Success in this effort will enable progress in solving multiple extreme-scale problems in the complex system design and operations arising from DoE missions in energy, environment, and national security.

24 POWER TRANSMISSION AND DISTRIBUTION↗

Transport of cosmic-ray protons in intermittent heliospheric turbulence: Model and simulations

The transport of charged energetic particles in the presence of strong intermittent heliospheric turbulence is computationally analyzed based on known properties of the interplanetary magnetic field and solar wind plasma at 1 astronomical unit. The turbulence is assumed to be static, composite, and quasi-three-dimensional with a varying energy distribution between a one-dimensional Alfvénic (slab) and a structured two-dimensional component. The spatial fluctuations of the turbulent magnetic field are modeled either as homogeneous with a Gaussian probability distribution function (PDF), or as intermittent on large and small scales with a q-Gaussian PDF. Simulations showed that energetic particle diffusion coefficients both parallel and perpendicular to the background magnetic field are significantly affected by intermittency in the turbulence. This effect is especially strong for parallel transport where for large-scale intermittency results show an extended phase of subdiffusive parallel transport during which cross-field transport diffusion dominates. The effects of intermittency are found to depend on particle rigidity and the fraction of slab energy in the turbulence, yielding a perpendicular to parallel mean free path ratio close to 1 for large-scale intermittency. Investigation of higher order transport moments (kurtosis) indicates that non-Gaussian statistical properties of the intermittent turbulent magnetic field are present in the parallel transport, especially for low rigidity particles at all times.

79 ASTRONOMY AND ASTROPHYSICS↗

Aspherical particle models for molecular dynamics simulation

In traditional molecular dynamics (MD) simulations, atoms and coarse-grained particles are modeled as point masses interacting via isotropic potentials. For studies where particle shape plays a vital role, more complex models are required. Here, we describe a spectrum of approaches for modeling aspherical particles, all of which are now available (some recently) as options within the LAMMPS MD package. Broadly these include two classes of models. In the first, individual particles are aspherical, either via a pairwise anisotropic potential which implicitly assigns a simple geometric shape to each particle, or in a more general way where particles store internal state which can explicitly define a complex geometric shape. In the second class of models, individual particles are simple points or spheres, but rigid body constraints are used to create composite aspherical particles in a variety of complex shapes. This work discusses parallel algorithms and associated data structures for both kinds of models, which enable dynamics simulations of aspherical particle systems across a wide range of length and time scales. We also highlight parallel performance and scalability and give a few illustrative examples of aspherical models in different contexts.

74 ATOMIC AND MOLECULAR PHYSICS↗

Parallel multiple-chain DRAM MCMC for large-scale geosteering inversion and uncertainty quantification

Geosteering is the proactive control of a wellbore placement based on the downhole measurements, aiming at maximizing economic production from the well. We report the recent development of azimuthal resistivity logging-whiledrilling (LWD) tools have a much larger depth of detection, thus sets higher demands for geosteering inversion as more unknown parameters need to be taken into account in the earth model to be inverted. For complicated nonlinear problems, traditional deterministic inversion methods are more likely to be trapped near local optima. In general, Markov chain Monte Carlo (MCMC) inversion methods are more capable of finding the global optimal solution and providing additional uncertainty analyses by sampling from the target distribution. However, MCMC methods usually have the problem of slow convergence. Even though optimization methods like delayed rejection (DR) and adaptive Metropolis (AM) were proposed to speed up the convergence, using MCMC methods to solve geosteering inversion problems may still incur unacceptable time cost. To reduce the sampling cost of MCMC methods, we use parallel multiple-chain DRAM MCMC methods to solve geosteering inverse problems and estimate the corresponding uncertainty. A clustering method, density-based spatial clustering of applications with noise (DBSCAN), is applied to select the best solution among many results. Lastly, the simulation results show that running many relatively short MCMC chains for one problem can obtain a similar result as running a long single chain. Besides, avoiding communications between multiple Markov chains during the sampling process yields almost linear scalability.

97 MATHEMATICS AND COMPUTING↗

Partitioning of unstructured problems for parallel processing

Many large-scale computational problems are based on unstructured computational domains. Primary examples are unstructured grid calculations based on finite volume methods in computational fluid dynamics, or structural analysis problems based on finite element approximations. The question of how to distribute such unstructured computational domains over a large number of processors in a MIMD machine with distributed memory is addressed. A graph theoretical framework for these problems is established. Based on this framework three decomposition algorithms are introduced. In particular a new decomposition algorithm is discussed, which is based on the computation of an eigenvector of the Laplacian matrix associated with the graph. Numerical comparisons on large-scale two- and three-dimensional problems demonstrate the superiority of the new spectral bisection algorithm.

Simon, H. D.↗

Dispersive and viscous scale lengths in the two-stage ion heating at quasi-parallel collisionless shocks

A simulation of quasi-parallel (QP) collisionless shock waves is used to deduce the dispersive and viscous scale lengths of QP shocks. The findings were found to be consistent with the observational results obtained by Scudder et al. (1984) and Kennel et al. (1984). It is shown that the upstream wavelength is the dispersive scale length which governs the magnetic-field structure of the shock. It is also shown that the two-stage ion heating obtained by Mandt and Kan (1988) is an intrinsic characteristic of the ion heating process in QP shocks.

Mandt, M. E.↗

Hot proton anisotropies and cool proton temperatures in the outer magnetosphere

The hot protons of the outer magnetosphere typically exhibit a temperature anistropy such that T(sub perp)/T(sub parallel) greater than 1, where perpendicular and parallel symbols denote directions relative to the background magnetic field. If this anisotrpy is sufficiently large, the electomagneitc proton cyclotron anistropy instability may be excited. This instability is studied using linear Vlasov theory and one-dimensional hybrid simulations for a homogeneous plasma model representative of conditions in the outer magnetosphere with a hot anisotropic proton component (denoted by subscript h) and a cool, initially isotropic proton component (subscript c). Linear theory yields an instability threshold condition on the hot proton temperature anistropy where as the simulations imply an upper bound on T(sub perp h)/T(sub parallel h); both the threshold and the upper bound have similar scaling with the maximum growth rate gamma (sub m), the parallel beta of the hot component, beta(sub parallel h), and the relative density of the hot component n(sub h)/n(sub e). An anlysis of plasma observations from the Los Alamos magnetospheric plasma analyzer (MPA) in geosynchronous orbits finds that the maximum value of the hot proton temperature anisotropy approximately satisfies the predicted scaling with beta(sub parallel h) and nu(sub h)/n(sub e) and yields the proportionality factor that quantifies this upper bound. The simulations are also used to examine the heating of the cool proton cyclotron instability. The simulations yield a scaling for the dimensionless late-time cool proton average temperature T(sub c)/T(sub parallel h) as (n(sub h)/n(sub e))/beta(sub parallel h exp 0.5). Analysis of MPA data shows that the observed values of T(sub c)/T(sub parallel h) have similar scaling and again yield the proportionality factor which quantifies this relationship.

Gary, S. Peter↗

Intrinsic rotation driven by the electrostatic turbulence in up-down asymmetric toroidal plasmas

The transport of parallel momentum by small scale fluctuations is intrinsically linked to symmetry breaking in the direction of the magnetic field. In tokamaks, an up-down asymmetry in the equilibrium proves to be an efficient parallel symmetry breaking mechanism leading to the generation of a net radial flux of parallel momentum by the electrostatic turbulence [Y. Camenen et al., Phys. Rev. Lett. 102, 125001 (2009)]. This flux is neither proportional to the toroidal rotation nor to its gradient and arises from an incomplete cancellation of the local contributions to the parallel momentum flux under the flux surface average. The flux of parallel momentum then depends on the asymmetry of the curvature drift and on the extension of the fluctuations around the low field side midplane. In this paper, the mechanisms underlying the generation of the flux of parallel momentum are highlighted and the main dependences on plasma parameters investigated using linear gyrokinetic simulations.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗

Scalable training of graph convolutional neural networks for fast and accurate predictions of HOMO-LUMO gap in molecules

Abstract Graph Convolutional Neural Network (GCNN) is a popular class of deep learning (DL) models in material science to predict material properties from the graph representation of molecular structures. Training an accurate and comprehensive GCNN surrogate for molecular design requires large-scale graph datasets and is usually a time-consuming process. Recent advances in GPUs and distributed computing open a path to reduce the computational cost for GCNN training effectively. However, efficient utilization of high performance computing (HPC) resources for training requires simultaneously optimizing large-scale data management and scalable stochastic batched optimization techniques. In this work, we focus on building GCNN models on HPC systems to predict material properties of millions of molecules. We use HydraGNN, our in-house library for large-scale GCNN training, leveraging distributed data parallelism in PyTorch. We use ADIOS, a high-performance data management framework for efficient storage and reading of large molecular graph data. We perform parallel training on two open-source large-scale graph datasets to build a GCNN predictor for an important quantum property known as the HOMO-LUMO gap. We measure the scalability, accuracy, and convergence of our approach on two DOE supercomputers: the Summit supercomputer at the Oak Ridge Leadership Computing Facility (OLCF) and the Perlmutter system at the National Energy Research Scientific Computing Center (NERSC). We present our experimental results with HydraGNN showing (i) reduction of data loading time up to 4.2 times compared with a conventional method and (ii) linear scaling performance for training up to 1024 GPUs on both Summit and Perlmutter.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

Approximate Weighted Matching On Emerging Manycore and Multithreaded Architectures

Graph matching is a prototypical combinatorial problem with many applications in computer science and scientific computing, but algorithms for computing optimal matchings are challenging to parallelize. Approximate matching algorithms provide an alternate route for parallelization, and in many contexts compute near-optimal matchings for large-scale graphs. We present sharedmemory parallel implementations for computing half-approximate weighted matching on state-of-the-art multicore (Intel Nehalem and AMD Magny-Cours), manycore (Nvidia Tesla and Nvidia Fermi) and massively multithreaded (Cray XMT) platforms. We provide two implementations: the first implementation uses shared work queues, and is suited to all these platforms; the second implementation is based on dataflow principles, and exploits the architectural features of the Cray XMT. Using a carefully chosen dataset that exhibits characteristics from a wide range of real-world applications, we show scalable performance across different platforms. In particular, for one instance of the input, an R-MAT graph (RMAT-G), we show speedups of: about 32 on 48 cores of an AMD Magny-Cours; 7 on 8 cores of Intel Nehalem; 3 on Nvidia Tesla and 10 on Nvidia Fermi relative to one core of Intel Nehalem; and 60 on 128 processors of Cray XMT. We demonstrate good weak and strong scaling for graphs with up to a billion edges using up to 12, 800 threads. Given the breadth of this work, we focus on simplicity and portability of software rather than excessive fine-tuning for each platform. To the best of our knowledge, this is the first such large-scale study of the half-approximate weighted matching problem on shared-memory platforms. Driven by the critical enabling role of combinatorial algorithms such as matching in scientific computing and the emergence of informatics applications, there is a growing demand to support irregular computations on current and future computing platforms. In this context, we evaluate the capability of emerging multithreaded platforms to tolerate latency induced by irregular memory access patterns, and to support fine-grained parallelism via light-weight synchronization mechanisms. By contrasting the architectural features of these platforms against the Cray XMT, which is specifically designed to support irregular memory-intensive applications, we delineate the impact of these choices on performance.

Halappanavar, Mahantesh↗

Machine Learning Assisted HPC Workload Trace Generation for Leadership Scale Storage Systems

Monitoring and analyzing a wide range of I/O activities in an HPC cluster is important in maintaining mission-critical performance in a large-scale, multi-user, parallel storage system. Center-wide I/O traces can provide high-level information and fine-grained activities per application or per user running in the system. Studying such large-scale traces can provide helpful insights into the system. It can be used to develop predictive methods for making predictive decisions, adjusting scheduling policies, or providing decisions for the design of next-generation systems. However, sharing real-world I/O traces to expedite such research efforts leaves a few concerns; i) the cost of sharing the large traces is expensive due to this large size, and ii) privacy concern is an issue.We address such issues by building an end-to-end machine learn- ing (ML) workflow that can generate I/O traces for large-scale HPC applications. We leverage ML based feature selection and gener- ative models for I/O trace generation. The generative models are trained on I/O traces collected by the darshan I/O characterization tool over a period of one year. We present a two-step generation process consisting of two deep-learning models, called the feature generator and the trace generator. The combination of two-step generative models provides robustness by reducing the bias of the model and accounting for the stochastic nature of the I/O traces across different runs of an application. We evaluate the performance of the generative models and show that the two-step model can generate time-series I/O traces with less than 20% root mean square error.

Paul, Arnab↗

Evidence of the ion cyclotron resonance at proton kinetic scales in the solar wind

The k-filtering and wave polarization techniques are applied to Cluster magnetic field data at four intervals of fast free (not connected to Earth's foreshock) solar wind. In addition to the commonly observed population of magnetic field fluctuations propagating at quasi-perpendicular angles to the global mean field B{sub 0}, a population of fluctuations propagating at quasi-parallel angles are also observed when no local plasma instability can be identified. At low wavenumbers (kv{sub A}/Ω{sub p} ≤ 0.6) both components are present, and have powers of similar strength, while at higher wavenumbers (kv{sub A}Ω{sub p} > 0.6) only the perpendicular component can be identified. Here v{sub A} and Ω{sub p} are the Alfvén speed and the proton (angular) gyration frequency, k denotes the wavenumber. The dispersion curve obtained (and the ratio of the magnitudes of left and right hand polarized fluctuations) shows a sharp transition (and decrease) at (kv{sub A}/Ω{sub p}) ∼ 0.6, which corresponds to a spacecraft frequency f {sub sc} = 0.32 Hz. At higher wavenumbers the measurements agree with previous studies indicating the presence of kinetic Alfvén wave (KAW) turbulence or a mixture of KAW turbulence and convected structures. The parallel component displays a scaling of k{sub ∥}{sup −δ} where the spectral index δ ranges between about 5/3 and 2 although the number of data points does not warrant an accurate determination. The origins of the observed parallel component are unclear. The presence of a parallel component is consistent with ion cyclotron waves, and we interpret the sharp drop of this component at higher wavenumbers as evidence that the ion cyclotron resonance is occurring in the solar wind intervals studied.

79 ASTRONOMY AND ASTROPHYSICS↗

Ion temperature and rotation fluctuation measurements with ultra-fast charge exchange recombination spectroscopy (UF-CHERS) in the DIII-D tokamak

An upgraded detector and several optimizations have significantly improved the Ultra-Fast Charge Exchange Recombination Spectroscopy (UF-CHERS) diagnostic sensitivity to ion temperature and parallel velocity fluctuations at turbulence relevant spatio-temporal scales. Normalized broadband ion temperature and parallel velocity fluctuations down to x̃x∼1% (x = Ti, v∥) and up to ∼450 kHz have been measured in a variety of plasmas. The multi-field nature of the CHERS technique also allows measurements of the cross-phase angles of the fluctuating fields. UF-CHERS is optimized to observe emissions from the electron exchange reaction between intrinsic C6+ and hydrogenic neutral beam injected particles near 529 nm. UF-CHERS consists of two chords separated by ∼1 cm radially, less than the turbulence correlation length in DIII-D plasmas, which enables correlated measurements to suppress incoherent electronic and photon noise. The optical components of the spectrometer include a volume-phase-holographic grating with >90% transmission between 528 and 530 nm and f/2 200-mm lenses, selected to maximize the optical efficiency and photon flux. Diffracted light from each chord is collected in eight spectral bins, each with a bandwidth of ∼0.25 nm, and detected and amplified by chilled avalanche photodiodes and custom high-gain, wide bandwidth low-noise preamplifiers to achieve the optimal signal-to-noise ratio. The resulting signals are digitized at 1 MHz, 103–104× faster than the conventional CHERS diagnostics. Spatial coverage is achieved by repositioning a motorized fiber tray between plasmas. UF-CHERS measurements will advance the understanding of turbulent ion transport and contribute to the validation of transport models and simulations.

Truong, D. D. (ORCID:0000000285732539)↗

Hierarchical Parallelism in Finite Difference Analysis of Heat Conduction

Based on the concept of hierarchical parallelism, this research effort resulted in highly efficient parallel solution strategies for very large scale heat conduction problems. Overall, the method of hierarchical parallelism involves the partitioning of thermal models into several substructured levels wherein an optimal balance into various associated bandwidths is achieved. The details are described in this report. Overall, the report is organized into two parts. Part 1 describes the parallel modelling methodology and associated multilevel direct, iterative and mixed solution schemes. Part 2 establishes both the formal and computational properties of the scheme.

Padovan, Joseph↗

Simulating Billion-Task Parallel Programs

In simulating large parallel systems, bottom-up approaches exercise detailed hardware models with effects from simplified software models or traces, whereas top-down approaches evaluate the timing and functionality of detailed software models over coarse hardware models. Here, we focus on the top-down approach and significantly advance the scale of the simulated parallel programs. Via the direct execution technique combined with parallel discrete event simulation, we stretch the limits of the top-down approach by simulating message passing interface (MPI) programs with millions of tasks. Using a timing-validated benchmark application, a proof-of-concept scaling level is achieved to over 0.22 billion virtual MPI processes on 216,000 cores of a Cray XT5 supercomputer, representing one of the largest direct execution simulations to date, combined with a multiplexing ratio of 1024 simulated tasks per real task.

Perumalla, Kalyan S↗

On the use of a multigrid-reduction-in-time algorithm for multiscale convergence of turbulence simulations

Simulations of turbulent flow present challenges in terms of accuracy and affordability on modern highly-parallel computer architectures. A multigrid-reduction-in-time algorithm is used to provide a framework for separately evolving different scales of turbulence and for parallelizing the temporal domain, thereby increasing the concurrency. It is hypothesized that the space–time locality of the small scales of turbulence can be used to circumvent difficulties in applying temporal multigrid to flows dominated by inertial physics. For algorithms that fall well short of spectral accuracy (fourth-order is used in this work) attention must be paid to the accuracy of features on scales transferred between multigrid levels. Numerical experiments were performed using implicit large-eddy simulation. Results from applying the approach to an infinite-Reynolds number Taylor–Green flow and a double-shear flow at a Reynolds number of 11650 provide strong evidence that the approach has merit. The multigrid-reduction-in-time framework can be used to parallelize the temporal domain of a high-Reynolds-number turbulent flow and permit independent convergence of different scales. Establishing this foundation allows for future research in reducing the wall-clock time to solve turbulent flows while retaining the same accuracy as sequential solvers. In conclusion, current performance results from parallelizing the temporal domain are not competitive with those from sequential-in-time methods.

97 MATHEMATICS AND COMPUTING↗

TriC: Distributed-memory Triangle Counting by Exploiting the Graph Structure

Graph analytics has emerged as an important tool in the analysis of large scale data from diverse application domains such as social networks, cyber security and bioinformatics. Counting the number of triangles in a graph is a fundamental kernel with several applications such as detecting the community structure of a graph or in identifying important vertices in a graph. The ubiquity of massive datasets is driving the need to scale graph analytics on parallel systems. However, numerous challenges exist in efficiently parallelizing graph algorithms, especially on distributed-memory systems. Irregular memory accesses and communication patterns, low computation to communication ratios, and the need for frequent synchronization are some of the leading challenges. In this paper, we present TriC, our distributed-memory implementation of triangle counting in graphs using the Message Passing Interface (MPI), as a submission to the 2020 GraphChallenge competition. Using a set of synthetic and real-world inputs from the challenge, we demonstrate a speedup of up to 90x relative to previous work on 32 processor-cores of a NERSC Cori node. We also provide details from distributed runs with up to8192 processes along with strong scaling results. The observations presented in this work provide an understanding of the system-level bottlenecks at scale that specifically impact sparse-irregular workloads and will therefore benefit other efforts to parallelize graph algorithms.

Halappanavar, Mahantesh↗