Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “parallel, scaling”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 91 records · Page 5

A generalized massively parallel ultra-high order FFT-based Maxwell solver

Dispersion-free ultra-high order FFT-based Maxwell solvers have recently proven to be paramount to a large range of applications, including the high-fidelity modeling of high-intensity laser–matter interactions with Particle-In-Cell (PIC) codes. To enable a massively parallel scaling of these solvers, a novel parallelization technique was recently proposed, which consists in splitting the simulation domain into several processor sub-domains, with guard regions appended at each sub-domain boundary. Maxwell's equations are advanced independently on each sub-domain using local shared-memory FFTs (instead of a single distributed global FFT). This implies small truncation errors at sub-domain boundaries, the amplitude of which depends on guard regions sizes and order of the Maxwell solver. For moderate guard region sizes, this ’local’ technique proved to be highly scalable on up to a million cores and notably enabled the 3D modeling of so-called plasma mirrors, for which 8 guard cells only were enough to prevent truncation error growth. Yet, for other applications, the required number of guard cells might be much higher, which would severely limit the parallel efficiency of this technique due to the large volume of guard cells to be exchanged between sub-domains. In this context, we propose a novel parallelization technique that ensures very good scaling of FFT-based solvers with an arbitrarily high number of guard cells. Our ’hybrid’ technique consists in performing distributed FFTs on local groups of processors with guard regions now appended to boundaries of each group of processors. It uses a dual domain decomposition method for the Maxwell solver and other parts of the PIC cycle to keep the simulation load-balanced. This ’hybrid’ technique was implemented in the open source exascale library PICSAR. Benchmarks show that for a large number of guard cells (>16), the ’hybrid’ technique offers up to ×3 speed-up and ×8 memory savings compared to the ’local’ one.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗

Pushing configuration-interaction to the limit: Towards massively parallel MCSCF calculations

A new large-scale parallel multiconfigurational self-consistent field (MCSCF) implementation in the open-source NWChem computational chemistry code is presented. The generalized active space approach is used to partition large configuration interaction (CI) vectors and generate a sufficient number of batches that can be distributed to the available cores. Massively parallel CI calculations with large active spaces can be performed. The new parallel MCSCF implementation is tested for the chromium trimer and for an active space of 20 electrons in 20 orbitals, which can now routinely be performed. Unprecedented CI calculations with an active space of 22 electrons in 22 orbitals for the pentacene systems were performed and a single CI iteration calculation with an active space of 24 electrons in 24 orbitals for the chromium tetramer was possible. In conclusion, the chromium tetramer corresponds to a CI expansion of one trillion Slater determinants (914 058 513 424) and is the largest conventional CI calculation attempted up to date.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

Accelerated Adaptive MGS Phase Retrieval

The Modified Gerchberg-Saxton (MGS) algorithm is an image-based wavefront-sensing method that can turn any science instrument focal plane into a wavefront sensor. MGS characterizes optical systems by estimating the wavefront errors in the exit pupil using only intensity images of a star or other point source of light. This innovative implementation of MGS significantly accelerates the MGS phase retrieval algorithm by using stream-processing hardware on conventional graphics cards. Stream processing is a relatively new, yet powerful, paradigm to allow parallel processing of certain applications that apply single instructions to multiple data (SIMD). These stream processors are designed specifically to support large-scale parallel computing on a single graphics chip. Computationally intensive algorithms, such as the Fast Fourier Transform (FFT), are particularly well suited for this computing environment. This high-speed version of MGS exploits commercially available hardware to accomplish the same objective in a fraction of the original time. The exploit involves performing matrix calculations in nVidia graphic cards. The graphical processor unit (GPU) is hardware that is specialized for computationally intensive, highly parallel computation. From the software perspective, a parallel programming model is used, called CUDA, to transparently scale multicore parallelism in hardware. This technology gives computationally intensive applications access to the processing power of the nVidia GPUs through a C/C++ programming interface. The AAMGS (Accelerated Adaptive MGS) software takes advantage of these advanced technologies, to accelerate the optical phase error characterization. With a single PC that contains four nVidia GTX-280 graphic cards, the new implementation can process four images simultaneously to produce a JWST (James Webb Space Telescope) wavefront measurement 60 times faster than the previous code.

Lam, Raymond K.↗

OBSERVATIONS OF ANISOTROPIC SCALING OF SOLAR WIND TURBULENCE

Using high-speed solar wind data recorded by the Ulysses spacecraft, we investigate and estimate the anisotropic inertial range scaling of the interplanetary magnetic field. We apply the method of the magnetic structure function (MSF), S{sup n} ({tau}) = (|b(t + {tau}) - b(t)| {sup n}) {proportional_to} {tau}{sup {zeta}(n)}, to analyze the scaling of solar wind turbulence over the range from 1 s to 10{sup 4} s. By sorting the fluctuations according to the direction of the local mean magnetic field, we obtain a second-order structure function in (r, {Theta}) coordinates that reveals the scale-dependent anisotropy of the power spectrum. The scale-dependent anisotropy of the MSF indicates that the fluctuation energy tends to cascade toward the direction perpendicular to the local field. The dependence of the MSF scaling index {zeta} on the direction of the local field is found to be similar to that reported in Horbury et al. and Podesta, with {zeta}{sub perpendicular} = 0.53 {+-} 0.18 and {zeta}{sub ||} = 1.00 {+-} 0.14. Furthermore, we estimate and find the scaling law between the perpendicular and parallel scales r {sub ||} {proportional_to} r {sup 0.614} {sub perpendicular}, which implies the elongation along the parallel direction as the turbulence eddy evolves toward the small lengthscales. These results are in agreement with the predictions of magnetohydrodynamic turbulence theory.

79 ASTRONOMY AND ASTROPHYSICS↗

Scaling the SIESTA Magnetohydrodynamics Equilibrium Code

We report the results of a scaling effort that increases both the speed and resolution of the SIESTA magnetohydrodynamic equilibrium code. SIESTA is capable of computing three-dimensional plasma equilibria with magnetic islands at high spatial resolutions for toroidally confined plasmas. Starting with a small-scale parallel im- plementation, we identified scale-dependent bottlenecks of the code and developed scalable alternatives for each performance-significant functionality, cumulatively increasing the runtime speed and scalability with the number of processors. The net outcome is an improvement in speed by over ten-fold, utilizing a few thousand processors, en- abling SIESTA to simulate high spatial-resolution scenarios in under an hour for the first time.

Seal, Sudip K↗

Building problem solving environments with the arches framework

The computational problems that scientists face are rapidly escalating in size and scope. Moreover, the computer systems used to solve these problems are becoming significantly more complex than the familiar, well-understood sequential model on their desktops. While it is possible to re-train scientists to use emerging high-performance computing (HPC) models, it is much more effective to provide them with a higher-level programming environment that has been specialized to their particular domain. By fostering interaction between HPC specialists and the domain scientists, problem-solving environments (PSEs) provide a collaborative environment. A PSE environment allows scientists to focus on expressing their computational problem while the PSE and associated tools support mapping that domain-specific problem to a high-performance computing system. This article describes Arches, an object-oriented framework for building domain-specific PSEs. The framework was designed to support a wide range of problem domains and to be extensible to support very different high-performance computing targets. To demonstrate this flexibility, two PSEs have been developed from the Arches framework to solve problem in two different domains and target very different computing platforms. The Coven PSE supports parallel applications that require large-scale parallelism found in cost-effective Beowulf clusters. In contrast, RCADE targets FPGA-based reconfigurable computing and was originally designed to aid NASA Earth scientists studying satellite instrument data.

99 GENERAL AND MISCELLANEOUS↗

A GPU‐Accelerated Generative Adversarial Model for Causal Inference

We develop a GPU-accelerated machine learning generative adversarial model designed to facilitate causal inferences from observational data. Our model's theoretical framework is conceptualized in a manner that is amenable to being operable and scalable for high-performance computing platforms. We leverage GPU acceleration to develop a parallel evolutionary algorithm to achieve large-scale parallel computation of the model within a now widely accessible computing platform. This capability both enhances computational speedup and efficiency and also extends the use of the model to a broader range of substantive research domains while maintaining the underlying theoretical properties of the model.

GPU↗

Exponential Runge-Kutta Parareal for non-diffusive equations

Parareal is a well-known parallel-in-time algorithm that combines a coarse and fine propagator within a parallel iteration. It allows for large-scale parallelism that leads to significantly reduced computational time compared to serial time-stepping methods. However, like many parallel-in-time methods it can fail to converge when applied to non-diffusive equations such as hyperbolic systems or dispersive nonlinear wave equations. Here, this paper explores the use of exponential integrators within the Parareal iteration. Exponential integrators are particularly interesting candidates for Parareal because of their ability to resolve fast-moving waves, even at the large stepsizes used by coarse propagators. This work begins with an introduction to exponential Parareal integrators followed by several motivating numerical experiments involving the nonlinear Schrödinger equation. These experiments are then analyzed using linear analysis that approximates the stability and convergence properties of the exponential Parareal iteration on nonlinear problems. The paper concludes with two additional numerical experiments involving the dispersive Kadomtsev-Petviashvili equation and the hyperbolic Vlasov-Poisson equation. These experiments demonstrate that exponential Parareal methods offer improved time-to-solution compared to serial exponential integrators when solving certain non-diffusive equations.

97 MATHEMATICS AND COMPUTING↗

Grappolo

Community detection has become a fundamental operation in numerous graph-theoretic applications. It is used to reveal natural divisions that exist within real world networks without imposing prior size or cardinality constraints on the set of communities. Despite its potential for application, there is only limited support for community detection on large-scale parallel computers, largely owing to the irregular and inherently sequential nature of the underlying heuristics. In this paper, we present parallelization heuristics for fast community detection using the Louvain method as the serial template. The Louvain method is an iterative heuristic for modularity optimization. Originally developed by Blondel et al. in 2008, the method has become increasingly popular owing to its ability to detect high modularity community partitions in a fast and memory-efficient manner. However, the method is also inherently sequential, thereby limiting its scalability. Here, we observe certain key properties of this method that present challenges for its parallelization, and consequently propose heuristics that are designed to break the sequential barrier. For evaluation purposes, we implemented our heuristics using OpenMP multithreading, and tested them over real world graphs derived from multiple application domains (e.g., internet, citation, biological). Compared to the serial Louvain implementation, our parallel implementation is able to produce community outputs with a higher modularity for most of the inputs tested, in comparable number of iterations, while providing real speedups of up to 8x using 32 threads. In addition, our parallel implementation was able to exhibit weak scaling properties on up to 32 threads.

Halappanavar, Mahantesh↗

Fluctuation Dynamo in a Collisionless, Weakly Magnetized Plasma

Results from a numerical study of fluctuation dynamo in a collisionless, weakly magnetized plasma are presented. The key difference between this dynamo and its magnetohydrodynamic (MHD) counterpart is the adiabatic production of magnetic-field-aligned pressure anisotropy by the amplification of a weak seed field. This, in turn, drives kinetic instabilities on the ion-Larmor scale—namely, firehose and mirror—which sever the adiabatic link between the thermal and magnetic pressures, thereby allowing the dynamo to proceed. After an initial phase of rapid growth driven by these instabilities, the magnetic energy grows exponentially and exhibits a ${k}^{3/2}$ spectrum that peaks near the resistive scale, similar to the large-magnetic-Prandtl-number ($\mathrm{Pm}\gg 1$) MHD dynamo. The magnetic field self-organizes into a folded-sheet topology, with direction reversals at the resistive scale and field lines curved at the parallel scale of the flow. The effective $\mathrm{Pm}$ is determined by whether the ion-Larmor scale is above or below the field-reversing scale: in the former case, particles undergo Bohm-like diffusion; in the latter case, particles scatter primarily off of firehose fluctuations residing at the ends of the magnetic folds, and the viscosity becomes anisotropic. The magnetic field ultimately saturates at dynamical strengths, with its spectral peak migrating toward larger scales. This feature, along with an anti-correlation of magnetic-field strength and field-line curvature and a gradual thinning of magnetic sheets into ribbons, resembles the saturated state of the large-$\mathrm{Pm}$ dynamo, the primary differences manifesting in firehose/mirror-unstable regions. Furthermore, these results have implications for magnetic-field growth in the weakly collisional intracluster medium of galaxy clusters.

79 ASTRONOMY AND ASTROPHYSICS↗

Efficient Extraction of Regional Subsets from Massive Climate Datasets using Parallel IO

The size of datasets produced by current climate models is increasing rapidly to the scale of petabytes. To handle data at this scale parallel analysis tools are required, however the majority of climate analysis software remains at the scale of workstations. Further, many climate analysis tools adequately process regularly gridded data but lack sufficient features when handling unstructured grids. This paper presents a data-parallel subsetter capable of correctly handling unstructured grids while scaling to over 2000 cores. The approach is based on the partitioned global address space (PGAS) parallel programming model and one-sided communication. The paper demonstrates that IO remains the single greatest bottleneck for this domain of applications and that parallel analysis of climate data succeeds in practice.

54 ENVIRONMENTAL SCIENCES↗

Enabling kilometer-scale E3SM land model simulation over North America: A new integrated framework solution

This study introduces a novel framework designed to enhance the performance, scalability, and portability of the kilometer-scale E3SM Land Model (km-ELM) within the E3SM modeling infrastructure. By seamlessly integrating cutting-edge data tools, we address existing challenges such as slow performance, limited scalability, and difficulties in software integration in current data-driven ELM simulation over large geographic areas. Our innovative approach leverages the KiloCraft data toolkit to generate unified inputs for simulations ranging from a single-cite case, to a 72,083-cell regional case to a continental configuration encompassing 21.6 million land grid cells at a 1 km × 1 km resolution. We conduct extensive strong- and weak-scaling experiments on three state-of-the-art supercomputers, utilizing up to 100,800 CPU cores across 2400 compute nodes to evaluate end-to-end metrics including wall-clock time, simulation-years-per-day (SYPD), initialization costs, and I/O throughput. Our results reveal the land (LND) component’s efficient scaling, demonstrating near-ideal weak scaling and strong-scaling parallel efficiencies reaching up to 87% at 50,400 cores. We confirm portability and reproducibility through bitwise-equivalent outputs across different machines using identical inputs over supported machines. Notably, at extreme scales, we identify I/O as a critical bottleneck and that leads to effective solution with the SCORPIO/ADIOS stack. Collectively, these findings validate the deployment of km-ELM at a continental scale with high parallel efficiency and provide essential guidance on configuration, decomposition, and I/O settings for optimized kilometer-scale land simulations in E3SM. This work emphasizes the innovative design and practical solutions that enhance the operational capabilities of km-ELM, focusing on software performance and scalability while leaving detailed scientific evaluations of simulated land processes for future investigations.

E3SM land model (ELM), km-ELM, scalability, perfor↗

Design Considerations for GPU-based Mixed Integer Programming on Parallel Computing Platforms

Mixed Integer Programming (MIP) is a powerful abstraction in combinatorial optimization that finds real-life application across many significant sectors. The recent proliferation of graphical processing unit (GPU)-based accelerated computing architectures in large-scale parallel computing or supercomputing presents new opportunities as well as challenges in the advancement of MIP solver technology to effectively use the new accelerated computing platforms and scale to large parallel systems. Here, we recount the conventional processor-based strategies and focus on configurations where the most promising intersection lies between parallel MIP solver approaches and the specific strengths of accelerated parallel platforms. We note that the best potential lies in solving problems whose individual matrix sizes (of the linear program relaxation) fit entirely within one accelerator's memory and whose branch-and-bound (or branch-and-cut) trees cannot be fully contained within a small number of computational nodes. Additionally, we identify ideal features of computational linear algebra support on GPU accelerators that would help advance this direction of scalable parallel solution of MIP problems on GPU-based accelerated computing architectures.

Perumalla, Kalyan↗

Parallel Simulation of Unsteady Turbulent Flames

Time-accurate simulation of turbulent flames in high Reynolds number flows is a challenging task since both fluid dynamics and combustion must be modeled accurately. To numerically simulate this phenomenon, very large computer resources (both time and memory) are required. Although current vector supercomputers are capable of providing adequate resources for simulations of this nature, the high cost and their limited availability, makes practical use of such machines less than satisfactory. At the same time, the explicit time integration algorithms used in unsteady flow simulations often possess a very high degree of parallelism, making them very amenable to efficient implementation on large-scale parallel computers. Under these circumstances, distributed memory parallel computers offer an excellent near-term solution for greatly increased computational speed and memory, at a cost that may render the unsteady simulations of the type discussed above more feasible and affordable.This paper discusses the study of unsteady turbulent flames using a simulation algorithm that is capable of retaining high parallel efficiency on distributed memory parallel architectures. Numerical studies are carried out using large-eddy simulation (LES). In LES, the scales larger than the grid are computed using a time- and space-accurate scheme, while the unresolved small scales are modeled using eddy viscosity based subgrid models. This is acceptable for the moment/energy closure since the small scales primarily provide a dissipative mechanism for the energy transferred from the large scales. However, for combustion to occur, the species must first undergo mixing at the small scales and then come into molecular contact. Therefore, global models cannot be used. Recently, a new model for turbulent combustion was developed, in which the combustion is modeled, within the subgrid (small-scales) using a methodology that simulates the mixing and the molecular transport and the chemical kinetics within each LES grid cell. Finite-rate kinetics can be included without any closure and this approach actually provides a means to predict the turbulent rates and the turbulent flame speed. The subgrid combustion model requires resolution of the local time scales associated with small-scale mixing, molecular diffusion and chemical kinetics and, therefore, within each grid cell, a significant amount of computations must be carried out before the large-scale (LES resolved) effects are incorporated. Therefore, this approach is uniquely suited for parallel processing and has been implemented on various systems such as: Intel Paragon, IBM SP-2, Cray T3D and SGI Power Challenge (PC) using the system independent Message Passing Interface (MPI) compiler. In this paper, timing data on these machines is reported along with some characteristic results.

Menon, Suresh↗

High-frequency heating of the solar wind triggered by low-frequency turbulence

The fast solar wind’s high speeds and non-thermal features require that considerable heating occurs well above the Sun’s surface. Two leading theories seem incompatible: low-frequency ‘Alfvénic’ turbulence, which transports energy outwards and is observed ubiquitously by spacecraft but seems insufficient to explain the observed dominance of ion over electron heating; and high-frequency ion-cyclotron waves, which explain the non-thermal heating of ions but lack an obvious source. Here we argue that the recently proposed ‘helicity barrier’ effect, which limits electron heating by inhibiting the turbulent cascade of energy to the smallest scales, can unify these two paradigms. Our six-dimensional simulations show how the helicity barrier causes the large-scale energy to grow through time, generating small parallel scales and high-frequency ion-cyclotron-wave heating from low-frequency turbulence, while simultaneously explaining various other long-standing observational puzzles. Furthermore, the predicted causal link between plasma expansion and the ion-to-electron heating ratio suggests that the helicity barrier could contribute to key observed differences between fast and slow wind streams.

79 ASTRONOMY AND ASTROPHYSICS↗

Portability and Cross-Platform Performance of an MPI-Based Parallel Polygon Renderer

Visualizing the results of computations performed on large-scale parallel computers is a challenging problem, due to the size of the datasets involved. One approach is to perform the visualization and graphics operations in place, exploiting the available parallelism to obtain the necessary rendering performance. Over the past several years, we have been developing algorithms and software to support visualization applications on NASA's parallel supercomputers. Our results have been incorporated into a parallel polygon rendering system called PGL. PGL was initially developed on tightly-coupled distributed-memory message-passing systems, including Intel's iPSC/860 and Paragon, and IBM's SP2. Over the past year, we have ported it to a variety of additional platforms, including the HP Exemplar, SGI Origin2OOO, Cray T3E, and clusters of Sun workstations. In implementing PGL, we have had two primary goals: cross-platform portability and high performance. Portability is important because (1) our manpower resources are limited, making it difficult to develop and maintain multiple versions of the code, and (2) NASA's complement of parallel computing platforms is diverse and subject to frequent change. Performance is important in delivering adequate rendering rates for complex scenes and ensuring that parallel computing resources are used effectively. Unfortunately, these two goals are often at odds. In this paper we report on our experiences with portability and performance of the PGL polygon renderer across a range of parallel computing platforms.

Crockett, Thomas W.↗

Advances in Parallelization for Large Scale Oct-Tree Mesh Generation

Despite great advancements in the parallelization of numerical simulation codes over the last 20 years, it is still common to perform grid generation in serial. Generating large scale grids in serial often requires using special "grid generation" compute machines that can have more than ten times the memory of average machines. While some parallel mesh generation techniques have been proposed, generating very large meshes for LES or aeroacoustic simulations is still a challenging problem. An automated method for the parallel generation of very large scale off-body hierarchical meshes is presented here. This work enables large scale parallel generation of off-body meshes by using a novel combination of parallel grid generation techniques and a hybrid "top down" and "bottom up" oct-tree method. Meshes are generated using hardware commonly found in parallel compute clusters. The capability to generate very large meshes is demonstrated by the generation of off-body meshes surrounding complex aerospace geometries. Results are shown including a one billion cell mesh generated around a Predator Unmanned Aerial Vehicle geometry, which was generated on 64 processors in under 45 minutes.

O'Connell, Matthew↗