Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Parallel Computing”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,189 records · Page 66
All sourcesNASA NTRSDOE OSTI
DOE OSTI2021

Immortal rays: Rethinking random ray neutron transport on GPU architectures

The Random Ray Method (TRRM) is a recently developed adaptation of the Method of Characteristics for neutral particle transport simulations. TRRM has demonstrated excellent performance on 3D nuclear reactor benchmark problems using CPU-based compute systems. When porting to GPU-based systems, however, new performance challenges arise that are unique to processors targeting massive fine-grained parallelism. For smaller problems, or for large problems that are domain decomposed across many computational nodes, the problem size per node has insufficient parallelism to saturate GPU node resources, thus greatly limiting speedup. In this study, we report on a newly developed “immortal ray” variant of TRRM. Here, the immortal ray technique exposes significantly more fine-grained parallelism by fundamentally reformulating the numerical details of ray discretization, resulting in performance tradeoffs with significant overall benefit on GPUs. For very small 2D simulation problems we found the new immortal ray variant allowed for up to a 4.4x speedup when run on a single GPU. For larger 3D simulation problems we found the new variant improved strong scaling by 3x when run on the Summit supercomputer.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2022

OpenMP Target Task: Tasking and Target Offloading on Heterogeneous Systems

This work evaluated the use of OpenMP tasking with target GPU offloading as a potential solution for programming productivity and performance on heterogeneous systems. Also, it is proposed a new OpenMP specification to make the implementation of heterogeneous codes simpler by using OpenMP target task, which integrates both OpenMP tasking and target GPU offloading in a single OpenMP pragma. As a test case, the authors used one of the most popular and widely used Basic Linear Algebra Subprogram Level-3 routines: triangular solver (TRSM). To benefit from the heterogeneity of the current high-performance computing systems, the authors propose a different parallelization of the algorithm by using a nonuniform decomposition of the problem. This work used target GPU offloading inside OpenMP tasks to address the heterogeneity found in the hardware. This new approach can outperform the state-of-the-art algorithms, which use a uniform decomposition of the data, on both the CPU-only and hybrid CPU-GPU systems, reaching speedups of up to one order of magnitude. The performance that this approach achieves is faster than the IBM ESSL math library on CPU and competitive relative to a highly optimized heterogeneous CUDA version. One node of Oak Ridge National Laboratory’s supercomputer, Summit, was used for performance analysis.

Valero Lara, Pedro↗
DOE OSTI2026

Multi-task Parallelism for Robust Pre-training of Graph Foundation Models on Multi-source, Multi-fidelity Atomistic Modeling Data

Graph foundation models using graph neural networks promise sustainable, efficient atomistic modeling. To tackle challenges of processing multi-source, multi-fidelity data during pre-training, recent studies employ multi-task learning, in which shared message passing layers initially process input atomistic structures regardless of source, then route them to multiple decoding heads that predict data-specific outputs. This approach stabilizes pre-training and enhances a model’s transferability to unexplored chemical regions. Preliminary results on approximately four million structures are encouraging, yet questions remain about generalizability to larger, more diverse datasets and scalability on supercomputers. We propose a multi-task parallelism method that distributes each head across computing resources with GPU acceleration. Implemented in the open-source HydraGNN architecture, our method was trained on over 24 million structures from five datasets and tested on the Perlmutter, Aurora, and Frontier supercomputers, demonstrating efficient scaling on all three highly heterogeneous super-computing architectures.

Lupo Pasini, Massimiliano [ORNL] (ORCID:0000000249↗
DOE OSTI2025

How does ion temperature gradient turbulence depend on magnetic geometry? Insights from data and machine learning

Magnetic geometry has a significant effect on the level of turbulent transport in fusion plasmas. Here, we model and analyse this dependence using multiple machine learning methods and a dataset of >200 000 nonlinear gyrokinetic simulations of ion-temperature-gradient turbulence in diverse non-axisymmetric geometries. The dataset is generated using a large collection of both optimised and randomly generated stellarator equilibria. At fixed gradients and other input parameters, the turbulent heat flux varies between geometries by several orders of magnitude. Trends are apparent among the configurations with particularly high or particularly low heat flux. Regression and classification techniques from machine learning are then applied to extract patterns in the dataset. Due to a symmetry of the gyrokinetic equation, the heat flux and regressions thereof should be invariant to translations of the raw features in the parallel coordinate, similar to translation invariance in computer vision applications. Multiple regression models including convolutional neural networks (CNNs) and decision trees can achieve reasonable predictive power for the heat flux in held-out test configurations, with highest accuracy for the CNNs. Using Spearman correlation, sequential feature selection and Shapley values to measure feature importance, it is consistently found that the most important geometric lever on the heat flux is the flux surface compression in regions of bad curvature. The second most important geometric feature relates to the magnitude of geodesic curvature. These two features align remarkably with surrogates that have been proposed based on theory, while the methods here allow a natural extension to more features for increased accuracy. The dataset, released with this publication, may also be used to test other proposed surrogates, and we find that many previously published proxies do correlate well with both the heat flux and stability boundary.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗
DOE OSTI2021

Proactive Operations and Investment Planning via Stochastic Optimization to Enhance Power Systems’ Extreme Weather Resilience

We present scalable stochastic optimization approaches for improving power systems’ resilience to extreme weather events. We consider both proactive redispatch and transmission line hardening as alternatives for mitigating expected load shed due to extreme weather, resulting in large-scale stochastic linear programs (LPs) and mixed-integer linear programs (MILPs). We solve these stochastic optimization problems with progressive hedging (PH), a parallel, scenario-based decomposition algorithm. Our computational experiments indicate that our proposed method for enhancing power system resilience can provide high-quality solutions efficiently. With up to 128 scenarios on a 2,000-bus network, the operations (redispatch) and investment (hardening) resilience problems can be solved in approximately 6 min and 2 h of wall-clock time, respectively. Additionally, we solve the investment problems with up to 512 scenarios, demonstrating that the approach scales very well with the number of scenarios. Moreover, the method produces high quality solutions that result in statistically significant reductions in expected load shed. Our proposed approach can be augmented to incorporate a variety of other operational and investment resilience strategies, or a combination of such strategies.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2016

Graph-based linear scaling electronic structure theory

We show how graph theory can be combined with quantum theory to calculate the electronic structure of large complex systems. The graph formalism is general and applicable to a broad range of electronic structure methods and materials, including challenging systems such as biomolecules. The methodology combines well-controlled accuracy, low computational cost, and natural low-communication parallelism. This combination addresses substantial shortcomings of linear scaling electronic structure theory, in particular with respect to quantum-based molecular dynamics simulations.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2018

Tomographic analysis of tangential viewing cameras (invited)

Many tokamaks now use visible light cameras to observe plasma-wall interactions and integrated line emission. The DIII-D coherence imaging spectroscopy diagnostic cameras image interferograms that encode line integrated velocity. By modeling the 2D camera image pixels as line of sight integrals through an axisymmetric discrete grid, it is possible to do tomographic analysis to determine the local plasma line emissivity and parallel velocity. Approaches to solve the inverse problem posed by these tangential viewing cameras are presented. The inversion begins with calculation of the sparse response matrix that encompasses all the geometry and diagnostic information and reduces the process of image formation to a sparse matrix-vector multiply. Our report includes techniques for determining the detailed geometry of the camera views and methods for handling physical quantities that vary spatially. Additionally, the size of the response matrix has driven the development of capability to distribute the coarse parallel calculation across a heterogeneous cluster of computers on the Energy Sciences Network. Iterative techniques are then used to solve the sparse matrix-vector linear system

47 OTHER INSTRUMENTATION↗
DOE OSTI2026

Application of Portable Parallelization Strategies for GPUs on track reconstruction kernels

Utilizing the computational power of GPUs is one of the key ingredients to meet the computing challenges presented to the next generation of High-Energy Physics (HEP) experiments. Unlike CPUs, developing software for GPUs often involves using architecturespecific programming languages promoted by the GPU vendors and hence limits the platform that the code can run on. Various portability solutions have been developed to achieve portable, performant software across different GPU vendors. Given the rapid evolution of these portability solutions, an early adoption of them in simple HEP testbed applications will help us understand the strengths and weaknesses of respective approaches.We apply several portability solutions, including Alpaka, Kokkos, SYCL and std::execution::par, on kernels for track propagation extracted from the mkFit project. We report on the development experience of the same application with different portability solutions, as well as their performance on GPUs, measured as the throughput of the kernels, from different manufacturers such as NVIDIA, AMD and Intel.

Kwok, Martin [Fermilab] (ORCID:0000000286936146)↗
DOE OSTI2023

A hybrid 3D/2D field response calculation for liquid argon detectors with PCB based anode plane

Liquid Argon Time Projection Chamber (LArTPC) technology is commonly utilized in neutrino detector designs. It enables detailed reconstruction of neutrino events with high spatial precision and low energy threshold. Its field response (FR) model describes the time-dependent electric currents induced in the anode-plane electrodes when ionization electrons drift nearby. An accurate and precise FR is a crucial input to LArTPC detector simulations and charge reconstruction. Established LArTPC designs have been based on parallel wire planes. It allows accurate and computationally economic two-dimensional (2D) FR models utilizing the translational symmetry along the direction of the wires. Recently, novel LArTPC designs utilize electrodes formed on printed circuit board (PCB) in the shape of strips with through holes. The translational symmetry is no longer a good approximation near the electrodes and a new FR calculation that employs regions with three dimensions (3D) has been developed. Extending the 2D models to 3D would be computationally expensive. Fortuitously, the nature of strips with through holes allows for a computationally economic approach based on the finite-difference method (FDM). In this paper, we present a new software package pochoir that calculates LArTPC field response for these new strip-based anode designs. This package combines 3D calculations in the volume near the electrodes with 2D far-field solutions to achieve fast and precise field response computation. We apply the resulting FR to simulate and reconstruct samples of cosmic-ray muons and 39 Ar decays from a Vertical Drift (VD) detector prototype operated at CERN. We find the difference between real and simulated data within 5%. Current state-of-the-art LArTPC software requires a 2D FR which we provide by averaging over one dimension and estimate that variations lost in this average are smaller than 7%.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗
DOE OSTI2020

Toward Large-Scale Image Segmentation on Summit

Semantic segmentation of images is an important computer vision task that emerges in a variety of application domains such as medical imaging, robotic vision and autonomous vehicles to name a few. While these domain-specific image analysis tasks involve relatively small image sizes (~ 10 2 × 10 2 ), there are many applications that need to train machine learning models on image data with extents that are orders of magnitude larger (~10 4 × 10 4 ). Training deep neural network (DNN) models on large extent images is extremely memory-intensive and often exceeds the memory limitations of a single graphical processing unit, a hardware accelerator of choice for computer vision workloads. Here, an efficient, sample parallel approach to train U-Net models on large extent image data sets is presented. Its advantages and limitations are analyzed and near-linear strong-scaling speedup demonstrated on 256 nodes (1536 GPUs) of the Summit supercomputer. Using a single node of the Summit supercomputer, an early evaluation of a recently released model parallel framework called GPipe is demonstrated to deliver ~ 2X speedup in executing a U-Net model with an order of magnitude larger number of trainable parameters than reported before. Performance bottlenecks for pipelined training of U-Net models are identified and mitigation strategies to improve the speedups are discussed. Together, these results open up the possibility of combining both approaches into a unified scalable pipelined and data parallel algorithm to efficiently train U-Net models with very large receptive fields on data sets of ultra-large extent images.

Seal, Sudip↗
DOE OSTI2018

CGYRO

CGYRO is a multi-species spectral gyrokinetic solver with sonic rotation capability. It is an Eulerian GK solver specifically designed and optimized for collisional, electromagnetic, multiscale simulation. A key algorithmic aspect of CGYRO is the radially spectral formulation used to reduce the complicated integral gyroaveraging kernel into a multiplication in wavenumber space, but retaining the ability to treat profile variation important for edge plasmas. This makes it suitable for next-generation computational systems that require high levels of parallel concurrency.

Belli, EmilyA↗
DOE OSTI2020

TChem 2.0

The TChem toolkit is an object-oriented C++ software library that enables numerical simulations using complex chemistry and facilitates the analysis of detailed kinetic models. The toolkit provides capabilities for thermodynamic properties and gas-phase and surface chemistry. The computations rely on the Kokkos framework for parallel execution on many-core and GPU-based platforms. Sandia National Laboratories is a multimission laboratory managed and operated by National Technology & Engineering Solutions of Sandia, LLC, a wholly owned subsidiary of Honeywell International Inc., for the U.S. Department of Energy's National Nuclear Security Administration under contract DE-NA0003525. SAND2021-2752 O

Kim, Kyungjoo↗
DOE OSTI2021

Theoretically and practically efficient parallel nucleus decomposition

This paper studies the nucleus decomposition problem, which has been shown to be useful in finding dense substructures in graphs. We present a novel parallel algorithm that is efficient both in theory and in practice. Our algorithm achieves a work complexity matching the best sequential algorithm while also having low depth (parallel running time), which significantly improves upon the only existing parallel nucleus decomposition algorithm (Sariyüce et al. , PVLDB 2018). The key to the theoretical efficiency of our algorithm is a new lemma that bounds the amount of work done when peeling cliques from the graph, combined with the use of a theoretically-efficient parallel algorithms for clique listing and bucketing. We introduce several new practical optimizations, including a new multi-level hash table structure to store information on cliques space-efficiently and a technique for traversing this structure cache-efficiently. On a 30-core machine with two-way hyper-threading on real-world graphs, we achieve up to a 55x speedup over the state-of-the-art parallel nucleus decomposition algorithm by Sariyüce et al. , and up to a 40x self-relative parallel speedup. We are able to efficiently compute larger nucleus decompositions than prior work on several million-scale graphs for the first time.

Computer Science↗
DOE OSTI2019

Center for Computing Research Highlights

Sandia has a legacy of leadership in the advancement of high performance computing (HPC) at extreme scales. First-of-a-kind scalable distributed-memory parallel platforms such as the Intel Paragon, ASCI Red (the world’s first teraflops computer), and Red Storm (co-developed with Cray) helped form the basis for one of the most successful supercomputer product lines ever: the Cray XT series. Sandia also has pioneered system software elements—including lightweight operating systems, the Portals network programming interface, advanced interconnection network designs, and scalable I/O— that are critical to achieving scalability on large computing systems.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2016

Model Reduction for Compressible Cavity Simulations Towards Uncertainty Quantification of Structural Loading

This report summarizes FY16 progress towards enabling uncertainty quantification for compressible cavity simulations using model order reduction (MOR). The targeted application is the quantification of the captive-carry environment for the design and qualification of nuclear weapons systems. To accurately simulate this scenario, Large Eddy Simulations (LES) require very fine meshes and long run times, which lead to week-long runs even on parallel state-of-the-art super- computers. MOR can reduce substantially the CPU-time requirement for these simulations. We describe two approaches for model order reduction for nonlinear systems, which can yield significant speed-ups when combined with hyper-reduction: the Proper Orthogonal Decomposition (POD)/Galerkin approach and the POD/Least-Squares Petrov Galerkin (LSPG) approach. The implementation of these methods within the in-house compressible flow solver SPARC is discussed. Next, a method for stabilizing and enhancing low-dimensional reduced bases that was developed as a part of this project is detailed. This approach is based on a premise termed "minimal subspace rotation", and has the advantage of yielding ROMs that are more stable and accurate for long-time compressible cavity simulations. Numerical results for some laminar cavity problems aimed at gauging the viability of the proposed model reduction methodologies are presented and discussed.

42 ENGINEERING↗
DOE OSTI2019

Breaking the Bottleneck of Genomes: Understanding Gene Function Across Taxa Workshop Report

In the last few decades, high-throughput technologies using various “omics” have enabled unprecedented views of biological systems at the molecular level. In parallel, the integration of omic datasets using computational modeling has provided new understanding of biological processes in organisms relevant to the U.S. Department of Energy’s (DOE) missions in energy and the environment. Collectively, these developments have spearheaded the advancement of systems biology, which can be viewed as a holistic approach for deciphering the complexity of biological systems. However, as high-throughput omic technologies and integrative systems biology efforts have improved our understanding of some biological systems, analyzing and finding meaningful answers within these massive datasets remain extremely challenging, in large part due to the lack of fundamental knowledge of gene function. Indeed, all sequenced genomes, both microbes and plants, contain large numbers of genes of “unknown function” that significantly limit scientists’ ability to model, predict, and engineer organisms with enhanced functions relevant to DOE. Current methodologies can be employed to decipher gene function, but they are typically slow, laborious, inefficient, and not scalable. This “bottleneck” in genome understanding could be broken with new, innovative, and transformative experimental tools, datasets, and computation that can define gene function on a massive and high-throughput scale compatible with the pace of DNA sequencing. In light of this grand challenge, DOE’s Office of Biological and Environmental Research (BER) convened the Breaking the Bottleneck of Genomes: Understanding Gene Function Across Taxa workshop on November 1–2, 2018. This workshop brought together leaders in microbiology, plant sciences, technology, and computation, who collectively identified the experimental and data analysis gaps preventing large-scale gene function determination as well as opportunities for overcoming these gaps.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2025

Toward a persistent event-streaming system for high-performance computing applications

High-performance computing (HPC) applications have traditionally relied on parallel file systems and file transfer services to manage data movement and storage. Alternative approaches have been proposed that use direct communications between application components, trading persistence and fault tolerance for speed. Event-driven architectures, as popularized in enterprise contexts, present a compelling middle ground, avoiding the performance cost and API constraints of parallel file systems while retaining persistence and offering impedance matching between application components. However, adapting streaming frameworks to HPC workloads requires addressing challenges unique to HPC systems. This paper investigates the potential for a streaming framework designed for HPC infrastructures and use cases. We introduce Mofka, a persistent event-streaming framework designed specifically for HPC environments. Mofka combines the capabilities of a traditional streaming service with optimizations tailored to the HPC context, such as support for massively multicore nodes, efficient scaling for large producer-consumer workflows, RDMA-enabled high-performance network communications, specialized network fabrics with multiple links per node, and efficient handling of large scientific data payloads. Built using the Mochi suite of HPC data service components, Mofka provides a lightweight, modular, and high-performance solution for persistent streaming in HPC systems. We present the architecture of Mofka and evaluate its performance against Kafka and Redpanda using benchmarks on diverse platforms, including Argonne's Polaris and Oak Ridge's Frontier supercomputers, showing up to 8× improvement in throughput in some scenarios. We then demonstrate its utility in several real-world applications: a tomographic reconstruction pipeline, a workflow for the discovery of metal-organic frameworks for carbon capture, and the instrumentation of Dask workflows for provenance tracking and performance analysis.

HPC↗
DOE OSTI2023

BOXKIT

SF-23-067 BoxKit is a library that provides building blocks to parallelize and scale data science, high performance computing, and machine learning applications for block-structured datasets. Spatial data from simulations and experiments can be accessed and managed using tools available in this library when working with more data analysis oriented packages like SciKit (https://github.com/scikit-learn/scikit-learn) and FlowNet (https://github.com/NVIDIA/flownet2-pytorch)

DHRUV, AKASH↗