SEARCH · Engineering Papers

Results for “distributed parallelization”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 253 records · Page 14

All sources NASA NTRS DOE OSTI

DOE OSTI2025

DIMPLES: Distributed Influence Maximization for Pandemic pLanning on Exascale Systems

We study exascale parallel algorithms for the selection of intervention or monitoring strategies in massive realistic socio-technical networks through scalable Influence Maximization (InfMax) algorithms. We employ novel techniques to enable efficient scaling on up to 8k nodes of OLCF Frontier, with 65k AMD GPUs and 458k AMD CPU cores. Current state-of-the-art InfMax tools are limited to networks with only a few million actors (vertices) and a few hundred million interactions (edges). By overcoming these limitations, we show that our approach is capable of processing a realistic social contact network of the United States with 285 million nodes and about 8 billion edges. This two orders-of-magnitude improvement over the previous state-of-the-art is obtained by leveraging algorithmic advancements for the InfMax problem and designing several problem-specific approaches to overlap communication with computation, improve GPU efficiency, and lower the application’s memory requirements. We evaluate strong scaling for computing 10k most influential seeds using up to 8k nodes of an exascale system, and weak scaling from 128 to 8k system nodes for seed sets ranging from 625 to 40k seeds. We achieve the fastest-known runtime of 25 minutes while performing 48 million diffusion simulations totaling 2.31 petabytes to identify 40k influential seeds using 8k nodes, and take 5.75 minutes to identify 10k seeds while using 4k nodes.

Minutoli, Marco [Pacific Northwest National Labora↗

DOE OSTI2025

New Results on Communication- and Memory-Aware Load Balancing Model and Algorithms

While load balancing in distributed-memory computing has been well-studied, we present an innovative approach to this problem: a unified, reduced-order model that combines three key components to describe “work” in a distributed system: computation, communication, and memory. Our model enables an optimizer to explore complex tradeoffs in task placement, such as augmented parallelism, at the expense of data replication increasing memory usage. We propose a fully distributed, heuristic-based load balancing optimization algorithm, and demonstrate that it quickly finds close-to-optimal solutions. We formalize the complex optimization problem as a mixed-integer linear program, and compare it to our strategy. Finally, we show that when applied to an electromagnetics code, our approach obtains up to 2.3x speedups for the imbalanced execution.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2020

Method of making a distributed optical fiber sensor having enhanced Rayleigh scattering and enhanced temperature stability, and monitoring systems employing same

A method of making an optical fiber sensor device for distributed sensing includes generating a laser beam comprising a plurality of ultrafast pulses, and focusing the laser beam into a core of an optical fiber to form a nanograting structure within the core, wherein the nanograting structure includes a plurality of spaced nanograting elements each extending substantially parallel to a longitudinal axis of optical fiber. Also, an optical fiber sensor device for distributed sensing includes an optical fiber having a longitudinal axis, a core, and a nanograting structure within the core, wherein the nanograting structure includes a plurality of spaced nanograting elements each extending substantially parallel to the longitudinal axis of the optical fiber. Also, a distributed sensing method and system and an energy production system that employs such an optical fiber sensor device.

42 ENGINEERING↗

DOE OSTI2022

Method of making a distributed optical fiber sensor having enhanced Rayleigh scattering and enhanced temperature stability, and monitoring systems employing same

Chen, Peng Kevin↗

DOE OSTI2024

Massively scalable workflows for quantum chemistry: BigChem and ChemCloud

Electronic structure theory, i.e., quantum chemistry, is the fundamental building block for many problems in computational chemistry. Here we present a new distributed computing framework (BigChem), which allows for an efficient solution of many quantum chemistry problems in parallel. BigChem is designed to be easily composable and leverages industry-standard middleware (e.g., Celery, RabbitMQ, and Redis) for distributed approaches to large scale problems. BigChem can harness any collection of worker nodes, including ones on cloud providers (such as AWS or Azure), local clusters, or supercomputer centers (and any mixture of these). BigChem builds upon MolSSI packages, such as QCEngine to standardize the operation of numerous computational chemistry programs, demonstrated here with Psi4, xtb, geomeTRIC, and TeraChem. BigChem delivers full utilization of compute resources at scale, offers a programable canvas for designing sophisticated quantum chemistry workflows, and is fault tolerant to node failures and network disruptions. We demonstrate linear scalability of BigChem running computational chemistry workloads on up to 125 GPUs. Finally, we present ChemCloud, a web API to BigChem and successor to TeraChem Cloud. ChemCloud delivers scalable and secure access to BigChem over the Internet.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

DOE OSTI2020

Mitigating Cascading Outages in Severe Weather Using Simulation-Based Optimization

Severe weather events can trigger cascading power outages and lead to significant losses. In this work, we investigate cascading outage mitigation under severe weather conditions. Given day-ahead weather forecasts and component failure models, we aim to identify a set of power lines that can be hardened to minimize the expected impact of potential cascading outages. Since the expected load shedding cannot be expressed as an explicit function of line hardening decisions and system states, we developed a cascading outage simulator to estimate the expected value of load shedding under various initial weather-related disruption scenarios generated using a weather forecast. To avoid massive enumeration of all possible combinations of line hardening decisions and reduce the simulation efforts, we employed an efficient simulation-based optimization approach that quickly identifies the (near) optimal line hardening decisions in the presence of both large simulation noises due to the highly variable initial disturbances and system states, and significant randomness in the subsequent cascades. Furthermore, the algorithm is also able to utilize parallel computing to dramatically reduce computation time to support decision making in preparation for severe weather conditions. We performed a case study on the Northeast Power Coordinating Council (NPCC) 140-bus system model to demonstrate that our approach can significantly improve power grid resilience to adverse weather events.

24 POWER TRANSMISSION AND DISTRIBUTION↗

DOE OSTI2025

In situ multi-tier auto-ignition detection applied to dual-fuel combustion simulations

Here we use an anomaly detection methodology that is centered on analyzing fourth-order joint moments (co-kurtosis), particularly focusing on its application in auto-ignition of combustion problems with large numbers of species. Unsupervised anomaly detection is challenging to generalize across problem types and domains. A recent technique, centered on analyzing information in the fourth-order joint moment co-kurtosis, has shown promise, especially for high-dimensional scientific data. In this work we present developments to the co-kurtosis based anomaly detection method needed to make it effective and scalable for large-scale distributed scientific data, such as those generated by massively parallel simulations. An in situ co-kurtosis algorithm is employed as the anomaly detection method for identifying ignition kernels in simulations of turbulent combustion. Here, we extend an existing methodology which identifies regions of the domain where anomalies are present, and add another tier of anomaly detection where the individual samples contributing to the anomaly are identified. We apply this algorithm on-the-fly to a variety of turbulent reacting flow problems and compare it to the widely used (but significantly more expensive) chemical explosive mode analysis (CEMA). We demonstrate the ability of the method to detect and identify the onset of low and high temperature ignition which can be used for computational steering, as chemical and combustion anomalies occur intermittently at spatio-temporal locations unknown a priori. Finally, we apply our lightweight in situ algorithm to an exascale high-fidelity simulation with a total of 2.4 Trillion degrees of freedom, performed using an adaptive mesh refinement solver. Furthermore, through a scalability analysis, we show that the relative computational cost of this in-situ anomaly detection algorithm compared to an iteration of the reacting flow solver is negligible.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2020

Self-sustaining sound in collisionless, high- β plasma

Using analytical theory and hybrid-kinetic numerical simulations, we demonstrate that, in a collisionless plasma, long-wavelength ion-acoustic waves (IAWs) with amplitudes $δn/n_0≳2/β$ (where $β\gg1$ is the ratio of thermal to magnetic pressure) generate sufficient pressure anisotropy to destabilize the plasma to firehose and mirror instabilities. These kinetic instabilities grow rapidly to reduce the pressure anisotropy by pitch-angle scattering and trapping particles, respectively, thereby impeding the maintenance of Landau resonances that enable such waves’ otherwise potent collisionless damping. The result is wave dynamics that evince a weakly collisional plasma: the ion distribution function is near-Maxwellian, the field-parallel flow of heat resembles its Braginskii form (except in regions where large-amplitude magnetic mirrors strongly suppress particle transport), and the relations between various thermodynamic quantities are more ‘fluid-like’ than kinetic. A nonlinear fluctuation–dissipation relation for self-sustaining IAWs is obtained by solving a plasma-kinetic Langevin problem, which demonstrates suppressed damping, enhanced fluctuation levels and weakly collisional thermodynamics when IAWs with $δn/n_0≳2/β$ are stochastically driven. We investigate how our results depend upon the scale separation between the wavelength of the IAW and the Larmor radius of the ions, and discuss briefly their implications for our understanding of turbulence and transport in the solar wind and the intracluster medium of galaxy clusters.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗

DOE OSTI2024

Designing workflows for materials characterization

Experimental science is enabled by the combination of synthesis, imaging, and functional characterization organized into evolving discovery loop. Synthesis of new material is typically followed by a set of characterization steps aiming to provide feedback for optimization or discover fundamental mechanisms. However, the sequence of synthesis and characterization methods and their interpretation, or research workflow, has traditionally been driven by human intuition and is highly domain specific. Here, we explore concepts of scientific workflows that emerge at the interface between theory, characterization, and imaging. In this study, we discuss the criteria by which these workflows can be constructed for special cases of multiresolution structural imaging and functional characterization, as a part of more general material synthesis workflows. Some considerations for theory–experiment workflows are provided. We further pose that the emergence of user facilities and cloud labs disrupts the classical progression from ideation, orchestration, and execution stages of workflow development. To accelerate this transition, we propose the framework for workflow design, including universal hyperlanguages describing laboratory operation, ontological domain matching, reward functions and their integration between domains, and policy development for workflow optimization. These tools will enable knowledge-based workflow optimization; enable lateral instrumental networks, sequential and parallel orchestration of characterization between dissimilar facilities; and empower distributed research.

36 MATERIALS SCIENCE↗

DOE OSTI2021

Structural Simluation Toolkit (SST) v.11.0

The SST provides a parallel framework to perform system simulation of computer architectures to determine their performance and power consumption. Additionally, the SST contains basic models of a computer processor, and interconnect and can connect to an external memory simulator (DRAMSim II). The SST framework provides a simple interface by which other computer simulation models can be combined under a common parallel discrete event-based simulation environment. This allows design exploration of future architectures, analysis of how current computer programs will function on future architectures. The SST provides a parallel discrete event simulation framework, including partitioning and object distribution over MPI. It also provides a mechanism by which components can report their power consumption for analysis.

Rodrigues, ArunF.↗

DOE OSTI2022

Twist-3 partonic distributions from lattice QCD

Twist-3 partonic distributions contain important information that characterizes nucleon’s structure. In this work, we show our lattice exploration of the twist-3 PDFs g_T(x) g T ( x ) , and h_L(x) h L ( x ) . We also present our preliminary results on the twist-3 GPD \tilde{G}_2(x) G ̃ 2 ( x ) . We use the quasi-distribution approach to connect the lattice-extracted matrix elements, renormalized in the RI/MOM scheme, to light-cone distributions, applying the matching procedure that we developed in parallel. We also calculate the twist-2 counterparts of g_T(x) g T ( x ) and h_L(x) h L ( x ) , i.e. g_1(x) g 1 ( x ) , and h_1(x) h 1 ( x ) , and test the Wandzura-Wilczek approximation.

Bhattacharya, Shohini↗

DOE OSTI2024

Scaling the SciDAC QuantOm Workflow

As part of the Scientific Discovery through Advanced Computing (SciDAC) program, the Quantum Chromodynamics Nuclear Tomography (QuantOM) project aims to analyze data from Deep Inelastic Scattering (DIS) experiments conducted at Jefferson Lab and the upcoming Electron Ion Collider. The DIS data analysis is performed on an event-level by combining the input from theoretical and experimental nuclear physics into a single, composable workflow. The optimization itself (I.e. fitting the experimental data with theoretical predictions) is carried out by a machine / deep learning algorithm. The size of the acquired DIS data as well as the complexity of the workflow itself require that the analysis is performed across multiple GPUs on high performance computing systems, such as Polaris at Argonne National Laboratory. This presentation discusses the novelties and challenges that came along with parallelizing this workflow. Recent results are compared to common distributed training techniques.

Lersch, Daniel↗

DOE OSTI2023

Evaluation of pre-training large language models on leadership-class supercomputers

Large language models (LLMs) have arisen rapidly to the center stage of artificial intelligence as the foundation models applicable to many downstream learning tasks. However, how to effectively build, train, and serve such models for many high-stake and first-principle-based scientific use cases are both of great interests and of great challenges. Moreover, pre-training LLMs with billions or even trillions of parameters can be prohibitively expensive not just for academic institutions, but also for well-funded industrial and government labs. Furthermore, the energy cost and the environmental impact of developing LLMs must be kept in mind. Here, in this work, we conduct a first-of-its-kind performance analysis to understand the time and energy cost of pre-training LLMs on the Department of Energy (DOE)’s leadership-class supercomputers. Employing state-of-the-art distributed training techniques, we evaluate the computational performance of various parallelization approaches at scale for a range of model sizes, and establish a projection model for the cost of full training. Our findings provide baseline results, best practices, and heuristics for pre-training such large models that should be valuable to HPC community at large. We also offer insights and optimization strategies for using the first exascale computing system, Frontier, to train models of the size of GPT-3 and beyond.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2025

Parallel derivative-free optimization for simulation-based design of behind-the-meter energy systems

In this work, the integrated design and dispatch of behind-the-meter or distributed resources (e.g. stationary battery storage and solar PV generation) is considered. A simulation-based framework is employed, generating high-fidelity results with closed-loop predictive control at a fine resolution, at the expense of high computational cost (several minutes to a few hours per design point). To address this challenge, parallel derivative-free design methods are considered. Four methods are compared, including state-of-the-art surrogate-based methods (Radial-Basis Functions and Gaussian processes) and sampling strategies, an evolutionary-based method, and a simple sequential grid refinement method. As a case study, two types of design problem with increasing complexity are considered, namely, the design of behind-the-meter resources (three design variables) and the inclusion of grid capacity (four design variables). The second yields a constrained design problem for which violations can only be determined after solving the computationally expensive simulation. For the three-dimensional case, all methods present a good performance, achieving a solution within 1% of the optimum after the first iteration, with the sequential grid refinement exhibiting the fastest convergence and achieving the best final objective value. This indicates that the parallel evaluation of multiple sampling points may be more important than the choice of method for small decision spaces. For the four-dimensional constrained case, the Genetic Algorithm presents the best tradeoff between performance and computational effort, while the rough objective function terrain generated by constraint violation penalties reduces the performance of surrogate-based methods. Contour plots with flat regions indicate flexibility in the optimal design and highlight the importance of characterizing the solution space.

24 POWER TRANSMISSION AND DISTRIBUTION↗

DOE OSTI2026

Mode Multiplexing for Scalable Cavity-Enhanced Operations in Neutral-Atom Arrays

Neutral-atom arrays provide a versatile platform for quantum information processing. However, in large-scale arrays, efficient photon collection remains a bottleneck for key tasks such as fast, nondestructive qubit readout and remote entanglement distribution. We propose a cavity-based approach that enables fast, parallel operations over many atoms using multiple modes of a single optical cavity. By selectively shifting the relevant atomic transitions, each atom can be coupled to a distinct cavity mode, allowing independent simultaneous processing. We present practical system designs that support cavity-mode multiplexing with up to 50 modes, enabling rapid mid-circuit syndrome extraction and significantly enhancing entanglement distribution rates between remote atom arrays. This approach offers a scalable solution to core challenges in neutral-atom arrays, advancing the development of practical quantum technologies.

Aqua, Ziv [Massachusetts Institute of Technology (↗

DOE OSTI2024

Parallel Runtime Interface for Fortran (PRIF): A Multi-Image Solution for LLVM Flang

Fortran compilers that provide support for Fortran’s native parallel features often do so with a runtime library that depends on details of both the compiler implementation and the communication library, while others provide limited or no support at all. This paper introduces a new generalized interface that is both compiler- and runtime-library-agnostic, providing flexibility while fully supporting all of Fortran’s parallel features. The Parallel Runtime Interface for Fortran (PRIF) was developed to be portable across shared- and distributed-memory systems, with varying operating systems, toolchains and architectures. It achieves this by defining a set of Fortran procedures corresponding to each of the parallel features defined in the Fortran standard that may be invoked by a Fortran compiler and implemented by a runtime library. PRIF aims to be used as the solution for LLVM Flang to provide parallel Fortran support. This paper also briefly describes our PRIF prototype implementation: Caffeine.

Bonachea, Dan↗

DOE OSTI2024

Parallel computing for power system climate resiliency: Solving a large-scale stochastic capacity expansion problem with mpi-sppy

Here we propose a nodal stochastic generation and transmission expansion planning model that incorporates the output from high-resolution global climate models through load and generation availability scenarios. We implement our model in Pyomo and perform computational studies on a realistically-sized test case of the California electric grid in a high performance computing environment. We propose model reformulations and algorithm tuning to efficiently solve this large problem using a variant of the Progressive Hedging Algorithm. We utilize the parallelization capabilities and overall versatility of mpi-sppy, exploiting its hub-and-spoke architecture to concurrently obtain inner and outer bounds on an optimal expansion plan. Initial results show that instances with 360 representative days on a system with over 8,000 buses can be solved to within 5% of optimality in under 4 h of wall clock time, a first step towards solving a large-scale power system expansion planning problem across a wide range of climate-informed operational scenarios.

24 POWER TRANSMISSION AND DISTRIBUTION↗

DOE OSTI2025

Initial position optimization in molecular dynamics simulations for a Coulomb system

A new algorithm for molecular dynamics (MD) simulations is developed to optimize plasma particle distributions at given initial temperatures. By combining velocity scaling and reassignment, the method effectively eliminates the initial rise and oscillation in temperatures observed with randomly distributed positions. These rises and oscillations are undesired numerical artifacts observed in conventional plasma MD simulations, arising from unoptimized particle positions. The algorithm demonstrates temperature relaxation without initial rises or oscillations, as well as precise flow velocity relaxation, enabling accurate measurement of relaxation times. The code is accelerated using graphics processing units for parallel processing, enhancing the study of plasma dynamics. The proposed method for distributing physically valid particles in MD simulations enables accurate studies of intrinsic collision processes in plasmas, including the dynamics of strongly coupled plasmas, plasma–wave interactions, and transport phenomena in magnetized plasmas. The paper concludes with a discussion of potential applications and future enhancements to the algorithm.

Jo, Jawon (ORCID:0009000924193285)↗