Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “parallel and distributed computing”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 379 records · Page 21
All sourcesNASA NTRSDOE OSTI
DOE OSTI2023

Distributed memory, GPU accelerated Fock construction for hybrid, Gaussian basis density functional theory

With the growing reliance of modern supercomputers on accelerator-based architecture such a graphics processing units (GPUs), the development and optimization of electronic structure methods to exploit these massively parallel resources has become a recent priority. While significant strides have been made in the development GPU accelerated, distributed memory algorithms for many modern electronic structure methods, the primary focus of GPU development for Gaussian basis atomic orbital methods has been for shared memory systems with only a handful of examples pursing massive parallelism. Here in this work, we present a set of distributed memory algorithms for the evaluation of the Coulomb and exact exchange matrices for hybrid Kohn–Sham DFT with Gaussian basis sets via direct density-fitted (DF-J-Engine) and seminumerical (sn-K) methods, respectively. The absolute performance and strong scalability of the developed methods are demonstrated on systems ranging from a few hundred to over one thousand atoms using up to 128 NVIDIA A100 GPUs on the Perlmutter supercomputer.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2020

The DEAL.II finite element library: Design, features, and insights

DEAL.II is a state-of-the-art finite element library focused on generality, dimension-independent programming, parallelism, and extensibility. In this paper, we outline its primary design considerations and its sophisticated features such as distributed meshes, h p -adaptivity, support for complex geometries, and matrix-free algorithms. But DEAL.II is more than just a software library: It is also a diverse and worldwide community of developers and users, as well as an educational platform. We therefore also discuss some of the technical and social challenges and lessons learned in running a large community software project over the course of two decades.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2023

Massively-parallel Lagrangian particle code and applications

Massively-parallel, distributed-memory algorithms for the Lagrangian particle hydrodynamic method (Samulyak et al., 2018) have been developed, verified, and implemented. The key component of parallel algorithms is a particle management module that includes a parallel construction of octree databases, dynamic adaptation and refinement of octrees, and particle migration between parallel subdomains. The particle management module is based on the p4est (parallel forest of k-trees) library. The massively-parallel Lagrangian particle code has been applied to a variety of fundamental science and applied problems. A summary of Lagrangian particle code applications to the injection of impurities into thermonuclear fusion devices and to the simulation of supersonic hydrogen jets in support of laser-plasma wakefield acceleration research has also been presented.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2026

Tula: Optimizing Time, Cost, and Generalization in Distributed Large-Batch Training

Distributed training increases the number of batches processed per iteration either by scaling-out (adding more nodes) or scaling-up (increasing the batch-size). However, the largest configuration does not necessarily yield the best performance. Horizontal scaling introduces additional communication overhead, while vertical scaling is constrained by computation cost and device memory limits. Thus, simply increasing the batch-size leads to diminishing returns: training time and cost decrease initially but eventually plateaus, creating a knee-point in the time/cost vs. batch-size pareto curve. The optimal batch-size therefore depends on the underlying model, data and available compute resources. Large batches also suffer from worse model quality due to the well-known “generalization gap”. In this paper, we present Tula, an online service that automatically optimizes time, cost, and convergence quality for large-batch training of convolutional models. It combines parallel-systems modeling with statistical performance prediction to identify the optimal batchsize. Tula predicts training time and cost within 7.5−14% error across multiple models, and achieves up to 20× overall speedup and improves test accuracy by ≈9% on average over standard large-batch training on various vision tasks, thus successfully mitigating the generalization gap and accelerating training at the same time.

Tyagi, Sahil [ORNL] (ORCID:0009000783144745)↗
DOE OSTI2023

Towards Lightweight Data Integration Using Multi-Workflow Provenance and Data Observability

Modern large-scale scientific discovery requires multidisciplinary collaboration across diverse computing facilities, including High Performance Computing (HPC) machines and the Edge-to-Cloud continuum. Integrated data analysis plays a crucial role in scientific discovery, especially in the current AI era, by enabling Responsible AI development, FAIR, Reproducibility, and User Steering. However, the heterogeneous nature of science poses challenges such as dealing with multiple supporting tools, cross-facility environments, and efficient HPC execution. Building on data observability, adapter system design, and provenance, we propose MIDA: an approach for lightweight runtime Multi-workflow Integrated Data Analysis. MIDA defines data observability strategies and adaptability methods for various parallel systems and machine learning tools. With observability, it intercepts the dataflows in the background without requiring instrumentation while integrating domain, provenance, and telemetry data at runtime into a unified database ready for user steering queries. We conduct experiments showing end-to-end multi-workflow analysis integrating data from Dask and MLFlow in a real distributed deep learning use case for materials science that runs on multiple environments with up to 276 GPUs in parallel. We show near-zero overhead running up to 100,000 tasks on 1,680 CPU cores on the Summit supercomputer.

Santos Souza, Renan↗
DOE OSTI2020

2020 IEEE PES Innovative Smart Grid Technologies Europe (ISGT-Europe)

Recent proliferation of distributed energy sources in distribution or sub-transmission systems necessitates close monitoring of these three-phase power grids which typically operate under unbalanced loading conditions. Unlike the transmission systems where the network equations are commonly based on the positive sequence component models, a detailed three phase model will have to be used in implementing network applications for these systems. In the specific case of the state estimator, where measurement and parameter errors may bias the solution, bad data and parameter error detection algorithms should also be incorporated. Implementing the state estimator and error detection algorithms for three-phase systems impose additional computational burden and modifications to the state estimation code. This paper proposes a practical solution to avoid these issues by using synchronized phasor measurements and modal decoupling. The previously developed parameter error detection algorithm based on the normalized Lagrange multipliers (NLM) test is applied to the measurements independently in each mode in parallel, not only saving CPU time but also avoiding new code development for a three-phase estimator. Different parameter error scenarios are created and tested to verify the effectiveness of the proposed error detection approach.

Khalili, Ramtin↗
DOE OSTI2022

Accelerating Simulation for High-Fidelity PV Inverter System Reliability Assessment with High-Performance Computing

The overall cost of photovoltaic (PV) systems has shown a downward trend during the last decade; however, PV inverter failures account for the highest cost of operation and maintenance. To address this, reliability tools with powerful computation and better accuracy are required for the lifetime prediction and degradation evaluation of PV inverters. This paper proposes an event-driven parallel computing-based simulator. The proposed simulator applies high-performance computing techniques and other accessory optimization techniques-including cluster merging, adaptive model updates, and steady-state identification-to make reliability assessments for PV inverters under given input mission profiles and operating conditions with high efficiency and high fidelity. The main idea of the simulator and its workflow are introduced. Then, a demo PV inverter system simulator is implemented, and the speedup of the total simulations of the switching model reaches 123.03 times.

high-performance computing↗
DOE OSTI2025

Computational Modeling of Graphite Degradation due to Molten Salt Infiltration and Wear

Molten-salt reactors (MSRs) represent a promising next-generation reactor design, with graphite serving as a moderator and/or reflector in several designs. However, due to limited experimental data and operational experience, a technical understanding of the structural integrity of graphite in molten salt environments remains incomplete. This report presents a modeling-based evaluation of graphite degradation in MSR environments, focusing on the effects of salt infiltration in fuel salt-based designs and surface wear in pebble bed reactor designs. The objective of this study is to enhance understanding of the structural integrity challenges posed by these degradation mechanisms and to provide a framework for assessing graphite behavior in MSRs. The first part of the report investigates the phenomenon of molten salt infiltration into graphite. This infiltration occurs when molten salt permeates the interconnected pore structure of the graphite moderator, driven by factors such as pressure differentials and the physical properties of both the salt and graphite. The infiltration process is influenced by characteristics of the pore structure, viscosity of the molten salt, and the interfacial energies between the graphite, salt, and the atmosphere within the graphite pore. Utilizing a coupled multiphysics modeling approach with Grizzly software, the study evaluates the stress induced by internal heat sources due to infiltration, which can lead to structural concerns. This evaluation is crucial for understanding how infiltration affects the mechanical integrity of graphite components in MSRs. The study considers the Molten-Salt Reactor Experiment (MSRE) graphite stringer geometry due to the availability of relevant data. Through detailed finite element analysis, the study examines stress distributions at varying infiltration percentages, revealing that stress levels increase with higher amounts of infiltration. Rare-event simulations, using the parallel subset simulation (PSS) framework, further quantify the failure probabilities under input uncertainties, with a user-specified failure metric. The PSS framework also identifies critical input parameters that significantly affect the stress values, including infiltration amount, thermal conductivity, and power density. Additionally, considering realistic reactor scenarios, the analysis was performed to account for the combined effects of radiation and infiltration, and modeling strategies on how to analyze new reactor designs or new graphite grades are discussed. The second part of the report focuses on wear mechanisms in pebble bed-based MSRs. As graphite fuel pebbles interact with the graphite reflector block, wear can result in material loss and the formation of surface defects, which may act as stress concentrators. A similar multiphysics modeling framework is employed to assess the impact of wear on the structural integrity of graphite components. This study considers a generic fluoride-cooled high-temperature reactor (gFHR) design due to the availability of comprehensive data. Worst-case scenario dimensions of the reflector blocks were analyzed under thermal and radiation conditions. Subsequently, wear in the form of idealized pits and grooves is modeled on the inner surface of the graphite block, with the maximum stress from previous simulations. The simulations show that groove-type defects are more detrimental than pits, leading to higher stress concentrations. Considering worst-case simulation scenarios and experimental wear rates, it was determined that the formation of a surface defect critical enough to affect the stress may not be possible in a gFHR design. Overall, the findings of this research contribute to the development of robust modeling tools for predicting graphite behavior under various operational conditions in MSRs.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗
DOE OSTI2020

Machine Committee Framework for Power Grid Disturbances Analysis Using Synchrophasors Data

Events detection is a key challenge in power grid frequency disturbances analysis. Accurate detection of events is crucial for situational awareness of the power system. In this paper, we study the problem of events detection in power grid frequency disturbance analysis using synchrophasors data streams. Current events detection approaches for power grid rely on individual detection algorithm. This study integrates some of the existing detection algorithms using the concept of machine committee to develop improved detection approaches for grid disturbance analysis. Specifically, we propose two algorithms—an Event Detection Machine Committee (EDMC) algorithm and a Change-Point Detection Machine Committee (CPDMC) algorithm. Both algorithms use parallel architecture to fuse detection knowledge of its individual methods to arrive at an overall output. The EDMC algorithm combines five individual event detection methods, while the CPDMC algorithm combines two change-point detection methods. Each method performs the detection task separately. The overall output of each algorithm is then computed using a voting strategy. The proposed algorithms are evaluated using three case studies of actual power grid disturbances. Compared with the individual results of the various detection methods, we found that the EDMC algorithm is a better fit for analyzing synchrophasors data; it improves the detection accuracy; and it is suitable for practical scenarios.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2024

Modeling transient edge plasma transport with dynamic recycling

The work presents numerical simulation studies of the role that dynamic plasma recycling on the main wall and divertor target surfaces plays in transient edge plasma transport phenomena, such as edge localized modes (ELMs). The studies are performed by coupling the edge plasma transport code UEDGE [Rognlien et al., J. Nucl. Mater. 196–198, 347 (1992)] and the wall reaction–diffusion transport code FACE [Smirnov et al., Fusion Sci. Technol. 71, 75 (2017)]. The two-dimensional, time-dependent, two-way coupling of the codes, in a realistic tokamak geometry, is accomplished using the Integrated Plasma Simulator framework [Elwasif et al., in 18th Euromicro Conference on Parallel, Distributed and Network-Based Processing (PDP 2010), Pisa, Italy (IEEE, 2010), pp. 419–427] for all modeled material plasma boundaries. The simulations show that dynamic plasma recycling has substantially different characteristics on the main wall and on the divertor plates. It is demonstrated that during an ELM cycle the outer wall can dynamically absorb and release a number of particles comparable to that expelled by the ELM from the core plasma, by far exceeding the dynamic retention capacity of the divertor surfaces. The resulting evolution of the edge and divertor plasma conditions during an ELM cycle is analyzed.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗
DOE OSTI2023

Learning local equivariant representations for large-scale atomistic dynamics

Abstract A simultaneously accurate and computationally efficient parametrization of the potential energy surface of molecules and materials is a long-standing goal in the natural sciences. While atom-centered message passing neural networks (MPNNs) have shown remarkable accuracy, their information propagation has limited the accessible length-scales. Local methods, conversely, scale to large simulations but have suffered from inferior accuracy. This work introduces Allegro, a strictly local equivariant deep neural network interatomic potential architecture that simultaneously exhibits excellent accuracy and scalability. Allegro represents a many-body potential using iterated tensor products of learned equivariant representations without atom-centered message passing. Allegro obtains improvements over state-of-the-art methods on QM9 and revMD17. A single tensor product layer outperforms existing deep MPNNs and transformers on QM9. Furthermore, Allegro displays remarkable generalization to out-of-distribution data. Molecular simulations using Allegro recover structural and kinetic properties of an amorphous electrolyte in excellent agreement with ab-initio simulations. Finally, we demonstrate parallelization with a simulation of 100 million atoms.

74 ATOMIC AND MOLECULAR PHYSICS↗
DOE OSTI2025

Snow ALbedo eVOlution (SALVO) Campaign Spectral Albedo and Related Measurements from April - June, 2024 in Utqiagivk, AK

A field-portable spectroradiometer, referred to herein as an ‘ASD’, was used to make spatially-distributed spectral albedo (350 – 2500 nm) measurements on tundra and sea ice surfaces. The ASD detector is carried in a backpack and controlled via a computer mounted on the front of the operator (see Figure 1). The ASD measures the spectral irradiance from a fiber optic cable that is routed from the backpack to a custom, gooseneck cosine collector mounted on the end of a 1-m long boom (Grenfell and Perovich, 2008). The boom was held at hip height (approximately 1 m) and had an integrated bubble level for levelling. To make an albedo measurement, first the operator collect an incident (down-welling) irradiance, followed by a reflected (up-welling) measurement. The time between incident and reflected measurements was typically between 11 and 26 seconds (interquartile range). For each measurement, 10 spectra are averaged together. Albedo is calculated as the ratio of the reflected to incident measurement, which obviates the need for absolute radiometric calibration. Albedo measurements were taken parallel to the 200-m albedo lines at 5-m increments (41 measurements) ~1 m south of the line. While the ASD operator was making measurements, an assistant kept notes on the scan number associated with each measurement, the surface type (see below), and collected photos of each measurement (see companion oblique photos data archive). Measurements were made within 3 hours of solar noon.

ASD Spectroradiometer↗
DOE OSTI2021

Transient cycling of nitrogen, organic carbon and oxygen within the free-flowing Columbia River corridor: Linking exposure time dependent biogeochemical reactions to river stage fluctuations (Final project report)

The objective of this project was to develop next-generation techniques for representing the transformations of complex reaction networks like those impacting transient river-corridors, then apply them to develop upscaling tools. The project contributed significantly to the development of Lagrangian “mass transfer particle tracking” (MTPT) tools that are the first numerical framework that explicitly separates mixing and spreading processes; a crucial distinction for accurate representations of reaction rates across scales. The MTPT approach uses an operator splitting scheme where physical transport processes are simulated using classical random walk methods and each particle is treated as a “container” that carries with it any number of chemical species. Mixing between particles is simulated using a colocation-probability based mass transfer kernel and reactions are evaluated on each particle after mixing. The method is stable, accurate, and also allows the explicit modeling of residence time distributions of the individual containers of mass. The MTPT scheme allowed us to explore several research questions related to reactive transport in river corridors and complex reaction networks. We found that it is possible to quantify the exposure time distributions (ETDs) of reactants and that this does suggest some pathways to upscaling. However, comparable approximations can be obtained using simplified (reduced dimensionality) MTPT simulations in less time with similar uncertainty, so development of extensive ETD-based methods was not productive. The major findings are that i) the MTPT schemes are robust and highly accurate across a range of arbitrarily complex reactions, ii) the methods can be efficiently parallelized and the parallel performance characteristics of MTPT are predictable, and iii) the MTPT tools allow simultaneous tracking of residence time. The development of these research tools into efficient software packages continues. The methods are already available to the community because open-source, working examples have been included with all publications.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2020

¬¬Integration of Quantification of Margins and Uncertainties Methodology into Parallel Discrete Event Simulator Framework

Parallel Discrete Events Simulation (PDES) is becoming increasingly important to lab efforts in security and intelligence. It is used to model complex asynchronous systems such as computer networks, satellite systems, vehicular traffic, and human performance. Uncertainty Quantification (UQ) techniques have become a mainstay of Verification and Validation (V&V) efforts on physics simulations. However, PDES models are very different from traditional physics simulations, and research into UQ techniques for PDES is in its infancy. It is not clear which traditional UQ techniques can be applied to PDES, or what new techniques will need to be developed. The goal of this project was to identify existing UQ techniques that can be applied to PDES, and to develop new techniques as necessary. The project implemented or developed techniques to handle issues that do not appear in traditional physics simulations, but are common among PDES, including sampling techniques for high-dimensional homogenous inputs, response surfaces for high-variance heteroskedastic output, and characterization of skewed output distributions. These are foundational UQ techniques that must be used in any complete UQ analysis (Tong 2018).

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2024

Parallel Simulation of Quantum Networks with Distributed Quantum State Management

Quantum network simulators offer the opportunity to cost-efficiently investigate potential avenues for building networks that scale with the number of users, communication distance, and application demands by simulating alternative hardware designs and control protocols. Several quantum network simulators have been recently developed with these goals in mind. As the size of the simulated networks increases, however, sequential execution becomes time-consuming. Parallel execution presents a suitable method for scalable simulations of large-scale quantum networks, but the unique attributes of quantum information create unexpected challenges. In this work, we identify requirements for parallel simulation of quantum networks and develop the first parallel discrete-event quantum network simulator by modifying the existing serial simulator SeQUeNCe. Our contributions include the design and development of a quantum state manager (QSM) that maintains shared quantum information distributed across multiple processes. We also optimize our parallel code by minimizing the overhead of the QSM and decreasing the amount of synchronization needed among processes. Using these techniques, we observe a speedup of 2 to 25 times when simulating a 1,024-node linear network topology using 2 to 128 processes. We also observe an efficiency greater than 0.5 for up to 32 processes in a linear network topology of the same size and with the same workload. We repeat this evaluation with a randomized workload on a caveman network. We also introduce several methods for partitioning networks by mapping them to different parallel simulation processes. We have released the parallel SeQUeNCe simulator as an open source tool alongside the existing sequential version.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2024

Regional-scale fault-to-structure earthquake simulations with the EQSIM framework: Workflow maturation and computational performance on GPU-accelerated exascale platforms

Continuous advancements in scientific and engineering understanding of earthquake phenomena, combined with the associated development of representative physics-based models, is providing a foundation for high-performance, fault-to-structure earthquake simulations. However, regional-scale applications of high-performance models have been challenged by the computational requirements at the resolutions required for engineering risk assessments. The EarthQuake SIMulation (EQSIM) framework, a software application development under the US Department of Energy (DOE) Exascale Computing Project, is focused on overcoming the existing computational barriers and enabling routine regional-scale simulations at resolutions relevant to a breadth of engineered systems. This multidisciplinary software development—drawing upon expertise in geophysics, engineering, applied math and computer science—is preparing the advanced computational workflow necessary to fully exploit the DOE’s exaflop computer platforms coming online in the 2023 to 2024 timeframe. Achievement of the computational performance required for high-resolution regional models containing upward of hundreds of billions to trillions of model grid points requires numerical efficiency in every phase of a regional simulation. This includes run time start-up and regional model generation, effective distribution of the computational workload across thousands of computer nodes, efficient coupling of regional geophysics and local engineering models, and application-tailored highly efficient transfer, storage, and interrogation of very large volumes of simulation data. This article summarizes the most recent advancements and refinements incorporated in the workflow design for the EQSIM integrated fault-to-structure framework, which are based on extensive numerical testing across multiple graphics processing unit (GPU)-accelerated platforms, and demonstrates the computational performance achieved on the world’s first exaflop computer platform through representative regional-scale earthquake simulations for the San Francisco Bay Area in California, USA.

58 GEOSCIENCES↗
DOE OSTI2020

High-Resolution Comonomer Sequencing of Blocky Brominated Syndiotactic Polystyrene Copolymers Using 13C NMR Spectroscopy and Computer Simulations

This work demonstrates the first high-resolution comonomer sequencing of Blocky brominated syndiotactic polystyrene (sPS-co-sPS-Br) copolymers based on pentad assign- ments of the quaternary carbon region of the nuclear magnetic resonance spectrum. Copolymers containing p-bromostyrene (Br-Sty) units were prepared in matched sets using postpolymerization bromination methods carried out in the heterogeneous gel state (Blocky) and homogeneous solution state (Random). Quantitative information from the quaternary carbon spectra, heteronuclear multiple bond correlation spectroscopy, electronic structure calculations, and simulated statistically random copolymers was correlated to confirm the carbon resonance assignments for all 20 possible pentad comonomer sequences. Using the experimental pentad sequence prevalences, a computer code was developed to simulate chains with microstructures typical of each sample as a means to visually represent the copolymer blockiness with quantitative precision. Based on the microstructure and distribution of run lengths in these chains, the simulations revealed that the Blocky copolymers contain a high degree of blockiness. By comparing the run lengths in the simulated chains to the average number of styrene units in a crystalline segment of sPS (found by small-angle X-ray scattering), copolymer crystallizability was predicted. For the simulated Blocky B-21% (21 mol % Br-Sty) chain, the probability of randomly selecting a styrene unit in a crystallizable block was 25.8%, while that in the simulated Random R-18% was zero, in excellent agreement with the experimental crystallization behavior measured by differential scanning calorimetry. Additionally, these predictions confirmed that the simulated chains accurately represent the ensemble of chains in their respective copolymer samples. Furthermore, each simulated Blocky chain contained one or more long sPS blocks that paralleled the measured 38-40 styrene units spanning a crystalline segment within the sPS/CCl4 gel. This finding affirmed that the long sPS segments originated from the precise lamellar structure within the heterogeneous gel morphology (i.e., block length is correlated with lamellar thickness). Overall, the ability to tailor the copolymer microstructure through control of the semicrystalline gel morphology opens the door to synthesizing ordered copolymers by postpolymerization functionalization processes with unprecedented levels of compositional control.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2022

Ab-Initio Investigation of Finite Size Effects in Rutile Titania Nanoparticles with Semilocal and Nonlocal Density Functionals

In this work, we employ hybrid and generalized gradient approximation (GGA) level density functional theory (DFT) calculations to investigate the convergence of surface properties and electronic gap of rutile titania nanoparticles with particle size. The surface energies and electronic gaps are calculated for cuboidal particles with minimum dimension ranging from 3.7 Angstrom (24 atoms) to 10.3 Angstrom (384 atoms) using a highly-parallel real-space DFT code to enable hybrid level DFT calculations of larger nanoparticles than are typically practical. We deconvolute the geometric and electronic finite size effects in surface energy, and evaluate the influence of defects on electronic gap and density of states (DOS). The electronic finite size effects in surface energy vanish when the minimum length scale of the nanoparticles becomes greater than 10 Angstrom. We show that this length scale is consistent with a computationally efficient numerical analysis of the characteristic length scale of electronic interactions. The surface energy of nanoparticles having minimum dimension beyond this characteristic length can be approximated using slab calculations that account for the geometric defects. In contrast, the finite size effects on the electronic gap and DOS is highly dependent on the shape and size of these particles. Furthermore, the DOS for cuboidal particles and more realistic particles constructed using the Wulff algorithm reveal that defect states within the electronic gap play a key role in determining the eigen value distribution of nanoparticles and the electronic gap does not converge to the bulk limit for the particle sizes investigated.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗