Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “data movement”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 109 records · Page 6
All sourcesNASA NTRSDOE OSTI
DOE OSTI2024

Quantum/AI Topology-Aware Latency-Adaptive HPC Workflow Scheduling Optimization

The growing demand for more powerful high-performance computing (HPC) systems has led to a steady rise in energy consumption by supercomputing worldwide. This study is focused on comparing our Application-Topology Mapper (ATMapper) to the popular Simple Linux Utility for Resource Management (SLURM) for the purpose of exploring methods that can further optimize job-scheduling within HPC systems. ATMapper is an Artificial-Intelligence based approach to job-scheduling that is currently being enhanced with quantum annealing (QA) to generate optimal schedules faster. We are applying QA to speedup our ATMapper process to achieve higher computing efficiency, thereby reducing HPC energy consumption. Here, we examine how four job-scheduling approaches perform in processor node assignment when using an example network architecture of 4 interconnected nodes. Using a specialized script, we are assessing the schedule of a computation flow with 11 interdependent tasks. The data movements among nodes were tracked to count for the number of interactions (network hops) between nodes needed to complete the tasks. The total number of hops and the job completion time were then used to quantify the efficiency of the different mapping approaches. In addition to SLURM, we also compare our ATMapper to the QA-enabled LBNL TIGER and the D-Wave Distributed Computing processor assignment approaches. The preliminary results showed that our topology-aware, latency-adaptive ATMapper is significantly more efficient when compared to the other scheduling approaches due to its load-imbalance network allocation. The scheduler displayed a computing efficiency of 53% by performing significantly fewer network hops than its alternatives. By reducing the number of hops, ATMapper was able to perform all 11 tasks by using only 3 nodes out of given 4. This research indicates the potential to use QA/AI for HPC job-scheduling. Later, we will test a SLURM simulator program to draw further comparisons on the effectiveness of ATMapper's scheduling approach. The results of this comparison will serve as a baseline for later improving SLURM's performance using a QA-enhanced ATMapper approach.

Caraveo, Braulio [University of Huston - Clear Lak↗
DOE OSTI2024

To Derive or Not to Derive: I/O Libraries Take Charge of Derived Quantities Computation

The ever-increasing volume of data produced by HPC simulations necessitates scalable methods for data exploration and knowledge extraction. Scientific data analysis often involves complex queries across distributed datasets, requiring manipulation of multiple primary variables and generating derived data that needs to be handled efficiently, creating challenges for applications that need to parse many large datasets. Relying on individual applications to handle all intermediate data generally leads to redundant computations across studies and unnecessary data transfers. In this paper, we investigate the performance of different approaches where applications define derived variables as quantities of interest (QoIs) and offload the computation and transfer of these QoIs to the I/O library. This significantly reduces redundancy and optimizes data movement across the distributed storage and processing infrastructure by allowing control over when and where derived variables are computed. We present a detailed analysis of the performance-storage trade-offs associated with different solutions and showcase results for our study on two large-scale datasets created from climate and combustion simulations.

Gainaru, Ana↗
DOE OSTI2022

Scalable Deep Learning-Based Microarchitecture Simulation on GPUs

Cycle-accurate microarchitecture simulators are essential tools for designers to architect, estimate, optimize, and manufacture new processors that meet specific design expectations. However, conventional simulators based on discrete-event methods often require an exceedingly long time-to-solution for the simulation of applications and architectures at full complexity and scale. Given the excitement around wielding the machine learning (ML) hammer to tackle various architecture problems, there have been attempts to employ ML to perform architecture simulations, such as Ithemal and SimNet. However, the direct application of existing ML approaches to architecture simulation may be even slower due to overwhelming memory traffic and stringent sequential computation logic. This work proposes the first graphics processing unit (GPU)-based microarchitecture simulator that fully unleashes the potential of GPUs to accelerate state-of-the-art ML-based simulators. First, considering the application traces are loaded from central processing unit (CPU) to GPU for simulation, we introduce various designs to reduce the data movement cost between CPUs and GPUs. Second, we propose a parallel simulation paradigm that partitions the application trace into sub-traces to simulate them in parallel with rigorous error analysis and effective error correction mechanisms. Combined, this scalable GPU-based simulator outperforms by orders of magnitude the traditional CPU-based simulators and the state-of-the-art ML-based simulators, i.e., SimNet and Ithemal.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2021

Functional Adaptation of LPS-affected Dentoalveolar Fibrous Joints in Rats

We report the functional interplay between cementum of the root and alveolar bone of the socket is tuned by a uniquely positioned 70–80 µm wide fibrous and lubricious ligament in a dentoalveolar joint (DAJ). In this study, structural and biomechanical properties of the DAJ, periodontal ligament space (PDL-space also known as the joint space), alveolar bone of the socket, and cementum of the tooth root that govern the biomechanics of a lipopolysaccharide (LPS)-affected DAJ were mapped both in space and time. The hemi-maxillae from 20 rats (4 control at 6 weeks of age, 4 control and 4 LPS-affected at 12 weeks of age, 4 control and 4 LPS-affected at 16 weeks of age) were investigated using a hybrid technique; micro-X-ray computed tomography (5 µm resolution) in combination with biomechanical testing in situ. Temporal variations in bone and cementum volume fractions were evaluated. Trends in mineral apposition rates (MAR) in additional six Sprague Dawley rats (3 controls, 3 LPS-affected) were revealed by transforming spatial fluorochrome signals to functional growth rates (linearity factor - RW) of bone, dentin, and cementum using a fast Fourier transform on fluorochrome signals from 100-µm hemi-maxillae sections. An overall change in LPS-affected DAJ biomechanics (a 2.5–4.5X increase in tooth displacement and 2X tooth rotation at 6 weeks, no increase in displacement and a 7X increase in rotation at 12 weeks; 27% increase in bone effective strain at 6 weeks and 11% at 12 weeks relative to control) was associated with structural changes in the coronal regions of the DAJ (15% increase in PDL-space from 0 to 6 weeks but only 5% from 6 to 12 weeks compared to control). A significant increase (p < 0.05) in PDL-space between ligated and age-matched control was observed. The bone fraction of ligated at 12 weeks was significantly lower than its age-matched control, and no significant differences (p > 0.05) between groups were observed at 6 weeks. Cementum in the apical regions grew faster but nonlinearly (11% and 20% increase in cementum fraction (CF) at 6 and 12 weeks) compared to control. Alveolar bone revealed site-specific nonlinear growth with an overall increase in MAR (108.5 µm/week to 126.7 µm/week after LPS treatment) compared to dentin (28.3 µm/week in control vs. 26.1 µm/week in LPS-affected) and cementum (126.5 µm/week in control vs. 119.9 µm/week in LPS-affected). A significant increase in CF (p < 0.05) in ligated specimens was observed at 6 weeks of age. Anatomy-specific responses of cementum and bone to the mechano-chemo stimuli, and their collective temporal contribution to observed changes in PDL-space were perpetuated by altered tooth movement. Data highlight the “resilience” of DAJ function through the predominance of nonlinear growth response of cementum, changes in PDL-space, and bone architecture. Despite the significant differences in bone and cementum architectures, data provided insights into the reactionary effects of cementum as a built-in compensatory mechanism to reestablish functional competence of the DAJ. The spatial shifts in architectures of alveolar bone and cementum, and consequently ligament space, highlight adaptations farther away from the site of insult, which also is another novel insight from this study. These adaptations when correlated within the context of joint function (biomechanics) illustrate that they are indeed necessary to sustain DAJ function albeit being pathological.

60 APPLIED LIFE SCIENCES↗
DOE OSTI2024

Communication Lower Bounds and Optimal Algorithms for Multiple Tensor-Times-Matrix Computation

Multiple tensor-times-matrix (Multi-TTM) is a key computation in algorithms for computing and operating with the Tucker tensor decomposition, which is frequently used in multidimensional data analysis. Here, we establish communication lower bounds that determine how much data movement is required (under mild conditions) to perform the Multi-TTM computation in parallel. The crux of the proof relies on analytically solving a constrained, nonlinear optimization problem. We also present a parallel algorithm to perform this computation that organizes the processors into a logical grid with twice as many modes as the input tensor. We show that, with correct choices of grid dimensions, the communication cost of the algorithm attains the lower bounds and is therefore communication optimal. Finally, we show that our algorithm can significantly reduce communication compared to the straightforward approach of expressing the computation as a sequence of tensor-times-matrix operations when the input and output tensors vary greatly in size.

HBL-inequalities↗
DOE OSTI2023

HQ-Sim: High-performance State Vector Simulation of Quantum Circuits on Heterogeneous HPC Systems

Quantum circuit simulations are applied in more and more circum- stances as the quantum computing community becomes broader. It helps researchers to evaluate quantum algorithms and relieve the burden of limited quantum computing resources. However, most of the state-of-the-art quantum simulators utilize either CPU or GPU to store and calculate the state vector, which results in resource starvation. Moreover, the maximum number of qubits supported by the simulator is bounded by the memory, since the memory utilization increases exponentially with the number of qubits. In this study, we leverage Heterogeneous computing to utilize both CPU and GPU to store and update state vectors. We also integrate lossy data compression to reduce memory requirements. Specifically, we develop a heterogeneous framework that has a dynamic scheduler to fully utilize the computing resources. We apply lossy compression to chunked state vector to make the maximum number of qubits higher than the regular simulators, the compression also benefits the data movement between CPU and GPU.

Zhang, Boyuan↗
DOE OSTI2023

cuAlign: Scalable Network Alignment on GPU Accelerators

Given two graphs, the objective of network alignment is to find the best one-to-one mapping of vertices in one graph (??) to vertices in the other (??), such that the number of overlaps is maximized. We say that edges(??, ??) ???and(??', ??') ??? are overlapped if ?? is mapped to ??' and ?? is mapped to??'. Network alignment is an important optimization problem with several applications in bioinformatics, computer vision and ontology matching. Since it is an NP-hard problem, efficient heuristics and scalable implementations are necessary. In this work, we introduce a new framework that combines the concepts of intra-network proximity using vertex embedding,Belief Propagation (BP) and approximate weighted matching, and provides qualitative improvements up to22%over state-of-the-art approaches. We also provide scalable implementations on GPU accelerators, demonstrating up to19×speedup for Belief Propagation and 3× speedup for approximate weighted matching relative to previous multithreaded implementation. A combination of combinatorial and algebraic kernels within the network alignment algorithm poses significant hurdles for parallelization. Load imbalance and irregular DRAM traffic limit achievable performance on GPUs. Our novel approach identifies and exploits unique structural proper-ties of the BP-based algorithm and employs code fusion to reduce data movement between different steps of the algorithm. Using a diverse set of inputs, we demonstrate qualitative improvements of our algorithms, and performance gains of our GPU-accelerated implementation. We believe that our work will enable algorithmic improvements and practical applications of network alignment.

Xiang, Lizhi↗
DOE OSTI2024

Accelerated Constrained Sparse Tensor Factorization on Massively Parallel Architectures

This study presents the first constrained sparse tensor factorization (cSTF) framework that optimizes and fully offloads computation to massively parallel GPU architectures, and the first performance characterization of cSTF on GPU architectures. In contrast to prior work on tensor factorization, where the matricized tensor times Khatri-Rao product (MTTKRP) is the primary performance bottleneck, our systematic analysis of the cSTF algorithm on GPUs reveals that adding constraints creates an additional bottleneck in the update operation for many real-world sparse tensors. While executing the update operation on the GPU brings significant speedup over its CPU counterpart, it remains a significant bottleneck. To further accelerate the update operation, we propose cuADMM, a new update algorithm that leverages algorithmic and code optimization strategies to minimize both computation and data movement on GPUs. As a result, our framework delivers significantly improved performance compared to prior state-of-the-art. On 10 real-world sparse tensors, our framework achieves geometric mean speedup of 5.1 × (max 41.59 ×) and 7.01 × (max 58.05 ×) on the NIVIDA A100 and H100 GPUs, respectively, over the state-of-the-art SPLATT library running on a 26-core Intel Ice Lake Xeon CPU.

Soh, Yongseok↗
DOE OSTI2025

CGSim: A Simulation Framework for Large Scale Distributed Computing Environment

Large-scale distributed computing infrastructures such as the Worldwide LHC Computing Grid (WLCG) require comprehensive simulation tools for evaluating performance, testing new algorithms, and optimizing resource allocation strategies. However, existing simulators suffer from limited scalability, hardwired algorithms, lack of real-time monitoring, and inability to generate datasets suitable for modern machine learning approaches. We present CGSim, a simulation framework for large-scale distributed computing environments that addresses these limitations. Built upon the validated SimGrid simulation framework, CGSim provides high-level abstractions for modeling heterogeneous grid environments while maintaining accuracy and scalability. Key features include a modular plugin mechanism for testing custom workflow scheduling and data movement policies, interactive real-time visualization dashboards, and automatic generation of event-level datasets suitable for AI-assisted performance modeling. We demonstrate CGSim’s capabilities through a comprehensive evaluation using production ATLAS PanDA workloads, showing significant calibration accuracy improvements across WLCG computing sites. Scalability experiments show near-linear scaling for multi-site simulations, with distributed workloads achieving 6 × better performance compared to single-site execution. The framework enables researchers to simulate WLCG-scale infrastructures with hundreds of sites and thousands of concurrent jobs within practical time budget constraints on commodity hardware.

Vatsavai, Sairam Sri [Brookhaven National Laborato↗
DOE OSTI2026

FLYING SERVING: On-the-Fly Parallelism Switching for Large Language Model Serving

Production LLM serving must simultaneously deliver high throughput, low latency, and sufficient context capacity under non-stationary traffic and mixed request requirements. Data parallelism (DP) maximizes throughput by running independent replicas, while tensor parallelism (TP) reduces per-request latency and pools memory for long-context inference. However, existing serving stacks typically commit to a static parallelism configuration at deployment; adapting to bursts, priorities, or long-context requests is often disruptive and slow. We present Flying Serving, a vLLM-based system that enables online DP-TP switching without restarting engine workers. Flying Serving makes reconfiguration practical by virtualizing the state that would otherwise force data movement: (i) a zero-copy Model Weights Manager that exposes TP shard views on demand, (ii) a KV Cache Adaptor that preserves request KV state across DP/TP layouts, (iii) an eagerly initialized Communicator Pool to amortize collective setup, and (iv) a deadlock-free scheduler that coordinates safe transitions under execution skew. Across three popular LLMs and realistic serving scenarios, Flying Serving improves performance by up to 4.79 × under high load and 3.47 × under low load while supporting latency- and memory-driven requests.

Gao, Shouwei [ORNL]↗
DOE OSTI2021

BASIC HARTREE-FOCK PROXY APPLICTION

This proxy application simulates the compute load and data-movement of the kernel of the Hartree-Fock method in quantum chemistry. The proxy features a simplified algorithm for computing electron repulsion integrals that is easily offloaded to GPUs.

FLETCHER, GRAHAMD↗
DOE OSTI2021

pnnl/arena

CFA ARENA is a novel programming model with the support of a runtime targeting asynchronous data-centric execution paradigm in a distributed system. All the machine nodes in ARENA are connected by a ring network to bring the specialized computation to the data rather than the reverse to minimize data movement. The programming interfaces are implemented using C++

Tan, Cheng↗
DOE OSTI2026

Janus v1.0

Janus provides a software framework for lightweight container management and orchestration. It's primary use cases are around deploying containerized services for high-performance data movement needs. Thus, Janus differentiates itself from systems like Kubernetes by tailoring the deployment of containers around network, storage, and host tuning optimizations. Janus uses the concept of profiles to capture repeatable deployment patterns and applies them to container execution across one or more endpoints. A Janus Agent component provides remote host tuning and monitoring capabilities.

Essiari, Abdelilah [Lawrence Berkeley National Lab↗
DOE OSTI2021

Bluecrab: Comprehensive Reactor Analysis Bundle

BlueCRAB is a combination of existing codes created in close collaboration with the U.S. NRC useful for various reactor safety analysis simulations. For the purpose of classification it should be noted that BlueCRAB (the bundle) can be split up in several ways. At the heart of the software is "wrapping" and "coupling" code for brining in several non-INL projects from the NRC and Argonne National Laboratory (SAM). These codes facilitate the building and linking of these various codes during compilation and assist with data movement during execution. BlueCRAB can optionally link in several other applications including the following: BISON: fuels performance Griffin: Reactor Physics Pronghorn: CFD IAPS95: EOS for water, helium, nitrogen TRACE: NRC Code for 2 phase flow (system analysis) FAST: NRC Code for fuels performance SAM: ANL Code for single phase system analysis

Permann, Cody↗
DOE OSTI2022

Parthenon—a performance portable block-structured adaptive mesh refinement framework

On the path to exascale the landscape of computer device architectures and corresponding programming models has become much more diverse. While various low-level performance portable programming models are available, support at the application level lacks behind. To address this issue, we present the performance portable block-structured adaptive mesh refinement (AMR) framework Parthenon, derived from the well-tested and widely used Athena++ astrophysical magnetohydrodynamics code, but generalized to serve as the foundation for a variety of downstream multi-physics codes. Parthenon adopts the Kokkos programming model, and provides various levels of abstractions from multidimensional variables, to packages defining and separating components, to launching of parallel compute kernels. Parthenon allocates all data in device memory to reduce data movement, supports the logical packing of variables and mesh blocks to reduce kernel launch overhead, and employs one-sided, asynchronous MPI calls to reduce communication overhead in multi-node simulations. Using a hydrodynamics miniapp, we demonstrate weak and strong scaling on various architectures including AMD and NVIDIA GPUs, Intel and AMD x86 CPUs, IBM Power9 CPUs, as well as Fujitsu A64FX CPUs. At the largest scale on Frontier (the first TOP500 exascale machine), the miniapp reaches a total of 1.7 × 10 13 zone-cycles/s on 9216 nodes (73,728 logical GPUs) at [Formula: see text] weak scaling parallel efficiency (starting from a single node). In combination with being an open, collaborative project, this makes Parthenon an ideal framework to target exascale simulations in which the downstream developers can focus on their specific application rather than on the complexity of handling massively-parallel, device-accelerated AMR.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2025

Preparing MPICH for exascale

The advent of exascale supercomputers heralds a new era of scientific discovery, yet it introduces significant architectural challenges that must be overcome for MPI applications to fully exploit its potential. Among these challenges is the adoption of heterogeneous architectures, particularly the integration of GPUs to accelerate computation. Additionally, the complexity of multithreaded programming models has also become a critical factor in achieving performance at scale. The efficient utilization of hardware acceleration for communication, provided by modern NICs, is also essential for achieving low latency and high throughput communication in such complex systems. In response to these challenges, the MPICH library, a high-performance and widely used Message Passing Interface (MPI) implementation, has undergone significant enhancements. Here, this paper presents four major contributions that prepare MPICH for the exascale transition. First, we describe a lightweight communication stack that leverages the advanced features of modern NICs to maximize hardware acceleration. Second, our work showcases a highly scalable multithreaded communication model that addresses the complexities of concurrent environments. Third, we introduce GPU-aware communication capabilities that optimize data movement in GPU-integrated systems. Finally, we present a new datatype engine aimed at accelerating the use of MPI derived datatypes on GPUs. These improvements in the MPICH library not only address the immediate needs of exascale computing architectures but also set a foundation for exploiting future innovations in high-performance computing. By embracing these new designs and approaches, MPICH-derived libraries from HPE Cray and Intel were able to achieve real exascale performance on OLCF Frontier and ALCF Aurora respectively.

Guo, Yanfei [Argonne National Laboratory (ANL), Ar↗
DOE OSTI2025

TxDOT Road Elevation Model Dataset

This dataset provides three formats of Road Elevation Model (REM) data: 3D road line/polygon GeoPackage (GPKG), road lidar LAZ and COPC LAZ, and road digital surface model (DSM) GeoTIFF. Data are produced from the ~50TB TxGIO (formerly TNRIS) state lidar collections. This dataset is currently organized by maintenance section in each TxDOT district. Computation is done on GPU computing resources at Oak Ridge National Laboratory (ORNL), through a Strategic Partnership Project with UT Austin and an NSF ACCESS computing allocation award that enables fast massive data movement between TACC Corral and ORNL CADES/OLCF using Globus. In addition to this release from ORNL, a copy of this dataset can also be downloaded at https://web.corral.tacc.utexas.edu/nfiedata/road3d/.

13 HYDRO ENERGY↗
DOE OSTI2020

UPC++ v1.0 Programmer’s Guide, Revision 2020.10.0

UPC++ is a C++11 library that provides Partitioned Global Address Space (PGAS) programming. It is designed for writing parallel programs that run efficiently and scale well on distributed-memory parallel computers. The PGAS model is single program, multiple-data (SPMD), with each separate constituent process having access to local memory as it would in C++. However, PGAS also provides access to a global address space, which is allocated in shared segments that are distributed over the processes. UPC++ provides numerous methods for accessing and using global memory. In UPC++, all operations that access remote memory are explicit, which encourages programmers to be aware of the cost of communication and data movement. Moreover, all remote-memory access operations are by default asynchronous, to enable programmers to write code that scales well even on hundreds of thousands of cores.

97 MATHEMATICS AND COMPUTING↗