Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “COMMUNICATIONS”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,027 records · Page 57

SV-Sim: Scalable PGAS-based State Vector Simulation of Quantum Circuits

High-performance quantum circuit simulation in a classic HPC is still imperative in the NISQ era. Observing that the major obstacle of scalable state-vector quantum simulation arises from the massively fine-grained irregular data-exchange with remote nodes, in this paper we present SV-Sim to apply the emerging PGAS-based communication models (i.e., direct peer access for intra-node CPUs/GPUs and SHMEM for inter-node CPU/GPU clusters) for efficient scalable quantum circuit simulation. Through an orchestrated device functional pointer design, SV-Sim is able to abstract the quantum gate sets across various heterogeneous backends, including IBM/Intel/AMD CPUs, NVIDIA /AMD GPUs, and Intel MIC, in a unified framework, but still asserting outstanding performance and tractable interface to higher-level quantum programming environments, such as IBM Qiskit, Microsoft Q\# and Google Cirq. Circumventing the disability of polymorphism in GPUs and leveraging the device-initiated one-sided communication, SV-Sim can process dynamically synthesized quantum circuit in a single GPU/CPU kernel without the need of expensive JIT or runtime branching, significantly improving the performance and simplifying the programming complexity for the emerging variational quantum algorithms. Evaluations on NVIDIA A100-DGX-1, V100-DGX-2, AMD MI100, ALCF Theta, and OLCF Summit HPCs show that SV-Sim can delivery scalable performance on various state-of-the-art HPC platforms, offering a useful tool for quantum algorithm validation and verification.

Li, Ang↗

MAPA: Multi-Accelerator Pattern Allocation Policy for Multi-Tenant GPU Servers

Multi-accelerator servers are increasingly being deployed in shared multi-tenant environments (such as in cloud data centers) in order to meet the demands of large-scale compute-intensive workloads. In addition, these accelerators are increasingly being inter-connected in complex topologies and workloads are exhibiting a wider variety of inter-accelerator communication patterns. However, existing al-location policies are ill-suited for these emerging use-cases. Specifically, this work identifies that multi-accelerator workloads are commonly fragmented leading to reduced bandwidth and increased latency for inter-accelerator communication. We propose Multi-Accelerator Pattern Allocation (MAPA), a graph pattern mining approach towards providing generalized allocation support for allocating multi-accelerator workloads on multi-accelerator servers. We demonstrate that MAPA is able to improve the execution time of multi-accelerator workloads and that MAPA is able to provide generalized benefits across various accelerator topologies. Finally, we demonstrate a speedup of 12.4% for75th percentile of jobs with the worst case execution time reduced by up to 35% against baseline policy using MAPA.

Ranganath, Kiran↗

Design and analysis of CXL performance models for tightly-coupled heterogeneous computing

Truly heterogeneous systems enable partitioned workloads to be mapped to the hardware that nets the best performance. However, current practice requires that inter-device communication between different vendors' hardware use host memory as an intermediary step. To date, there are no widely adopted solutions that allow accelerators to directly transfer data. A new cache-coherent protocol, CXL, aims to facilitate easier, fine-grained sharing between accelerators. In this work we analyze existing methods for designing heterogeneous applications that target GPUs and FPGAs working collaboratively, followed by an exploration to show the benefits of a CXL-enabled system. Specifically, we develop a test application that utilizes both an NVIDIA P100 GPU and a Xilinx U250 FPGA to show current communication limitations. From this application, we capture overall execution time and throughput measurements on the FPGA and GPU. We use these measurements as inputs to novel CXL performance models to show that using CXL caching instead of host memory results in a 1.31X speedup, while a more tightly-coupled pipelined implementation using CXL-enabled hardware would result in a speedup of 1.45X.

Cabrera, Anthony↗

FLASH: FPGA-Accelerated Smart Switches with GCN Case Study

Some communication switches, e.g., the Mellanox SHArP and those in the IBM BlueGene clusters, are augmented to process packets at the application level with fixed-function collectives. This approach, however, lacks flexibility, which limits their applicability in diverse and dynamic workloads. Recently, a new type of programmable packet processor, which uses high-level languages, e.g., P4, has emerged as possible candidates. P4-based switches, however, fall short in certain applications, including machine learning, where capabilities not currently supported by P4 are needed. These include more complex calculation, such as sparse computation and fused multiply-accumulate, data-intensive floating point operations, data reuse, and significant memory. The problem addressed here is that such a switch augmentation needs to support: a large amount of state, significant flexible compute capability, and ease of programming, all while maintaining full functionality, including ensuring high throughput, and demonstrating utility. In this work, we propose a programmable look-aside-type accelerator that can be embedded into, or attached to, existing communication switch pipelines and that is capable of processing packets at line-rate. The proposed in-switch accelerator is based on mixing an ISA (subset of RISC-V instructions) with dataflow graphs (found in CGRAs). To augment performance, vector instructions are also supported. To facilitate usability, we have developed a complete toolchain to compile user-provided C/C++ codes to appropriate back-end instructions for configuring the accelerator. While this approach is flexible enough to support various workloads, in this paper, we consider Graph Convolutional Networks (GCNs) as a case study. Experimental results show that this approach considerably improves the performance of distributed GCN applications.

Haghi, Pouya↗

Large-Scale Materials Modeling at Quantum Accuracy: Ab Initio Simulations of Quasicrystals and Interacting Extended Defects in Metallic Alloys

Ab initio electronic-structure has remained dichotomous between achievable accuracy and length-scale. Quantum many-body (QMB) methods realize quantum accuracy but fail to scale. Density functional theory (DFT) scales favorably but remains far from quantum accuracy. We present a framework that breaks this dichotomy by use of three interconnected modules: (i) invDFT: a methodological advance in inverse DFT linking QMB methods to DFT; (ii) MLXC: a machine-learned density functional trained with invDFT data, commensurate with quantum accuracy; (iii) DFT-FE-MLXC: an adaptive higher-order spectral finite-element (FE) based DFT implementation that integrates MLXC with efficient solver strategies and HPC innovations in FE-specific dense linear algebra, mixed-precision algorithms, and asynchronous compute-communication. Furthermore, we demonstrate a paradigm shift in DFT that not only provides an accuracy commensurate with QMB methods in ground-state energies, but also attains an unprecedented performance of 659.7 PFLOPS (43.1% peak FP64 performance) on 619,124 electrons using 8,000 GPU nodes of Frontier supercomputer.

density functional theory↗

CYDRES: CYber Defense and REsilient System for securing grid-interactive efficient buildings

Smart buildings, especially Grid-interactive Efficient Buildings (GEBs), suffer from cyber-attacks and physical faults due to the integration of a large number of sensors and controls, connected devices, and associated communication networks. This study demonstrated a real-time advanced building resilient platform, called CYber Defense and REsilient System (CYDRES), which is deployable for existing and emerging Building Automation Systems (BASs). CYDRES aims to empower GEBs with cyber-attack-immune capabilities through multi-layer prevention and adaptation mechanisms to monitor, detect, and respond to cyber-attacks and physical operational faults. CYDRES is demonstrated through real-time experiments in a Hardware-in-the-Loop (HIL) testbed.

Building automation system, Cyber-attacks, Physica↗

CommBench: Micro-Benchmarking Hierarchical Networks with Multi-GPU, Multi-NIC Nodes

Modern high-performance computing systems have multiple GPUs and network interface cards (NICs) per node. The resulting network architectures have multilevel hierarchies of subnetworks with different interconnect and software technologies. These systems offer multiple vendor-provided communication capabilities and library implementations (IPC, MPI, NCCL, RCCL, OneCCL) with APIs providing varying levels of performance across the different levels. Understanding this performance is currently difficult because of the wide range of architectures and programming models (CUDA, HIP, OneAPI). We present CommBench, a library with cross-system portability and a high-level API that enables developers to easily build microbenchmarks relevant to their use cases and gain insight into the performance (bandwidth & latency) of multiple implementation libraries on different networks. We demonstrate CommBench with three sets of microbenchmarks that profile the performance of six systems. Our experimental results reveal the effect of multiple NICs on optimizing the bandwidth across nodes and also present the performance characteristics of four available communication libraries within and across nodes of NVIDIA, AMD, and Intel GPU networks.

Hidayetoglu, Mert↗

RingX: Scalable Parallel Attention for Long-Context Learning on HPC

The attention mechanism has become foundational for remarkable AI breakthroughs since the introduction of the Transformer, driving the demand for increasingly longer context to power frontier models such as large-scale reasoning language models and high-resolution image/video generators. However, its quadratic computational and memory complexities present substantial challenges. Current state-of-the-art parallel attention methods, such as ring attention, are widely adopted for long-context training but utilize a point-to-point communication strategy that fails to fully exploit the capabilities of modern HPC network architectures. In this work, we propose ringX, a scalable family of parallel attention methods optimized explicitly for HPC systems. By enhancing workload partitioning, refining communication patterns, and improving load balancing, ringX achieves up to 3.4 × speedup compared to conventional ring attention on the Frontier supercomputer. Optimized for both bi-directional and causal attention mechanisms, ringX demonstrates its effectiveness through training benchmarks of a Vision Transformer (ViT) on a climate dataset and a Generative Pre-Trained Transformer (GPT) model, Llama3 8B. Our method attains an end-to-end training speedup of approximately 1.5 × in both scenarios. To our knowledge, the achieved 38% model FLOPs utilization (MFU) for training Llama3 8B with a 1M-token sequence length on 4,096 GPUs represents one of the highest training efficiencies reported for long-context learning on HPC systems. Our code implementation is available at https://github.com/jqyin/ringX-attention.

Yin, Junqi [ORNL] (ORCID:0000000338435520)↗

DoCeph: DPU-Offloaded Messaging in Ceph for Reduced Host CPU Utilization

Ceph is a widely used distributed object store, but its messenger layer imposes substantial CPU overhead on the host. To address this limitation, we propose DoCeph, a DPU-offloaded storage architecture for Ceph that disaggregates the system by offloading the communication-intensive messaging component to the DPU while retaining the storage backend on the host. The DPU efficiently manages communication, using lightweight RPC for metadata operations and DMA for data transfer. Moreover, DoCeph introduces a pipelining technique that overlaps data transmission with buffer preparation, mitigating hardware-imposed transfer size limitations. We implemented DoCeph on a Ceph cluster with NVIDIA BlueField-3 DPUs. Evaluation results indicate that DoCeph cuts host CPU usage by up to 92% while sustaining stable throughput and providing larger performance benefits for object writes over 1 MB.

Park, Kuri [Sogang University]↗

A Grassroots Network and Community Roadmap for Interconnected Autonomous Science Laboratories for Accelerated Discovery

Scientific discovery is being revolutionized by AI and autonomous systems, yet current autonomous laboratories remain isolated islands unable to collaborate across institutions. We present the Autonomous Interconnected Science Lab Ecosystem (AISLE), a grassroots network transforming fragmented capabilities into a unified system that shorten the path from ideation to innovation to impact and accelerates discovery from decades to months. AISLE addresses five critical dimensions: (1) cross-institutional equipment orchestration, (2) intelligent data management with FAIR compliance, (3) AI-agent driven orchestration grounded in scientific principles, (4) interoperable agent communication interfaces, and (5) AI/ML-integrated scientific education. By connecting autonomous agents across institutional boundaries, autonomous science can unlock research spaces inaccessible to traditional approaches while democratizing cutting-edge technologies. This paradigm shift toward collaborative autonomous science promises breakthroughs in sustainable energy, materials development, and public health.

Ferreira da Silva, Rafael [Oak Ridge National Labo↗

Scalable Federated Learning for Scientific Foundation Models on Leadership-Class Systems

Federated learning (FL) at leadership-class HPC systems remains largely unexplored, despite growing interest in deploying federated workflows on modern HPC systems. This paper provides the first system-level empirical characterization of federated fine-tuning of pretrained foundation models on an exascale supercomputer under a multi-node deployment. Using up to 96 concurrent FL clients deployed across Frontier nodes, we study the impact of client scale, model size, data heterogeneity, partial participation, and differential privacy on runtime, communication overhead, and convergence stability. Our results show that pretrained transformer models remain robust to heterogeneity, client dropout, and privacy noise, while system efficiency degrades rapidly with scale as synchronizat and orchestration dominate runtime. We further demonstrate that system-aware execution strategies, including intra-node aggregation and early aggregation, significantly reduce wall-clock time without degrading model quality. These findings establish a practical performance baseline and inform the design of communication-efficient FL systems on leadership-class HPC platforms.

Kotevska, Olivera [ORNL] (ORCID:0000000316772243)↗

EASEE: Edge Advertisements into Snapshots using Evolving Expectations

Code developed in conjunction with a research paper. This code demonstrates how to identify static graph snapshots from a stream of communication events. A second, related code generates simulated communication events with known split moments for accuracy testing.SAND2020-12245 M Sandia National Laboratories is a multimission laboratory managed and operated by National Technology & Engineering Solutions of Sandia, LLC, a wholly owned subsidiary of Honeywell International Inc., for the U.S. Department of Energy's National Nuclear Security Administration under contract DE-NA0003525.

Wendt, Jeremy↗

ModuleOT

ModuleOT is an open hardware security platform which provides all features necessary for securing remote energy resources. The platform consists of a physical bump in-the wire device which runs a custom-built application built with Go and Python and leverages AES-NI Instruction set available on modern hardware for cryptographic acceleration. By combining these features, ModuleOT acts as an all-in-one low-cost solution to enable cryptographically secured communications to any critical remote servers or devices. Because the software application has been built using Golang, this source can be easily compiled for different hardware platforms. The module is designed to provide the following core features: (1) encrypted communications across an untrusted network, (2) certificate-based authentication with secure storage, (3) hardware cryptographic acceleration, (4) IP-based whitelisting, (5) local firewall management, and (6) legacy (RS485) device support.

Hasandka, Adarsh↗

XVC FTDI JTAG v1.0

the Xilinx tools relay on a component on the board (FTDI chip) to have a small memory from a particular manufacturer (Digilent) to serve as a bridge to the FPGA. What the software we're requesting to release simply does is to run between the Xilinx tools (one of them is called Vivado but there are others) and our boards so that we don't need that memory on our boards. The protocol the software uses to communicate with the Xilinx tools is called XVC (Xilinx Virtual Cable). This server code runs on the computer that is physically connected to the FPGA board, but the Xilinx tools that communicate with it can run either on that computer or some other computer connected to it over the network, this is where the network component comes into play.

Norum, William↗

OpenDSS-wrapper (Distribution System Co-simulator with Distributed Energy Resource Controls)

Electric grid transformation with the proliferation of distributed energy resources (DER), such as solar photovoltaic (PV), wind, advanced energy storage technologies, and electric vehicles, and the growing use of communication technologies in both transmission and distribution systems are increasing the need to capture the interactions among these systems. Advanced modeling, control, and simulation tools that can perform co-simulation of electric power systems with other domains become indispensable to accurately model these interactions. To address this need, we have developed a codebase that integrates an electric power distribution system simulator, DER models, and DER controls. The codebase is based on OpenDSS, a distribution system simulator, and an existing open source co-simulation framework called HELICS. The contribution and uniqueness of the proposed codebase is that it tailors the generic HELICS framework specifically for distribution grid-related applications. The codebase includes an OpenDSS wrapper that controls the simulation, implements advanced DER controls, and extracts power flows, voltages, and other power system element information from the the distribution network modeled in OpenDSS. Sample HELICS federates, including a federate for OpenDSS, are provided that can communicate messages through the HELICS interface. Sample federates can be modified and additional HELICS federates can be added by the user depending on their use case requirements. SEE ALSO: https://github.com/NREL/dss-cosim

Blonsky, Michael↗

PyChargeModel (Oriented Programming Based Electric Vehicle and Electric Vehicle Supply Equipment Charging Model in Python) [SWR-22-39]

The PyChargeModel creates two classes called "ElectricVehicles" and "evse_class", which can be used to create multiple instances of electric vehicles (EVs) and electric vehicle supply equipment (EVSE or charging ports) and simulate charging behavior. These objects can be instantiated with several properties such as battery chemistries, battery pack sizes, cell sizes, charging port power, dc or ac chargers etc. The objects can communicate with each other by calling different methods built within the classes. Through these methods, each EV object can be assigned to an EVSE, charged either using a default protocol or using setpoint values communicated from a site controller via the EVSE object.

Mishra, Partha↗

AutodiDAQt v1.1.0

AutodiDAQt automates and simplifies writing data acquisition software for spectroscopy and microscopy. After defining only how to communicate with instruments and hardware, autodiDAQt generates user interfaces for long running acquisition applications, handlings data collation and retention, and provides remote communication to analysis computers. This reduces the time to get experiments running from months to hours and increases reliability for scientific experiments. AutodiDAQt metaprograms from instrument drivers directly, where possible.

Stansbury, Conrad↗

Caffeine v0.1.0

Caffeine is the CoArray Fortran Framework of Efficient Interfaces to Network Environments. Caffeine aims to produce a parallel runtime library that will support Fortran compilers with a programming-model-agnostic application binary interface (ABI) to various lower-level communication libraries. The current version of Caffeine uses the GASNet-EX networking middleware, also developed at Berkeley Lab. On many combinations of applications and platforms, GASNet-EX outperforms the widely used Message Passing Interface (MPI). Through GASNet-EX's support for communicating between graphics processing units (GPU), GASNet-EX has features that specifically target the emerging, leading-edge exascale computing platforms.

Rouson, Damian↗