Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “latency”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 91 records · Page 5

Even Higher-Level Synthesis: An Exploration of AI Hardware Accelerators using HLS4ML

With the rise of artificial intelligence, the popularization of deep learning, and a constantly evolving industry, the demand for flexible and efficient tools has never been greater. As algorithms grow more complex, their runtime and energy consumption increase exponentially. Customized hardware accelerators, long used for specific mathematical operations, remain essential for managing modern applications' computational and power demands. Hardware accelerators can speed up complex computations by orders of magnitude, but their manual design and verification processes are often challenging and time-consuming. High-Level Synthesis (HLS) provides a solution by transforming high-level algorithm descriptions, typically written in C++ or SystemC, into synthesizable RTL suitable for hardware implementation. This approach reduces development time for RTL engineers while offering flexibility beyond what traditional handwritten RTL can provide. We extended this capability to the machine-learning domain with the open-source framework hls4ml, which allows neural networks trained in Python frameworks like Tensorflow or PyTorch to be synthesized into efficient hardware representations for the traditional FPGA and ASIC flows. This breakthrough addresses the growing need for reduced design turnaround and easy verification of ML hardware accelerators with low latency and power efficiency constraints. During this tutorial, we will demonstrate how Python complements HLS by simplifying the ML design process, bridging the gap between software and hardware development. Attendees will explore how we translate neural networks modeled in Python into fixed-point C++ models suitable for HLS workflows. We will dive into strategies like Value-Range Analysis and Quantization-Aware Training, which optimize these designs for deployment and evaluate their accuracy, power consumption, and energy efficiency. To exemplify these concepts, experts from Fermilab will share their experiences applying this technology to high-energy physics experiments, where real-time, low-latency processing is critical. Over the years, Fermilab engineers have demonstrated how deep neural networks, optimized for hardware using hls4ml, can meet the stringent requirements of trigger systems at the CERN Large Hadron Collider. These systems rely on rapid decision-making to process immense data volumes while retaining only the most relevant events for further analysis. The application of hls4ml has also been extended to innovative technologies like smart pixel arrays. These smart pixels integrate ML inference capabilities directly into sensor devices, enabling localized data processing at the pixel level. This approach drastically reduces the need to transmit raw data to external processing units, significantly decreasing power consumption and latency. By embedding neural networks within the pixel architecture, the smart pixels can identify and prioritize relevant data in real time, providing a highly efficient solution for edge computing in scenarios such as particle detectors and imaging systems. Fermilab's work highlights the potential of hardware-accelerated ML in scenarios where both speed and power efficiency are mission-critical. Through this tutorial, attendees will gain valuable insights into the challenges and solutions of deploying ML in hardware. Understanding how HLS and hls4ml streamline the development of neural network-based hardware accelerators is fundamental for the industry's future. Participants will learn how these technologies are shaping the future of AI and scientific computing.

Di Guglielmo, Giuseppe [Fermilab]↗

Intelligent Experiments through Real-Time AI: Fast Data Processing and Autonomous Detector Control for High-Energy Nuclear Experiments

The aim of this project is to develop software and hardware for fast real-time data processing and autonomous detector control and calibration for the sPHENIX and the future EIC experiments. Below summarizes Georgia Tech team efforts in the past year: 1. We developed a real-time clustering algorithm and FPGA-based pipeline architecture for processing fired pixel data from ALPIDE sensors in sPHENIX experiments. Our Columnar Clustering Co-Design introduces a hardware-aware, stream-friendly approach that segments pixel data by column pairs using a Column Pair Clustering (CPC) strategy, followed by Cluster Stitching to merge adjacent subclusters. Implemented in Vitis HLS, the pipeline comprises five stages—read-in, subclustering, stitching, analysis, and write-out—connected by tagged HLS streams with custom end-of-event signaling for robust synchronization. We designed a pipelined dataflow model optimized for throughput, low latency, and minimal buffering, enabling scalable clustering across events of arbitrary size. Our system maintains spatial precision via center-of-mass and shape key extraction and efficiently handles edge cases such as fragmented or nested clusters. Compared against DBSCAN in both software and hardware, our approach demonstrates competitive performance under FPGA constraints. 2. We also conducted a comprehensive algorithm-to-hardware co-design of connected component analysis tailored for sPHENIX experiments, focusing on real-time, low-latency processing using FPGAs and High-Level Synthesis (HLS). Starting from a Python-based particle tracking pipeline, the team translated the core logic—graph traversal via DFS and Union-Find—into an HLS-compatible C++ model, replacing dynamic memory and recursion with static arrays and pipelined control flow. The final design includes a fully streamed and dataflow-compatible Union-Find kernel optimized across five iterations, incorporating loop pipelining, array partitioning, AXI/FIFO interface tuning, and function flattening. Experimental results show up to 14.8× speedup over the CPU baseline, reducing per-graph latency to 1.58 μs and demonstrating strong resource efficiency with only ~7k LUTs and zero BRAM usage. The design maintains functional correctness against the Python reference using a Python-based C-simulation framework and Mean Squared Error metrics. This work validates the potential of HLS-driven FPGA designs for edge-level HEP data acquisition, laying a scalable foundation for future integration with real-time detector pipelines and multi-graph processing systems.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗

CMS Storage Performance with RNTuple

CMS is transitioning to use ROOT’s new RNTuple data storage format for the files CMS will write in the HL-LHC era. Based on initial tests, CMS expects faster I/O and smaller files compared to the present TTree storage format. This contribution will show a comprehensive performance comparison between RNTuple and TTree I/O using CMS AOD and MiniAOD data formats as test cases for both simulation and collision data corresponding to similar data taking conditions of LHC Run 3. Quantities such as the resulting file size, the memory usage of the I/O components, and the rate of events being read from a file or written to a file will be measured. CMS’ data processing relies heavily on reading files over the local or wide area networks. The file read patterns are important because the latencies have been seen to influence the total production job times. Therefore a study on the file read patterns will be conducted by recording traces of the offset, size, and timestamp of each read request for both RNTuple and TTree. The behavior of network reads will be mimicked by reading local files where artificial latency will be added to the read requests. The effect of different latency values on the job times will be studied.

Jones, Christopher D. [Fermilab]↗

Real-Time Inference For MI/RR Deblending

The Fermilab Main Injector (MI) and Recycler Ring (RR) share a common beam loss monitor (BLM) system, making loss events difficult to attribute to their source machine when beam is present in both simultaneously. The Real-time Edge AI for Distributed Systems (READS) project addresses this by deblending BLM readings in real time using machine learning (ML). The current FPGA based implementation meets the sub-3 ms latency requirement but carries a resource intensive hls4ml development cycle, motivating exploration of GPU based deployment. This paper characterizes inference latency on an NVIDIA Jetson Orin Nano and introduces a packet organization scheme for assembling synchronized event frames from seven distributed BLM DAQ streams. Using a Python based DAQ simulation with injected timing jitter in place of unavailable live beam data, the pipeline achieved an average end to end latency of 0.456 ms (σ = 0.122 ms) across 167,000 test frames, comfortably meeting the timing constraint. Early outliers were attributed to TensorRT warm-up rather than steady state limitations, suggesting GPU based inference is a viable alternative to the existing FPGA implementation.

Yu, Kellen [Cornell U.]↗

SuperLab 2.0 Showcase: Connecting Five Labs to Tackle Grid Complexity and Unlock Unique Grid Asset Potential

SuperLab 2.0 (5-Lab Demo) is a collaborative, national-scale experiment showcasing the coordination of geographically distributed energy assets in real time. The demonstration integrates 25 physical and digital assets, spanning wind, PV, batteries, electrolyzers, DC fast chargers, microgrid controllers, building automation systems, small modular reactor (SMR), control centers, and gas turbines, across five DOE national laboratories-NLR, INL, NETL, LBNL, and SNL. These assets are unified using Energy Sciences Network (ESnet), a low-latency, high-performance U.S. Department of Energy's (DOE) network, and controlled via a centralized energy controller hosted at NLR's ARIES facility. The demonstration validates the ability to stress-test hybrid energy systems under dynamic scenarios to de-risk advanced control strategies for greater resilience and flexibility. SuperLab 2.0 (5-Lab Demo) showcased a major advancement in federated national laboratory collaboration, enabling real-time, cross-laboratory experimentation to coordinate geographically dispersed distributed energy resources (DERs) using various communication protocols and networks. SuperLab 2.0 (5-Lab Demo) built on previous demonstrations conducted between NLR-PNNL and NLR-INL connecting diverse assets including distant protection devices, a SMR simulator, and a high temperature electrolyzer (HTE). Previous demos were based on a single connection between two labs with minimal coordination challenges. The 5-Lab demo with a centralized controller, distributed testbeds across different geographical locations, and use of protocols-based communication represents a scenario closer to real-world grid operations that coordinate resources across a region to meet system needs. This experiment studied how local DER controllers interact with a centralized energy controller during normal and abnormal events to maintain reliability. The SuperLab team across the five labs implemented a notional power system model equivalent of transmission and distribution lines, represented by the data networks interconnecting the labs. Each lab continuously exchanged local parameters (such as P and Q) from its Hardware-In-Loop (CHIL) and Power Hardware-In-Loop (PHIL) assets through centralized energy controller at NLR, enabling real-time interaction and coordination across sites. By leveraging ESnet as the communication backbone, the team successfully operated the distributed assets as a unified power system, with each bus represented by a different laboratory. This setup mirrors how assets interact in real-world power systems across dispersed locations with various protocols and latencies. At each lab site, assets were operated using their own local controllers which were coordinated through an overarching operation and control layer of centralized energy controller, equivalent to how an energy management system (EMS) orchestrates assets across a regional or national grid. SuperLab's federated connectivity utilized a Digital Real-Time Simulators (DRTS)-type gateway to connect Controller Hardware-In-Loop (CHIL) and PHIL assets between labs. To enable this federated connection through ESnet, a deterministic network was established where latency variations were consistent. This consistency allowed the development of digital filters for the power system assets across CHIL and PHIL interfaces to avoid unstable and unreliable grid conditions. This report provides an overview of the cross-laboratory configuration and offers insights into interconnecting geographically distributed research assets to test them as if they were co-located. This experiment represents a step toward linking nine DOE national laboratories, enabling nation-wide simulations that can address utility-driven challenges with grid resilience, flexibility, and modernization.

24 POWER TRANSMISSION AND DISTRIBUTION↗

Streaming authentication and multi-level security for communications networks using quantum cryptography

Message authenticators for quantum-secured communications facilitate low-latency authentication with assurances of security. Low-latency message authenticators are especially valuable in infrastructure systems where security and latency constraints are difficult to satisfy with conventional non-quantum cryptography. For example, a message transmitter receives a message and derives an authentication tag for the message based at least in part on an authenticator that uses one or more quantum keys. The message transmitter outputs the message and its authentication tag. A message receiver receives a message and authentication tag for the message. The message receiver derives a comparison tag for the message based at least in part on an authenticator that uses one or more quantum keys. The message receiver checks whether the message is authentic based on a comparison of the authentication tag and the comparison tag. In example implementations, the authenticator uses stream-wise cyclic redundancy code operations.

97 MATHEMATICS AND COMPUTING↗

Streaming authentication and multi-level security for communications networks using quantum cryptography

Message authenticators for quantum-secured communications facilitate low-latency authentication with assurances of security. Low-latency message authenticators are especially valuable in infrastructure systems where security and latency constraints are difficult to satisfy with conventional non-quantum cryptography. For example, a message transmitter receives a message and derives an authentication tag for the message based at least in part on an authenticator that uses one or more quantum keys. The message transmitter outputs the message and its authentication tag. A message receiver receives a message and authentication tag for the message. The message receiver derives a comparison tag for the message based at least in part on an authenticator that uses one or more quantum keys. The message receiver checks whether the message is authentic based on a comparison of the authentication tag and the comparison tag. In example implementations, the authenticator uses stream-wise cyclic redundancy code operations.

Hughes, Richard J.↗

Devices and methods for increasing the speed or power efficiency of a computer when performing machine learning using spiking neural networks

A method for increasing a speed and efficiency of a computer when performing machine learning using spiking neural networks. The method includes computer-implemented operations; that is, operations that are solely executed on a computer. The method includes receiving, in a spiking neural network, a plurality of input values upon which a machine learning algorithm is based. The method also includes correlating, for each input value, a corresponding response speed of a corresponding neuron to a corresponding equivalence relationship between the input value to a corresponding latency of the corresponding neuron. Neurons that trigger faster than other neurons represent close relationships between input values and neuron latencies. Latencies of the neurons represent data points used in performing the machine learning. A plurality of equivalence relationships are formed as a result of correlating. The method includes performing the machine learning using the plurality of equivalence relationships.

Vineyard, Craig Michael↗

Exploration of Real Time Inference for MI-RR Deblending on GPU/TPU Systems

The Fermilab Main Injector (MI) and Recycler Ring (RR) share a common beam loss monitor (BLM) system, making loss events difficult to attribute to their source machine when beam is present in both simultaneously. The Real-time Edge AI for Distributed Systems (READS) project addresses this by deblending BLM readings in real time using machine learning (ML). The current FPGA based implementation meets the sub-3 ms latency requirement but carries a resource intensive hls4ml development cycle, motivating exploration of GPU based deployment. This paper characterizes inference latency on an NVIDIA Jetson Orin Nano and introduces a packet organization scheme for assembling synchronized event frames from seven distributed BLM DAQ streams. Using a Python based DAQ simulation with injected timing jitter in place of unavailable live beam data, the pipeline achieved an average end to end latency of 0.456 ms (σ = 0.122 ms) across 167,000 test frames, comfortably meeting the timing constraint. Early outliers were attributed to TensorRT warm-up rather than steady state limitations, suggesting GPU based inference is a viable alternative to the existing FPGA implementation.

Yu, Kellen [Fermilab; Cornell U.]↗

PtCoO 2 for Scaled Interconnects

Copper (Cu) interconnects are an increasingly important bottleneck in integrated circuits due to energy consumption and latency caused by the notable increase in Cu resistivity as dimensions decrease, primarily due to electron scattering at surfaces. Herein, the potential of a directional conductor, PtCoO 2 , which has a low bulk resistivity and a distinctive anisotropic structure that mitigates electron surface scattering is showcased. Thin films of PtCoO 2 of various thicknesses are synthesized by molecular beam epitaxy (MBE) coupled with a postdeposition annealing process and the superior quality of PtCoO 2 films is demonstrated by multiple characterization techniques. The thickness‐dependent resistivity curve illustrates that PtCoO 2 significantly outperforms effective Cu (Cu with TaN barriers) and Ru in resistivity below 20.0 nm with a more than 6x reduction compared to effective Cu below 6.0 nm, having a value of only 6.32 μΩ cm at 3.3 nm. It is determined that grain boundary scattering can still be improved for even lower resistivities in this material system through a combination of experiments and theoretical simulations. PtCoO 2 is therefore a highly promising alternative material for future interconnect technologies promising lower resistivities, better stability, and significant improvements in energy efficiency and latency for advanced integrated circuits.

Li, Yansong [Department of Electrical Engineering ↗

Scalable Data-Intensive Geocomputation: A Design for Real-Time Continental Flood Inundation Mapping

The convergence of data-intensive and extreme-scale computing enables an integrated software and data ecosystem for scientific discovery. Developments in this realm will fuel transformative research in data-driven interdisciplinary domains. Geocomputation provides computing paradigms in Geographic Information Systems (GIS) for interactive computing of geographic data, processes, models, and maps. Because GIS is data-driven, the computational scalability of a geocomputation workflow is directly related to the scale of the GIS data layers, their resolution and extent, as well as the velocity of the geo-located data streams to be processed. Unique in high user interactivity and low end-to-end latency requirements, geocomputation applications will dramatically benefit from the convergence of high-end data analytics (HDA) and high-performance computing (HPC). The application level challenge, however, is to identify and eliminate computational bottlenecks that arise along a geocomputation workflow. Indeed, poor scalability at any of the workflow components is detrimental to the entire end-to-end pipeline. Here, we study a large geocomputation use case in flood inundation mapping that handles multiple national-scale geospatial datasets and targets low end-to-end latency. We discuss benefits and challenges for harnessing both HDA and HPC for data-intensive geospatial data processing and intensive numerical modeling of geographic processes. We propose an HDA+HPC geocomputation architecture design that couples HDA (e.g., Spark)-based spatial data handling and HPC-based parallel data modeling. Key techniques for coupling HDA and HPC to bridge the two different software stacks are reviewed and discussed.

Liu, Yan↗

Distributed out-of-memory NMF on CPU/GPU architectures

We propose an efficient distributed out-of-memory implementation of the non-negative matrix factorization (NMF) algorithm for heterogeneous high-performance-computing systems. The proposed implementation is based on prior work on NMFk, which can perform automatic model selection and extract latent variables and patterns from data. In this work, we extend NMFk by adding support for dense and sparse matrix operation on multi-node, multi-GPU systems. The resulting algorithm is optimized for out-of-memory problems where the memory required to factorize a given matrix is greater than the available GPU memory. Memory complexity is reduced by batching/tiling strategies, and sparse and dense matrix operations are significantly accelerated with GPU cores (or tensor cores when available). Input/output latency associated with batch copies between host and device is hidden using CUDA streams to overlap data transfers and compute asynchronously, and latency associated with collective communications (both intra-node and inter-node) is reduced using optimized NVIDIA Collective Communication Library (NCCL) based communicators. Benchmark results show significant improvement, from 32X to 76x speedup, with the new implementation using GPUs over the CPU-based NMFk. Good weak scaling was demonstrated on up to 4096 multi-GPU cluster nodes with approximately 25,000 GPUs when decomposing a dense 340 Terabyte-size matrix and an 11 Exabyte-size sparse matrix of density 10 -6 .

97 MATHEMATICS AND COMPUTING↗

DLHub: Simplifying publication, discovery, and use of machine learning models in science

Machine Learning (ML) has become a critical tool enabling new methods of analysis and driving deeper understanding of phenomena across scientific disciplines. There is a growing need for "learning systems" to support various phases in the ML lifecycle. While others have focused on supporting model development, training, and inference, few have focused on the unique challenges inherent in science, such as the need to publish and share models and to serve them on a range of available computing resources. In this paper, we present the Data and Learning Hub for science (DLHub), a learning system designed to support these use cases. Specifically, DLHub enables publication of models, with descriptive metadata, persistent identifiers, and flexible access control. It packages arbitrary models into portable servable containers, and enables low-latency, distributed serving of these models on heterogeneous compute resources. In this work, we show that DLHub supports low-latency model inference comparable to other model serving systems including TensorFlow Serving, SageMaker, and Clipper, and improved performance, by up to 95%, with batching and memoization enabled. We also show that DLHub can scale to concurrently serve models on 500 containers. Finally, we describe five case studies that highlight the use of DLHub for scientific applications.

97 MATHEMATICS AND COMPUTING↗

Multiplicity trigger detector for the S π RIT experiment

A multiplicity trigger detector (MTD) was developed for the SπRIT experiment that aims to probe the high- density symmetry energy via heavy-ion collisions (HICs). The MTD is designed to measure the charged-particle multiplicity in HICs to provide a trigger signal that can be used to select high-multiplicity events induced by central collisions. The MTD consists of two side-walls segmented into 30 plastic scintillation paddles, each equipped with a multi-pixel photon counter. A custom board with EASIROC front-end ASIC is used as readout electronics. A multiplicity logic signal is generated by an on-board FPGA that processes the discriminated signals from EASIROCs. The overall latency of the whole detector system for outputting the trigger is less than 100 ns, including 52 ns of electronics latency for processing the signals. During the measurement of 112 Sn + 124 Sn reactions at an incident energy of 270 MeV/nucleon, the minimum multiplicity threshold of the MTD was set to 4, where the overall trigger efficiency was obtained to be about 39 % for inclusive nuclear reactions. Furthermore, a simulation study with a numerical calculation of HICs found that the central collisions of impact parameters of less than 4 fm can be triggered with a high efficiency of more than 95 % by the experimental trigger condition.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗

In-pixel AI for lossy data compression at source for X-ray detectors

Integrating neural networks for data compression directly in the Read-Out Integrated Circuits (ROICs), i.e. the pixelated front-end, would result in a significant reduction in off-chip data transfer, overcoming the I/O bottleneck. Our ROIC test chip (AI-In-Pixel-65) is designed in a 65 nm Low Power CMOS process for the readout of pixelated X-ray detectors. Each pixel consists of an analog front-end for signal processing and a 10b analog-to-digital converter operating at 100KSPS. Here, we compare two non-reconfigurable techniques, Principal Component Analysis (PCA) and an AutoEncoder (AE) as lossy data compression engines implemented within the pixelated area. The PCA algorithm achieves 50$×$ compression, adds one clock cycle latency, and results in a 21% increase in the pixel area. The AE achieves 70$×$ compression, adds 30 clock cycle latency, and results in a similar area increase.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗

All-optical image denoising using a diffractive visual processor

Abstract Image denoising, one of the essential inverse problems, targets to remove noise/artifacts from input images. In general, digital image denoising algorithms, executed on computers, present latency due to several iterations implemented in, e.g., graphics processing units (GPUs). While deep learning-enabled methods can operate non-iteratively, they also introduce latency and impose a significant computational burden, leading to increased power consumption. Here, we introduce an analog diffractive image denoiser to all-optically and non-iteratively clean various forms of noise and artifacts from input images – implemented at the speed of light propagation within a thin diffractive visual processor that axially spans <250 × λ, where λ is the wavelength of light. This all-optical image denoiser comprises passive transmissive layers optimized using deep learning to physically scatter the optical modes that represent various noise features, causing them to miss the output image Field-of-View (FoV) while retaining the object features of interest. Our results show that these diffractive denoisers can efficiently remove salt and pepper noise and image rendering-related spatial artifacts from input phase or intensity images while achieving an output power efficiency of ~30–40%. We experimentally demonstrated the effectiveness of this analog denoiser architecture using a 3D-printed diffractive visual processor operating at the terahertz spectrum. Owing to their speed, power-efficiency, and minimal computational overhead, all-optical diffractive denoisers can be transformative for various image display and projection systems, including, e.g., holographic displays.

36 MATERIALS SCIENCE↗

Embedding security into ferroelectric FET array via in situ memory operation

Non-volatile memories (NVMs) have the potential to reshape next-generation memory systems because of their promising properties of near-zero leakage power consumption, high density and non-volatility. However, NVMs also face critical security threats that exploit the non-volatile property. Compared to volatile memory, the capability of retaining data even after power down makes NVM more vulnerable. Existing solutions to address the security issues of NVMs are mainly based on Advanced Encryption Standard (AES), which incurs significant performance and power overhead. In this paper, we propose a lightweight memory encryption/decryption scheme by exploiting in-situ memory operations with negligible overhead. To validate the feasibility of the encryption/decryption scheme, device-level and array-level experiments are performed using ferroelectric field effect transistor (FeFET) as an example NVM without loss of generality. Besides, a comprehensive evaluation is performed on a 128 × 128 FeFET AND-type memory array in terms of area, latency, power and throughput. Compared with the AES-based scheme, our scheme shows ~22.6×/~14.1× increase in encryption/decryption throughput with negligible power penalty. Furthermore, we evaluate the performance of our scheme over the AES-based scheme when deploying different neural network workloads. Our scheme yields significant latency reduction by 90% on average for encryption and decryption processes.

47 OTHER INSTRUMENTATION↗

Nanosecond anomaly detection with decision trees and real-time application to exotic Higgs decays

Abstract We present an interpretable implementation of the autoencoding algorithm, used as an anomaly detector, built with a forest of deep decision trees on FPGA, field programmable gate arrays. Scenarios at the Large Hadron Collider at CERN are considered, for which the autoencoder is trained using known physical processes of the Standard Model. The design is then deployed in real-time trigger systems for anomaly detection of unknown physical processes, such as the detection of rare exotic decays of the Higgs boson. The inference is made with a latency value of 30 ns at percent-level resource usage using the Xilinx Virtex UltraScale+ VU9P FPGA. Our method offers anomaly detection at low latency values for edge AI users with resource constraints.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗