Results for “latency” — Page 27

DOE OSTI2021

Accelerating Transformer-based Deep Learning Models on FPGAs using Column Balanced Block Pruning

Although Transformer-based language representations achieve state-of-the-art accuracy on various natural language processing (NLP) tasks, the large model size has been challenging the resource constrained computing platforms. Weight pruning, as a popular and effective technique in reducing the number of weight parameters and accelerating the Transformer, has been investigated on GPUs. However, the Transformer acceleration using weight pruning on field-programmable gate array (FPGAs) remains unexplored. This paper investigates the column balanced block-wise pruning on Transformer and designs an FPGA acceleration engine to customize the balanced blockwise matrix multiplication. We implement the Transformer model with proper hardware scheduling, and the experiments show that the Transformer inference on FPGA achieves 10.35 ms latency with the batch size of 32, which is 10.96 × speed up comparing to CPU platform and 2.08 × speed up comparing to GPU platform.

Peng, Hongwu↗

DOE OSTI2020

Performance Potential of Mixed Data Management Modes for Heterogeneous Memory Systems

Many high-performance systems now include different types of memory devices within the same compute platform to meet strict performance and cost constraints. Such heterogeneous memory systems often include an upper-level tier with better performance, but limited capacity, and lower-level tiers with higher capacity, but less bandwidth and longer latencies for reads and writes. To utilize the different memory layers efficiently, current systems rely on hardware-directed, memory -side caching or they provide facilities in the operating system (OS) that allow applications to make their own data-tier assignments. Since these data management options each come with their own set of trade-offs, many systems also include mixed data management configurations that allow applications to employ hardware- and software-directed management simultaneously, but for different portions of their address space. Despite the opportunity to address limitations of stand-alone data management options, such mixed management modes are under-utilized in practice, and have not been evaluated in prior studies of complex memory hardware. In this work, we develop custom program profiling, configurations, and policies to study the potential of mixed data management modes to outperform hardware- or software-based management schemes alone. Our experiments, conducted on an Intel ® Knights Landing platform with high-bandwidth memory, demonstrate that the mixed data management mode achieves the same or better performance than the best stand-alone option for five memory intensive benchmark applications (run separately and in isolation), resulting in an average speedup compared to the best stand-alone policy of over 10 %, on average.

Effler, Chad↗

DOE OSTI2021

Productive Programming of Distributed Systems with the SHAD C++ Library

High-performance computing (HPC) is often perceived as a matter of making large-scale systems (e.g., clusters) run as fast as possible, regardless the required programming effort. However, the idea of "bringing HPC to the masses" has recently emerged. Inspired by this vision, we have designed SHAD, the Scalable High-performance Algorithms and Data-structures library. SHAD is open source software, written in C++, for C++ developers. Unlike other HPC libraries for distributed systems, which rely on SPMD models, SHAD adopts a shared-memory programming abstraction, to make C++ programmers feel at home. Underneath, SHAD manages tasking and data-movements, moving the computation where data resides and taking advantage of asynchrony to tolerate network latency. At the bottom of his stack, SHAD can interface with multiple runtime systems: this not only improves developer’s productivity, by hiding the complexity of such software and of the underlying hardware, but also greatly enhance code portability. Thanks to its abstraction layers, SHAD can indeed target different systems, ranging from laptops to HPC clusters, without any need for modifying the user-level code. We have prototyped and open-sourced the implementation of (a subset of) the C++ standard library (STL) targeting multi-node HPC clusters. Our work allows plain STL-based C++ code to scale on HPC systems, with no need for rewriting the code to exploit the complex hardware. SHAD is available under Apache v2 License at https://github.com/pnnl/SHAD. In this paper we overview the design of the SHAD library, depicting its main components: runtime systems abstractions for tasking; parallel and distributed data-structures; STL-compliant interfaces and algorithms.

Castellana, Vito G.↗

DOE OSTI2020

Cyclic Prefix Direct Sequence Spread Spectrum Capacity Analysis

Cyclic Prefix Direct Sequence Spread Spectrum (CP-DSSS) is a novel waveform that has the potential to solve 5G objectives such as ultra reliable low latency communication (URLLC) and massive machine Type Communication (mMTC). This paper analyzes the capacity of the CP-DSSS waveform in comparison with Orthogonal Frequency Domain Multiplexing OFDM). CP-DSSS can be optimized to achieve the same capacity as OFDM when optimized by the water-filling algorithm. A significant advantage for CP-DSSS is that this capacity can be achieved with all symbols being transmitted with the same effective rate. As a result, stronger forward error correction codes can be used in a CP-DSSS implementation compared to an OFDM implementation with resource block constraints. In addition, the applicability of CP-DSSS as a waveform for a secondary network operating in the same frequency band as the primary network is discussed.

5G and Beyond Communications↗

DOE OSTI2021

Multi-User Capacity for Cyclic Prefix Direct Sequence Spread Spectrum with Linear Detection and Precoding

Cyclic Prefix Direct Sequence Spread Spectrum (CP-DSSS) is a promising solution for futuristic 6G ultra-reliable low latency communications (URLLC) and massive machine type communication (mMTC) applications, where the CP-DSSS waveform would operate as a secondary network at the same frequencies as the primary network but at much lower SNR. In this paper, we show per-user capacity for multi-user scenarios, where simple matched filtering (MF) is performed on the uplink (UL) and time-reversal (TR) precoding is used on the downlink (DL). When operating in the low SNR regime, CP-DSSS achieves per-user capacity near the optimum single-user capacity by using a MF detector at the receiver for the UL. TR precoding converges to the optimal capacity as the number of antennas at the hub/gateway increases. Given the near-optimal performance of MF detection and TR precoding for each of the users, CP-DSSS can be implemented with simple device transceiver structures, reducing per-unit cost for massively deployed 6G networks.

5G and Beyond Communications↗

DOE OSTI2020

Accelerated Charged Particle Tracking with Graph Neural Networks on FPGAs

We develop and study FPGA implementations of algorithms for charged particle tracking based on graph neural networks. The two complementary FPGA designs are based on OpenCL, a framework for writing programs that execute across heterogeneous platforms, and hls4ml, a high-level-synthesis-based compiler for neural network to firmware conversion. We evaluate and compare the resource usage, latency, and tracking performance of our implementations based on a benchmark dataset. We find a considerable speedup over CPU-based execution is possible, potentially enabling such algorithms to be used effectively in future computing workflows and the FPGA-based Level-1 trigger at the CERN Large Hadron Collider.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗

DOE OSTI2021

Autoencoders on FPGAs for real-time, unsupervised new physics detection at 40 MHz at the Large Hadron Collider

In this paper, we show how to adapt and deploy anomaly detection algorithms based on deep autoencoders, for the unsupervised detection of new physics signatures in the extremely challenging environment of a real-time event selection system at the Large Hadron Collider (LHC). We demonstrate that new physics signatures can be enhanced by three orders of magnitude, while staying within the strict latency and resource constraints of a typical LHC event filtering system. This would allow for collecting datasets potentially enriched with high-purity contributions from new physics processes. Through per-layer, highly parallel implementations of network layers, support for autoencoder-specific losses on FPGAs and latent space based inference, we demonstrate that anomaly detection can be performed in as little as $80\,$ns using less than 3% of the logic resources in the Xilinx Virtex VU9P FPGA. Opening the way to real-life applications of this idea during the next data-taking campaign of the LHC.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗

DOE OSTI2022

Accelerating the Inference of the Exa.TrkX Pipeline

Recently, graph neural networks (GNNs) have been successfully used for a variety of particle reconstruction problems in high energy physics, including particle tracking. The Exa.TrkX pipeline based on GNNs demonstrated promising performance in reconstructing particle tracks in dense environments. It includes five discrete steps: data encoding, graph building, edge filtering, GNN, and track labeling. All steps were written in Python and run on both GPUs and CPUs. In this work, we accelerate the Python implementation of the pipeline through customized and commercial GPU-enabled software libraries, and develop a C++ implementation for inferencing the pipeline. The implementation features an improved, CUDA-enabled fixed-radius nearest neighbor search for graph building and a weakly connected component graph algorithm for track labeling. GNNs and other trained deep learning models are converted to ONNX and inferenced via the ONNX Runtime C++ API. The complete C++ implementation of the pipeline allows integration with existing tracking software. We report the memory usage and average event latency tracking performance of our implementation applied to the TrackML benchmark dataset.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗

DOE OSTI2020

Near Optimality of Matched Filter Detection for Cyclic Prefix Direct Sequence Spread Spectrum

Abstract—Cyclic Prefix Direct Sequence Spread Spectrum -DSSS) has been presented as a potential solution for ultrareliable low latency communications (URLLC) and massive machine type communication (mMTC), where the CP-DSSS waveform would operate as a secondary network at the same frequencies as the primary network but at much lower SNR. In this paper, we show that when operating in the low SNR regime, CP-DSSS achieves near optimum performance when using a matched filter (MF) detector at the receiver. Time reversal (TR) precoding at the transmitter is also analyzed. These results also carry forward into multi-antenna scenarios where array gain is preserved. With nearly optimal performance of MF detection, CP-DSSS can be implemented with simple device transceiver structures, reducing per-unit cost for massively deployed networks.

5G and Beyond Communications↗

DOE OSTI2022

Peak Prediction Using Multi Layer Perceptron (MLP) for Edge Computing ASICs Targeting Scientific Applications

High data rate detectors play an integral part in scientific research and their development is actively pursued at High Energy Physics (HEP) facilities around the world. Edge Machine Learning (ML) offers the ability to reduce data rates by integrating ML algorithms into Application Specific Integrated Circuits (ASICs) on the front end electronics. In this work, we explore a set of neural network architectures for predicting the peak amplitudes in the detector's sensor response. We have designed and synthesized several MLP based neural networks comparing their inference accuracy, power consumption, and area targeting for minimal latency. The neural networks are synthesized in a commercial 65nm process. The effect of quantizing the network's weights and biases on hardware performance and area is reported. We also conduct design space exploration to compare between design alternatives

47 OTHER INSTRUMENTATION↗

DOE OSTI2022

Synchronous High-frequency Distributed Readout For Edge Processing At The Fermilab Main Injector And Recycler

The Main Injector (MI) was commissioned using data acquisition systems developed for the Fermilab Main Ring in the 1980s. New VME-based instrumentation was commissioned in 2006 for beam loss monitors (BLM)[2], which provided a more systematic study of the machine and improved displays of routine operation. However, current projects are demanding more data and at a faster rate from this aging hardware. One such project, Real-time Edge AI for Distributed Systems (READS), requires the high-frequency, low-latency collection of synchronized BLM readings from around the approximately two-mile accelerator complex. Significant work has been done to develop new hardware to monitor the VME backplane and broadcast BLM measurements over Ethernet, while not disrupting the existing operations critical functions of the BLM system. This paper will detail the design, implementation, and testing of this parallel data pathway.

43 PARTICLE ACCELERATORS↗

DOE OSTI2022

Real-Time Hardware-in-the-Loop Testbed to Evaluate FLISR Implemented with OpenFMB

With the increasing complexity of the distribution smart grid architecture, algorithms such as the fault location, isolation, and service restoration (FLISR) scheme rely on robust communications that are resilient to natural and man-made adverse conditions and exhibit robustness. Existing communications infrastructure for information exchange are centralized at the distribution management system, with very little autonomy or intelligence at the grid-edge. As a first step towards achieving grid-edge self-healing, this paper aims to bridge this shortcoming by implementing a centrally coordinated rules-based FLISR scheme and integrating it with Open Field Message Bus (OpenFMB), which is a flexible publish-subscribe architecture with the potential to enable point-to-multipoint communications and is more robust and resilient to natural and man-made adverse conditions. A proof of concept is developed to validate the centrally coordinated FLISR and OpenFMB mounted on an SEL-3360 computer that interacts with a simple feeder network of five SEL-651R relays, an SEL-3530 RTAC, and a hardware-in-the-loop testbed. Results demonstrate the efficacy of this approach in enabling direct, low-latency information exchange. OpenFMB's publish-subscribe data model also opens new ways to enable grid-edge interoperability among devices of different vendors interacting with different protocols.

Sundararajan, Aditya↗

DOE OSTI2022

Open-source FPGA-ML codesign for the MLPerf Tiny Benchmark

We present our development experience and recent results for the MLPerf Tiny Inference Benchmark on field-programmable gate array (FPGA) platforms. We use the open-source hls4ml and FINN workflows, which aim to democratize AI-hardware codesign of optimized neural networks on FPGAs. We present the design and implementation process for the keyword spotting, anomaly detection, and image classification benchmark tasks. The resulting hardware implementations are quantized, configurable, spatial dataflow architectures tailored for speed and efficiency and introduce new generic optimizations and common workflows developed as a part of this work. The full workflow is presented from quantization-aware training to FPGA implementation. The solutions are deployed on system-on-chip (Pynq-Z2) and pure FPGA (Arty A7-100T) platforms. The resulting submissions achieve latencies as low as 20 $\mu$s and energy consumption as low as 30 $\mu$J per inference. We demonstrate how emerging ML benchmarks on heterogeneous hardware platforms can catalyze collaboration and the development of new techniques and more accessible tools.

Borras, Hendrik↗

DOE OSTI2022

NREL Stratus - Enabling Workflows to Fuse Data Streams, Modeling, Simulation, and Machine Learning

Integrating cloud services into advanced computing facilities provides significant new capabilities over focusing solely on traditional high performance computing (HPC) workloads. This brings complementary capabilities as well as enabling new focused roles for HPC. They are especially potent for workflows that fuse data streams, modeling and simulation ('modsim') and machine learning. A key challenge to adopting a hybrid edge-cloud-HPC model is to align optimal capability, data, and user intent on the right resources for each step in a workflow.?The NREL Stratus service provides a basis for this: Stratus layers capabilities needed to make?cloud services accessible to a lab-based scientific community on commercial offerings, and; currently supports upwards of 200 projects ranging from IOT integration to traditional modeling and simulation. This provides a real-world inventory of scientific workflow elements. A growing knowledge base enables placing these elements appropriately between the edge, cloud, and traditional HPC. This paper outlines a vision via reference architecture and the application of that architecture in a typical workflow highlighting multiple components: sensor data intake, cleaning and transforming (edge/cloud suitable); generation of synthetic data through modsim, computationally heavy ML training and hyperparameter optimization (HPC suitable), and; inference and deployment (cloud ideal). Every step in such a workflow involves a cost-benefit analysis regarding the data movement, computational efficiency, availability, latency, and resource capabilities. The reference architecture and examples outlined allow for understanding new opportunities in the context of emerging workflows that combine IOT, cloud, and HPC to bolster scientific productivity.

AI↗

DOE OSTI2022

Snowmass Topical Group Summary Report: IF04 -- Trigger and Data Acquisition Systems

A trend for future high energy physics experiments is an increase in the data bandwidth produced from the detectors. Datasets of the Petabyte scale have already become the norm, and the requirements of future experiments -- greater in size, exposure, and complexity -- will further push the limits of data acquisition technologies to data rates of exabytes per seconds. The challenge for these future data-intensive physics facilities lies in the reduction of the flow of data through a combination of sophisticated event selection in the form of high-performance triggers and improved data representation through compression and calculation of high-level quantities. These tasks must be performed with low-latency (i.e. in real-time) and often in extreme environments including high radiation, high magnetic fields, and cryogenic temperatures.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗

DOE OSTI2022

Snowmass Topical Group Summary Report: IF04 -- Trigger and Data Acquisition Systems

A trend for future high energy physics experiments is an increase in the data bandwidth produced from the detectors. Datasets of the Petabyte scale have already become the norm, and the requirements of future experiments -- greater in size, exposure, and complexity -- will further push the limits of data acquisition technologies to data rates of exabytes per seconds. The challenge for these future data-intensive physics facilities lies in the reduction of the flow of data through a combination of sophisticated event selection in the form of high-performance triggers and improved data representation through compression and calculation of high-level quantities. These tasks must be performed with low-latency (i.e. in real-time) and often in extreme environments including high radiation, high magnetic fields, and cryogenic temperatures.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗

DOE OSTI2022

AWC for Nuclear Material Transportation

Presentation to the Forefront22 Workshop in DC to NNSA DNN. Generalized presentation on Advanced Wireless Communications and areas where Nuclear Material Transportation may find interesting to investigate further for security and cybersecurity interests. Summary of the Forefront22 workshop Traditional wireless communication technologies, such as Wi-Fi and Cellular, have modernized industries worldwide. The Nuclear industry has also started to modernize its operations by utilizing wireless connectivity and is well poised to expand the use of wireless to an increasing number of use cases. The most recent cellular technology, called 5G for Fifth Generation, brings a transformation not only to wireless; but to industries worldwide by fueling the growth of Internet of Things (IoT) and Industrial IoT (IIoT). Availability of additional spectrum e.g., use of millimeter wave (mmWave) spectrum bands, is a critical boost to wireless growth. These advanced wireless technologies bring multiplicative gains in data rates, latency reduction, capacity growth, and localization accuracy, that can be harnessed for Nuclear Nonproliferation activities along with other mission critical applications. However, using these functional advances securely is of utmost important specially for these critical applications. The functional capabilities of these advanced technologies including 5G & Wi-Fi 6, and the associated security issues and mitigations will be discussed in this session. The future technologies including 6G which will follow 5G and newer versions of Wi-Fi will be also introduced.

99 GENERAL AND MISCELLANEOUS↗

DOE OSTI2022

28 NREL Stratus - Enabling Workflows to Fuse Data Streams, Modeling, Simulation, and Machine Learning: Preprint

Integrating cloud services into advanced computing facilities provides significant new capabilities over focusing solely on traditional high performance computing (HPC) workloads. This brings complementary capabilities as well as enabling new focused roles for HPC. They are especially potent for workflows that fuse data streams, modeling and simulation ('modsim') and machine learning. A key challenge to adopting a hybrid edge-cloud-HPC model is to align optimal capability, data, and user intent on the right resources for each step in a workflow.?The NREL Stratus service provides a basis for this: Stratus layers capabilities needed to make?cloud services accessible to a lab-based scientific community on commercial offerings, and; currently supports upwards of 200 projects ranging from IOT integration to traditional modeling and simulation. This provides a real-world inventory of scientific workflow elements. A growing knowledge base enables placing these elements appropriately between the edge, cloud, and traditional HPC. This paper outlines a vision via reference architecture and the application of that architecture in a typical workflow highlighting multiple components: sensor data intake, cleaning and transforming (edge/cloud suitable); generation of synthetic data through modsim, computationally heavy ML training and hyperparameter optimization (HPC suitable), and; inference and deployment (cloud ideal). Every step in such a workflow involves a cost-benefit analysis regarding the data movement, computational efficiency, availability, latency, and resource capabilities. The reference architecture and examples outlined allow for understanding new opportunities in the context of emerging workflows that combine IOT, cloud, and HPC to bolster scientific productivity.

AI↗