Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “streaming algorithms”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 199 records · Page 11
All sourcesNASA NTRSDOE OSTI
DOE OSTI2020

A VXS [VITA41] Trigger Processor for the 12GEV Experimental Programs at Jefferson Lab

The VXS_Trigger_Processor [VTP] was developed and commissioned for CLAS12 in the fall of 2016. This board is a VITA41 switch card and it collects data from a variety of front-end TDC and Flash ADC modules. The VTP has since been used in several experiments at Jefferson Lab serving as the L1 trigger module for a variety of detector types, such as stacked calorimeters, strip calorimeters, time-of-flight, Cerenkov, hodoscopes, drift chambers, and silicon strips. Trigger algorithms implemented include cluster finding (1D, 2D), drift chamber segment and road finding, geometry matching between various detectors, particle counting, and general global trigger bit processing. The VTP is also capable of reading out each front-end crate with up to 40Gbps Ethernet which is an enormous increase compared to the currently used 200MB/s VME bus. Recent progress has been made to show that a firmware and software upgrade can enable existing Jefferson Lab front-end crates to operate in a streaming DAQ mode. In February 2020, tests will be performed on a full calorimeter and matched hodoscope which are components of the CLAS12 Forward Tagger detector system with beam in Hall B. This paper details the hardware performance, triggered, and streaming applications that have been implemented using the VTP for several experiments at Jefferson Lab.

ABBOTT, David↗
DOE OSTI2024

Real-time charged track reconstruction for CLAS12

Abstract This paper presents the results of charged particle track reconstruction in CLAS12 using artificial intelligence. In our approach, we use machine learning algorithms to reconstruct tracks, including their momentum and direction, with high accuracy from raw hits of the CLAS12 drift chambers. The reconstruction is performed in real-time, with the rate of data acquisition, and allows for the identification of event topologies in real-time. This approach revolutionizes the Nuclear Physics experiments' data processing, allowing us to identify and categorize the experimental data on the fly, and will lead to a significant reduction in experiment data processing. It can also be used in streaming readout applications leading to more efficient data acquisition and post-processing.

Instruments & Instrumentation↗
DOE OSTI2023

Demonstrating Cross-Facility Data Processing at Scale with Laue Microdiffraction

In February and April 2023 live, at-scale data processing demonstrations were conducted between the Advanced Photon Source (APS), a synchrotron light source, and the Argonne Leadership Computing Facility (ALCF). These tests were run as part of a novel beamline technique: coded aperture laue micro-diffraction. This technique requires a significant amount of compute to decode appeture patterns embedded in the detector stream. An autonomous system was able to send data to ALCF during an experiment, utilize 50 nodes of the Polaris supercomputer to process 6-12 hour scans, and return the data back to the APS within 12-15 minutes behind the detector. With scan points arriving every 72 seconds, the system kept up with the beamline, potentially enabling in-experiment analysis. The data processing system utilizes Globus infrastructure and an on-demand queue to dynamically acquire nodes on Polaris. The underlying reconstruction algorithms were parallelized via MPI and accelerated with custom CUDA kernels.

Prince, Michael↗
DOE OSTI2024

Scaling and performance portability of the particle-in-cell scheme for plasma physics applications through mini-apps targeting exascale architectures

We perform a scaling and performance portability study of the particle-in-cell scheme for plasma physics applications through a set of mini-apps we name "Alpine", which can make use of exascale computing capabilities. The mini-apps are based on Independent Parallel Particle Layer, a framework that is designed around performance portable and dimension independent particles and fields. We benchmark the simulations with varying parameters such as grid resolutions (5123 to 20483) and number of simulation particles (109 to 1011) with the following mini-apps: weak and strong Landau damping, bump-on-tail and two-stream instabilities, and the dynamics of an electron bunch in a charge-neutral Penning trap. We show strong and weak scaling and analyze the performance of different components on several pre-exascale architectures such as Piz-Daint, Cori, Summit and Perlmutter. While the scaling and portability study helps identify the performance critical components of the particle-in-cell scheme in the current state-of-the-art computing architectures, the mini-apps by themselves can be used to develop new algorithms and optimize their high performance implementations targeting exascale architectures.

Muralikrishnan, Sriramkrishnan↗
DOE OSTI2024

Design and optimization of a modular hydrogen-based integrated energy system to maximize revenue via nuclear-renewable sources

Here, this paper demonstrates a novel modular distributed framework that uses optimal energy-dispatching strategies to enable greater flexibility and profitability in nuclear-renewable integrated energy systems (NR-IES). Hydrogen is used as a commodity in this framework since its production can improve grid stability and system operational flexibility, decarbonize heavy industry, and create an additional revenue stream for electricity generators, particularly nuclear power plants with high operational expenses. The proposed solution addresses the challenges associated with merging multiple software and services from various domains by using functional mock-up units (FMU) to co-simulate diverse subsystems designed in various platforms. The tightly coupled integrated energy system (IES) is optimized to maximize revenue by utilizing the deep reinforcement learning (DRL) technique to make smart dispatching decisions based on variable electricity prices and the availability of renewable energy. Proximal policy optimization (PPO) algorithm is used in training and testing the DRL agent. Over a period of 120 days, the proposed hydrogen-based IES framework showed about 10% revenue boost compared to a non-hydrogen generating baseline IES while also providing an easily-adoptable framework which can help to improve the flexibility of future generation nuclear power plants.

29 ENERGY PLANNING, POLICY, AND ECONOMY↗
DOE OSTI2022

Machine Learning–Based Detection of Weather Fronts and Associated Extreme Precipitation in Historical and Future Climates

Extreme precipitation events, including those associated with weather fronts, have wide-ranging impacts across the world. Here we use a deep learning algorithm to identify weather fronts in high resolution Community Earth System Model (CESM) simulations over the contiguous United States (CONUS), and evaluate the results using observational and reanalysis products. We further compare results between CESM simulations using present-day and future climate forcing, to study how these features might change with climate change. We find that detected front frequencies in CESM have seasonally varying spatial patterns and responses to climate change and are found to be associated with modeled changes in large scale circulation such as the jet stream. We also associate the detected fronts with precipitation and find that total and extreme frontal precipitation mostly decreases with climate change, with some seasonal and regional differences. Here, decreases in Northern Hemisphere summer frontal precipitation are largely driven by changes in the frequency of different front types, especially cold and stationary fronts. On the other hand, Northern Hemisphere winter exhibits some regional increases in frontal precipitation that are largely driven by changes in frontal precipitation intensity. While CONUS mean and extreme precipitation generally increase during all seasons in these climate change simulations, the likelihood of frontal extreme precipitation decreases, demonstrating that extreme precipitation has seasonally varying sources and mechanisms that will continue to evolve with climate change.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2022

Monitoring Noble Gases (Xe and Kr) and Aerosols (Cs and Rb) in a Molten Salt Reactor Surrogate Off-Gas Stream Using Laser-Induced Breakdown Spectroscopy (LIBS)

In this study with surrogate materials we show that laser-induced breakdown spectroscopy (LIBS) is a robust tool with promising capability toward monitoring gaseous (Xe and Kr) and aerosol (Cs and Rb) species in an off-gas stream from a molten salt reactor (MSR). MSRs will continually evolve fission products into the cover gas flowing across the reactor headspace. The cover gas entrains Xe and Kr gases, along with aerosol particles, before passing into an off-gas treatment system. Univariate models of Xe and Kr peaks showed a strong correlation to concentration indicated by their coefficients of determination of 0.983 and 0.997, respectively. Multivariate models were built for all four analytes using partial least squares regression coupled with preprocessing steps including normalization, trimming, and/or genetic algorithm derived filters. The models were evaluated by predicting the concentrations of the analytes in four validation samples, in which all calibration models were successfully validated at a confidence interval of 99.9%. Finally, pressure controllers were used to regulate the mass flow rate of Kr flowing into the measurement cell in sinusoidal and stepwise waveforms to test the real-time monitoring capabilities of the regression models. Both univariate and partial least squares Kr models were able to successfully quantify the gas concentration in the real-time evaluation. The root mean squared error of prediction (RMSEP) values for these real-time tests were calculated to be 0.051, 0.060, and 0.121 mol% demonstrating the measurement systems’ capability to perform online monitoring with acceptable accuracy.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗
DOE OSTI2026

Preliminary proof-of-concept of real-time divertor heat flux control from infrared cameras with nitrogen injection in the DIII-D tokamak

In future tokamak reactors like ITER and the Fusion Pilot Plant (FPP), real-time feedback control of heat flux to the plasma-facing components (PFC) will be critical for steady-state operation. This work presents the first experimental demonstration of real-time divertor heat flux estimation with infrared thermography and feedback control with impurity seeding on the DIII-D tokamak. The flexible infrastructure of the Plasma Control System (PCS) on DIII-D makes this new capability possible. The PCS software runs on a gateway computer system, and five real-time compute nodes. An array of low latency streaming digitizers from D-TACQ Solutions connects to these real-time computers to collect and process data, and send commands to actuators during plasma discharges. This system handles the signal IO from the tokamak and allows the PCS to utilize the diagnostic data necessary to perform control in real-time. Feedback control on heat flux was accomplished by feeding infrared camera data from the “Infrared TV” (IRTV) camera to a custom-developed User Datagram Protocol (UDP) server. This server transmits infrared data to a newly developed PCS algorithm that estimates the heat flux to PFC. Here, a proportional integral derivative (PID) controller minimizes the error between a heat flux reference and the real-time estimate by injecting nitrogen gas into the divertor.

Algorithm development↗
DOE OSTI2025

IRIS-MASH: Efficient Multi-device Asynchronous Multi-Stream Heterogeneous Computing

In the rapidly evolving field of high-performance computing (HPC), effectively leveraging heterogeneous devices through asynchronous task programming is paramount. This paper presents a robust asynchronous task programming model tailored for a multi-device, multi-stream execution environment that incorporates a diverse array of heterogeneous computing units, including GPUs from various vendors and other accelerators. Current state-of-the-art task programming models provide methodologies to support asynchronous task executions, but they typically handle homogeneous devices using native programming languages, while support for heterogeneous devices is limited to frameworks like OpenCL. This gap presents significant challenges in abstracting heterogeneous devices to harness their true asynchronous capabilities effectively using their native programming languages. By implementing asynchronous task execution, our model significantly boosts the performance of tiled algorithm task graphs through overlapping data transfers with computation and enabling the simultaneous execution of multiple kernels. We integrate this approach into a heterogeneous Intelligent Runtime System (IRIS) and assess its performance using a suite of tiled algorithm benchmarks from the heterogeneous math kernels library (MatRIS) based on IRIS. Experimental results demonstrate a performance improvement ranging from 1.6 × to 2 × over IRIS without asynchronous support, and a notable 22% performance enhancement compared to established runtime systems such as StarPU and PaRSEC. This approach significantly improves computation efficiency of HPC workflows and provides a solid base for future exploration and development in the area of asynchronous task programming in heterogeneous systems.

Miniskar, Narasinga Rao [ORNL] (ORCID:000000018259↗
DOE OSTI2024

Multi-frame x-ray radiography and image tracking for quantification of expansion in laser-driven tin ejecta microjets

One regime of experimental particle-laden flow study involves ejecta microjets—often defined as a stream of micrometer-scale particles generated through shock interaction with a non-uniform surface and generally travel above 1 km/s. In order to capture the change in characteristics as a function of propagation time, we apply a multi-frame x-ray radiography platform to observe and track the jet transport dynamics. A synchrotron x-ray source allows us to perform quantitative analyses and comparisons between the eight images captured by the imaging system. Observation of a single jet through time allows the use of a cross correlation algorithm to independently track various regions within the jet and quantify the jet expansion over time using normalized area and normalized areal density values. Through a comparison with the calculated values of ballistic transport, these findings show less expansion than expected for ballistically transporting particles. In conclusion, this work combines multi-frame synchrotron radiography with image tracking to establish a foundation for future studies on jet transport and particle interaction dynamics.

Sun, Yuchen [Lawrence Livermore National Laborator↗
DOE OSTI2020

Programming Strategies for Irregular Algorithms on the Emu Chick

The Emu Chick prototype implements migratory memory-side processing in a novel hardware system. Rather than transferring large amounts of data across the system interconnect, the Emu Chick moves lightweight thread contexts to near-memory cores before the beginning of each remote memory read. Previous work has characterized the performance of the Chick prototype in terms of memory bandwidth and programming differences from more typical, non-migratory platforms, but there has not yet been an analysis of algorithms on this system. This work evaluates irregular algorithms that could benefit from the lightweight, memory-side processing of the Chick and demonstrates techniques and optimization strategies for achieving performance in sparse matrix-vector multiply operation (SpMV), breadth-first search (BFS), and graph alignment across up to eight distributed nodes encompassing 64 nodelets in the Chick system. We also define and justify relative metrics to compare prototype FPGA-based hardware with established ASIC architectures. The Chick currently supports up to 68x scaling for graph alignment, 80 MTEPS for BFS on balanced graphs, and 50% of measured STREAM bandwidth for SpMV.

Hein, Eric↗
DOE OSTI2026

Real-time plasma monitoring framework for advanced plasma control and ML-research in DIII-D

Real-time and adaptive plasma control is crucial for robust tokamak operation, requiring sensitivity and tolerance measurements of the plasma state. This paper presents the implementation of an integrated real-time plasma monitoring framework on the DIII-D tokamak to support advanced control approaches, including machine-learning (ML) methods. The system is built on the SHIELD framework, a high-performance modular architecture that provides a unified pipeline for integrating diverse diagnostics. The framework leverages high-bandwidth digitizers, fast numerical processing, and deterministic, low-latency interconnects to stream high-fidelity data from diagnostics such as electron cyclotron emission (ECE), beam emission spectroscopy (BES), CO interferometers, and a visible tangential divertor camera (TangTV). The system’s validity is demonstrated through direct comparisons of real-time and offline data. Furthermore, we present two key applications of the developed plasma monitoring system with ML-based plasma control strategies, including real-time divertor detachment and active Alfvén Eigenmode control. As a result, this work presents a robust and scalable approach for integrating high-frequency, multidimensional diagnostics into advanced control algorithms for future fusion devices.

AI/ML↗
DOE OSTI2024

RTDP: Streaming Readout Real-Time Development and Testing Platform

The Thomas Jefferson National Accelerator Facility (JLab) has created and is currently working on various tools to facilitate streaming readout (SRO) for upcoming experiments. These include reconstruction frameworks with support for Artificial Intelligence/Machine Learning, distributed High Throughput Computing (HTC), and heterogeneous computing which all contribute significantly to swift data processing and analysis. Designing SRO systems that combine such components for new experiments would benefit from a platform that would combine both simulation and execution components for simulation, testing, and validation before large investments are made. The Real-Time Development Platform (RTDP) is being developed as part of an LDRD funded project at JLab. RTDP aims to establish a seamless connection between algorithms, facilitating the seamless processing of data from SRO to analysis, as well as enabling the execution of these algorithms in various configurations on compute and data centers. Individual software components simulating specific hardware can be replaced with actual hardware when it is available.

Gyurjyan, Vardan↗
DOE OSTI2022

Multimodal sensor fusion framework for residential building occupancy detection

For several years now, smart building energy systems have been a research area of intensive activity. In light of the increasing need for sustainable buildings and energy systems, this trend motivates an increasing need for a solution to reduce carbon dioxide emissions and improve energy efficiency. This work proposes a high-performing and transferable occupancy detection framework that combines sensor data from different data modalities, including time series environmental data (temperature, humidity, and illuminance), image data, and acoustic energy data using ensemble method. To draw out the best prediction performance in each modality, the proposed framework was developed, including various models that were designed to learn the occupancy patterns reflected in the physical data streams. To tackle the time series environmental data, we designed two variants of an occupancy detection spatiotemporal pattern network (Occ-STPN) that performs both feature level and decision level fusion, respectively. We also propose a new metric; the fading memory mean square error (FMMSE), that provides a fair evaluation and penalization of delayed occupancy predictions. Multiple open-sourced datasets, including the Electricity Consumption and Occupancy and the University of California, Irvine's (UCI) building occupancy detection dataset, along with our own real data collected from six different houses, were used to validate the algorithms' performance. The experimental results presented herein break down the performance for each sensing modality, and a detailed analysis of the performance is also discussed.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗
DOE OSTI2021

System, algorithm, and method using short pulse interrogation with neutrons to detect and identify matter

A system, method, and algorithm for detecting and identifying special nuclear material (SNM) (including fissionable material), explosives, or drugs is disclosed. This material detection system relies on active interrogation of material using a short, intense neutron pulse, and characterization of the resulting prompt gamma response in two short time windows, the first concurrent with and the second immediately following the end of the neutron pulse, optionally subject to a total transit delay time of the neutrons from the neutron source to the target and from the target to the gamma detector. A high data rate analysis system implements data stream analysis and statistical correlations of the two short time window gamma responses to rapidly detect or identify SNM, explosives, or drugs. The duration of the neutron pulse is so short that minimal dose results. Various shields help to minimize the background signal falling on the gamma detector to improve sensitivity.

Derzon, Mark S.↗
DOE OSTI2022

Combinatorial Evaluation of Physical Feature Engineering, Classical Machine Learning, and Deep Learning Models for Synchrophasor Data at Scale

A major objective of the project was to train and evaluate the effectiveness of multiple event and anomaly detection, identification and classification deep temporal learning models for processing of real-time phasor measurement unit (PMU) data streams. A vast dataset, consisting of two years of phasor measurements from all three U.S. Interconnections, was curated and released by the Department of Energy (DOE) through Pacific Northwest National Laboratory (PNNL). The dataset also included an event log that provided event times and types (e.g. generator trips, line trips, planned service events, transformer operations, etc.). Our analysis of this dataset addressed six (6) of the eleven (11) research priorities identified in Funding Opportunity Announcement (FOA) DE-FOA-0001861 “Big Data Analysis of Synchrophasor Data” (FOA 1861). Rather than being limited to pre-determined specific algorithms, this project relied on the uniquely structured, highly performant underlying time series database capabilities of the PredictiveGrid platform to assess the vast dataset utilizing a wide variety of algorithms.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2022

Nonlinear spectral analysis of ion acoustic solitons arising from a streaming charged object using the numerical inverse scattering transform

Observations of nonlinear coherent plasma phenomena in spacecraft and terrestrial experiments often rely on visual identification of solitary modes because nonlinear coherent modes are broadband in a Fourier transform based analysis. We implement an alternative spectral decomposition known as the inverse scattering transform and demonstrate its ability to successfully isolate nonlinear modes on the homogeneous and forced Korteweg–De Vries equation which models these nonlinear modes. We also demonstrate for the first time that this decomposition is useful when forcing is applied. This is because the stable modes generated by localized forcing are similar to the homogeneous solutions where the inverse scattering transform is a rigorous decomposition. This spectral technique is then applied to simulations of ion acoustic waves generated by a 10 mm spherical debris object interacting with the ionospheric plasma orbiting at 2000 km altitude. The algorithm is found to successfully detect the resultant solitons. This demonstrates the feasibility of using this spectral technique as a real time analysis tool for screening spacecraft data for nonlinear solitary modes.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗
DOE OSTI2022

Numerical algorithms for water waves with background flow over obstacles and topography

Abstract We present two accurate and efficient algorithms for solving the incompressible, irrotational Euler equations with a free surface in two dimensions with background flow over a periodic, multiply connected fluid domain that includes stationary obstacles and variable bottom topography. One approach is formulated in terms of the surface velocity potential while the other evolves the vortex sheet strength. Both methods employ layer potentials in the form of periodized Cauchy integrals to compute the normal velocity of the free surface, are compatible with arbitrary parameterizations of the free surface and boundaries, and allow for circulation around each obstacle, which leads to multiple-valued velocity potentials but single-valued stream functions. We prove that the resulting second-kind Fredholm integral equations are invertible, possibly after a physically motivated finite-rank correction. In an angle-arclength setting, we show how to avoid curve reconstruction errors that are incompatible with spatial periodicity. We use the proposed methods to study gravity-capillary waves generated by flow around several elliptical obstacles above a flat or variable bottom boundary. In each case, the free surface eventually self-intersects in a splash singularity or collides with a boundary. We also show how to evaluate the velocity and pressure with spectral accuracy throughout the fluid, including near the free surface and solid boundaries. To assess the accuracy of the time evolution, we monitor energy conservation and the decay of Fourier modes and compare the numerical results of the two methods to each other. We implement several solvers for the discretized linear systems and compare their performance. The fastest approach employs a graphics processing unit (GPU) to construct the matrices and carry out iterations of the generalized minimal residual method (GMRES).

Ambrose, David M.↗