Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “streaming data”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 55 records · Page 3
All sourcesNASA NTRSDOE OSTI
DOE OSTI2022

EdgeAI: Machine learning via direct attached accelerator for streaming data processing at high shot rate x-ray free-electron lasers

We present a case for low batch-size inference with the potential for adaptive training of a lean encoder model. We do so in the context of a paradigmatic example of machine learning as applied in data acquisition at high data velocity scientific user facilities such as the Linac Coherent Light Source-II x-ray Free-Electron Laser. We discuss how a low-latency inference model operating at the data acquisition edge can capitalize on the naturally stochastic nature of such sources. We simulate the method of attosecond angular streaking to produce representative results whereby simulated input data reproduce high-resolution ground truth probability distributions. By minimizing the mean-squared error between the decoded output of the latent representation and the ground truth distributions, we ensure that the encoding layers and resulting latent representation maintains full fidelity for any downstream task, be it classification or regression. We present throughput results for data-parallel inference of various batch sizes, some with throughput exceeding 100 k images per second. We also show in situ training below 10 s per epoch for the full encoder–decoder model as would be relevant for streaming and adaptive real-time data production at our nation’s scientific light sources.

97 MATHEMATICS AND COMPUTING↗
NASA NTRS1977

Continuous data stream FIFO magnetic bubble domain shift register

There is described a simple first-in, first out (FIFO) magnetic bubble domain shift register which has continuous data storage capability. In a preferred embodiment, two parallel detector branches are associated with a main storage loop. The bubbles in the storage loop are replicated toward each detector branch by passive replicators in the storage loop. Annihilators associated with each of the replicators and each of the detectors are arranged an appropriate distance from the replicators so that selective energization of the annihilators permits certain bits to propagate to one detector and other bits to propagate to the other detector. Connection of the detectors in an appropriate bridge circuit permits full data rate reclamation.

Chen, Thomas T.↗
NASA NTRS2018

Enriching the Twitter Stream Increasing Data Mining Yield and Quality Using Machine Learning

Social media data streams are important sources of real-time and historical global information for science applications. At the NASA Goddard Earth Sciences Data and Information Services Center (GES DISC), we are exploring the Twitter data stream for its potential in augmenting the validation program of NASA Earth science missions, specifically the Global Precipitation Measurement (GPM) mission. We have implemented a tweet processing infrastructure that outputs classified precipitation tweets. Inputs are "passive" tweets, along with a smaller number of tweets from "active" participants, i.e., those knowingly contributing to our effort. The "active" tweets, presumably of higher quality, enrich the Twitter stream. "Active" sources include data scraped from other social media (e.g., public Facebook posts) and data from existing crowdsourcing programs (e.g., mPING reports). In addition, there is likely relevant precipitation information in images and documents that are the end points of links often included in tweets. Information derived from these "active" sources could then be tweeted into the Twitter stream, thus enriching its quality. The objective of our current work is to mine these tweet­ linked images and documents, using neural networks, to increase the information content and quality related to precipitation. For images, we classified them as either precipitation-related or not. For training and validation, we used images obtained via the Google custom search API. We created two models: (1) by training a simple Convolutional Neural Network and (2) by using transfer learning principles to adapt a pre-trained object recognition model. For documents, both those linked to tweets and the tweet contents, we trained Hierarchical Attention Networks to determine precipitation occurrence, type, and intensity. For training and validation, we used a keyword-filtered tweet data set labelled with ground truth data from Dark Sky (an API to retrieve weather-related labels) and the National Severe Storms Laboratory's Multi­ Radar/Multi-Sensor (MRMS) system. Our results demonstrated the efficacy of our machine learning approaches for enriching the Twitter stream, to derive information potentially useful for validation of earth science satellite data.

Albayrak, Arif↗
DOE OSTI2022

Search for long-lived particles decaying into muon pairs in proton-proton collisions at $\sqrt{s}$ = 13 TeV collected with a dedicated high-rate data stream

A search for long-lived particles decaying into muon pairs is performed using proton-proton collisions at a center-of-mass energy of 13 TeV, collected by the CMS experiment at the LHC in 2017 and 2018, corresponding to an integrated luminosity of 101 fb -1 . The data sets used in this search were collected with a dedicated dimuon trigger stream with low transverse momentum thresholds, recorded at high rate by retaining a reduced amount of information, in order to explore otherwise inaccessible phase space at low dimuon mass and nonzero displacement from the primary interaction vertex. No significant excess of events beyond the standard model expectation is found. Upper limits on branching fractions at 95% confidence level are set on a wide range of mass and lifetime hypotheses in beyond the standard model frameworks with the Higgs boson decaying into a pair of long-lived dark photons, or with a long-lived scalar resonance arising from a decay of a b hadron. The limits are the most stringent to date for substantial regions of the parameter space. These results can be also used to constrain models of displaced dimuons that are not explicitly considered in this paper.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗
NASA NTRS2001

Chip for CCSDS Compatible Serial Data Streams

A configurable service processor for telemetry ground stations is totally implemented in VLSI/ASIC hardware and finds use in spacecraft systems and other communications systems that operate according to CCSDS and CCSDS-like protocols. The service processor performs the traditional functions of data extraction at very high data and packet rates.

Jason T Dowling↗
DOE OSTI2025

A Digital Twin for an Inverter-Based Resource Power Plant: Real-time data streaming unlocks situation awareness

Here, this study presents the development and successful implementation of a digital twin specifically designed for a grid-connected IBR power plant. By integrating a reduced-order model of the IBR system and dynamically updating the grid impedance with real-time data, the digital twin effectively captures and replicates the behavior of the physical system. Its accuracy and reliability are validated through critical test scenarios, including a three-phase fault and a line-tripping event. The results confirm that the digital twin closely emulates its physical counterpart, demonstrating its strong potential for real-time analysis, system monitoring, and predictive decision making in modern power systems.

Digital twins↗
DOE OSTI2022

Investigation of Multiple Data Streams for Gearbox Bearing Fault Prediction Through Machine-Learning Models

Operations and maintenance (O&M) cost of wind plant accounts up to 30% of total energy cost, which can be reduced through continuous monitoring and successfully detecting incipient wind turbine failures. To accomplish this, condition monitoring and predictive maintenance systems are being implemented in wind industry to support O&M decision making. A wide range of approaches for condition monitoring and fault prediction have been developed. These approaches generally use historical data of wind turbines collected by Supervisory Control and Data Acquisition (SCADA) system to identify patterns that lead to failure. These SCADA data show the overall condition of a wind turbine and can be leveraged to detect when the turbine's performance is degrading and to identify if a fault is developing. However, it becomes challenging to predict the failure of a specific wind turbine gearbox bearing, because the SCADA data are often not directly linked to the component. To bridge the gap, we have investigated features calculated from SCADA data using physics-based models and the gearbox design over the years. The damaged metric we used in the physics domain is frictional energy. Combining these physics domain variables with SCADA data as inputs to various machine learning models for gearbox bearing fault prediction, we have demonstrated the benefits of leveraging both physics and data domain models. It was an attempt to improve frictional-energy-based damage metric by adding data domain inputs, as we had learned that the frictional-energy-based damage metric alone is not sufficient to single out failed bearings from healthy. As condition monitoring data (either vibration or oil debris data) has become available at more and more wind plants, we would like to evaluate whether by adding the condition monitoring data can help further improve the performance of frictional-energy-based damage metric for gearbox bearing fault prediction. Both cases by modeling through various machine learning algorithms are discussed in this study along with some observations.

fault prediction↗
DOE OSTI2023

Online real-time learning of dynamical systems from noisy streaming data

Abstract Recent advancements in sensing and communication facilitate obtaining high-frequency real-time data from various physical systems like power networks, climate systems, biological networks, etc. However, since the data are recorded by physical sensors, it is natural that the obtained data is corrupted by measurement noise. In this paper, we present a novel algorithm for online real-time learning of dynamical systems from noisy time-series data, which employs the Robust Koopman operator framework to mitigate the effect of measurement noise. The proposed algorithm has three main advantages: (a) it allows for online real-time monitoring of a dynamical system; (b) it obtains a linear representation of the underlying dynamical system, thus enabling the user to use linear systems theory for analysis and control of the system; (c) it is computationally fast and less intensive than the popular extended dynamic mode decomposition (EDMD) algorithm. We illustrate the efficiency of the proposed algorithm by applying it to identify the Van der Pol oscillator, the chaotic attractor of the Henon map, the IEEE 68 bus system, and a ring network of Van der Pol oscillators.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2022

Calculation of Nuclear Reactor Cooling Tower Performance With Limited Data Streams

Monitoring of cooling tower performance in a nuclear reactor facility is necessary to ensure safe operation; however, instrumentation for measuring performance characteristics can be difficult to install and may malfunction or break down over long duration experiments. This paper describes employing a thermodynamic approach to quantify cooling tower performance, the Merkel model, which requires only five parameters, namely, inlet water temperature, outlet water temperature, liquid mass flowrate, gas mass flowrate, and wet bulb temperature. Using this model, a general method to determine cooling tower operation for a nuclear reactor was developed in situations when neither the outlet water temperature nor gas mass flowrate are available, the former being a critical piece of information to bound the Merkel integral. Furthermore, when multiple cooling tower cells are used in parallel (as would be in the case of large-scale cooling operations), only the average outlet temperature of the cooling system is used as feedback for fan speed control, increasing the difficulty of obtaining the outlet water temperature for each cell. To address these shortcomings, this paper describes a method to obtain individual cell outlet water temperatures for mechanical forced-air cooling towers via parametric analysis and optimization. In this method, the outlet water temperature for an individual cooling tower cell is acquired as a function of the liquid-to-gas ratio (L/G). Leveraging the tight tolerance on the average outlet water temperature, an error function is generated to describe the deviation of the parameterized L/G to the highly controlled average outlet temperature. The method was able to determine the gas flowrate at rated conditions to be within 3.9% from that obtained from the manufacturer’s specification, while the average error for the four individual cooling cell outlet water temperatures were 1.6 °C, -0.5 °C, -1.0 °C, and 0.3 °C.

21 SPECIFIC NUCLEAR REACTORS AND ASSOCIATED PLANTS↗
DOE OSTI2020

Streaming Data Reorganization at Scale with DeltaFS Indexed Massive Directories

We report complex storage stacks providing data compression, indexing, and analytics help leverage the massive amounts of data generated today to derive insights. It is challenging to perform this computation, however, while fully utilizing the underlying storage media. This is because, while storage servers with large core counts are widely available, single-core performance and memory bandwidth per core grow slower than the core count per die. Computational storage offers a promising solution to this problem by utilizing dedicated compute resources along the storage processing path. We present DeltaFS Indexed Massive Directories (IMDs), a new approach to computational storage. DeltaFS IMDs harvest available (i.e., not dedicated) compute, memory, and network resources on the compute nodes of an application to perform computation on data. We demonstrate the efficiency of DeltaFS IMDs by using them to dynamically reorganize the output of a real-world simulation application across 131,072 CPU cores. DeltaFS IMDs speed up reads by 1,740x while only slightly slowing down the writing of data during simulation I/O for in situ data processing.

97 MATHEMATICS AND COMPUTING↗