Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Streaming data”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 739 records · Page 41

A snapshot in time - the new TRUplanner database

A new database was developed to augment transuranic (TRU) waste inventory information already captured within the Comprehensive Inventory Database (CID). This new database, known as the TRUplanner, includes information focused on site capability for TRU waste certification and availability to ship to either the Waste Isolation Pilot Plant (WIPP) or Idaho National Laboratory for final characterization and certification before coming to WIPP. Waste stream specific data is imported from the CID to the TRUplanner, and additional data is collected and entered as a result of interviews and site visits. This data is continuously updated as sites are visited.

54 ENVIRONMENTAL SCIENCES↗

Real Time implementation of Artificial Intelligence compression algorithm for High-Speed Streaming Readout signals

The new generation of high-energy physics experiments plans to acquire data in streaming mode. With this approach, it is possible to access the information of the whole detector (organized in time slices) for optimal and lossless triggering of data acquisitions. With this approach, data rates, especially in large detectors, are often very high, and the network is likely to be the bottleneck for the entire Streaming Read Out system. The aim of this work is to study the implementation of a lossy compression algorithm based on Artificial Intelligence: an Autoencoder. With Machine Learning it is possible to achieve a high compression ratio and fast inference time with only a small degradation of the signals, almost negligible for the specific application. This work explores different configurations of the Autoencoder and the implementation on different hardware. Different Autoencoder configurations are explored to find the best trade-off between compression ratio and reconstruction loss, both for signals and energy spectrum. Different hardware implementations are also explored to find the best platform to achieve real-time performance for the specific application.

Rossi, Fabio (ORCID:0009000385713885)↗

A Larger Extent for the Ophiuchus Stream

We present new kinematic data for the Ophiuchus stellar stream. Spectra have been taken of member candidates at the MMT telescope using Hectospec, Hectochelle, and Binospec, which provide more than 1800 new velocities. Combined with proper-motion measurements of stars in the field by the Gaia—DR2 catalog, we have derived stream membership probabilities, resulting in the detection of more than 200 likely members. These data show the stream extends to more than three times the length shown in the discovery data. A spur to the main stream is also detected. The high-resolution spectra allow us to resolve the stellar velocity dispersion, found to be 1.6 ± 0.3 km s{sup −1}.

79 ASTRONOMY AND ASTROPHYSICS↗

High speed, real-time, camera bandwidth converter

Image data from a CMOS sensor with 10 bit resolution is reformatted in real time to allow the data to stream through communications equipment that is designed to transport data with 8 bit resolution. The incoming image data has 10 bit resolution. The communication equipment can transport image data with 8 bit resolution. Image data with 10 bit resolution is transmitted in real-time, without a frame delay, through the communication equipment by reformatting the image data.

97 MATHEMATICS AND COMPUTING↗

Encryption of Signal Pulses to Replace Tamper-indicating Conduit

In order to verify signal integrity and point of origin for TTL pulse data used in IAEA systems, and to avoid the need for expensive tamper-indicating conduit or electronic techniques, we propose the development of a signal pulse signing and encryption in-line device. In measurement applications in which the data acquisition electronics is separate from the enclosed, sealed detector, tamper-indicating techniques are required to protect raw TTL pulse streams between the detector and the data acquisition module, i.e UMSR. The goal of this proposed project is to design a rad-tolerant transmitter that would mount inside the sealed detector system and a receiver in the sealed electronics cabinet with the data acquisition instrument. This transmitter/receiver pair would digitally sign and encrypt the pulse stream data at the detector then transmit the data to the sealed cabinet where the receiver would decrypt the data and reproduce the original pulse stream. Existing tamper indicating techniques, such as LiveWire’s spread spectrum time domain reflectometry rely on detecting physical changes to the wiring system and can be blind to fast coupling of micro-second wide pulses. Digital signing and encryption techniques such as the Sandia Laboratories Enhanced Data Authentication System (EDAS) are capable of encrypting communications data, i.e. RS-232, but are not capable reproducing a critical time correlated data streams. Recent NA-241 Safeguards Technology supported developments have reduced the need for special conduit to transmit data via Ethernet by incorporating the IAEA RAINSTORM data encryption and authentication protocol, a tamper indicator is still required to protect raw pulse data from detectors to the acquisition electronics. Encrypting pulse data is especially complicated for radiation detection instruments due to the time correlation data analysis that is performed on this data stream. Any corruption in the timing information will produce errors in measurement values.

97 MATHEMATICS AND COMPUTING↗

Long-Term Data Reveal Patterns and Controls on Stream Water Chemistry in a Forested Stream: Walker Branch, Tennessee

We present 20 years of weekly stream water chemistry, hydrology, and climate data for the Walker Branch watershed in eastern Tennessee, USA. Since 1989, the watershed has experienced a similar to 1.08 degrees C increase in mean annual temperature, a similar to 20% decline in precipitation, and a similar to 30% increase in forest evapotranspiration rates. As a result, stream runoff has declined by similar to 34%. We evaluate long-term trends in stream water concentrations and fluxes for nine solutes and use wet deposition data to calculate approximate watershed input-output budgets. Dissolved constituents were classified as geochemical solutes (Ca2+, Mg2+, and SO42-) or nutrients (NH4+, NO3-, soluble reactive phosphorus [SRP], total soluble nitrogen [TSN], total soluble phosphorus [TSP], and dissolved organic carbon [DOC]). Geochemical solutes are predominantly controlled by discharge, and the long-term changes in catchment hydrology have led to significant trends in the concentrations and fluxes of these solutes. Further, the trends in geochemical solute concentrations indicate shifting soil flowpath contributions to streamflow generation through time, with deep groundwater having a greater proportional contribution in recent years. Despite dramatic changes in watershed runoff, there were no trends in inorganic nutrient concentrations (NH4+, NO3-, and SRP). While most nutrients entering the watershed are retained, stream fluxes of nutrient solutes have declined significantly as a result of decreasing runoff. Nutrient concentrations in the stream exhibit large seasonality controlled by in-stream biological uptake. Stream benthic communities are sensitive to hydrologic disturbance, and changes in the frequency or intensity of storm events through time can affect nutrient fluxes. Stream NO3- concentrations are also sensitive to drought, with concentrations decreasing (increasing) if conditions during the three years prior to the time of sampling were drier (wetter) than the long-term mean. Future changes in the incidence of storm events, as well as the number and duration of droughts, have the potential to significantly alter watershed nutrient losses. Our analysis indicates that changing climates can differentially affect watershed element cycles either through changes in biogeochemical process rates or through changes in catchment hydrology. Furthermore, climate change can include both long-term trending in mean climate variables, as well as changes in the frequency and intensity of storms and droughts, with each of these types of change having distinct effects on the biological and geochemical processes governing different solutes.

Lutz, Brian D↗

Regex-Stream

Log files are typically semi-or un-structured. To be useable, they need to be parsed into a standard, structured format. Regex-Stream facilitates parsing text files into structured data (JSON) in streams of data.

Goodall, John↗

Regex-Stream

Log files are typically semi-or un-structured. To be useable, they need to be parsed into a standard, structured format. Regex-Stream facilitates parsing text files into structured data (JSON) in streams of data.

Goodall, John↗

Joint Modeling of GD-1 and C-19 as Old Streams

DESI observational data for the GD-1 and C-19 streams are compared to stream simulations in a common evolving multi-halo potential of a Milky Way-like galaxy based on a cosmological simulation. The goal is to find the best match of the stream velocity spread and the density power spectrum stream density to simulations having either CDM or WDM subhalos. The cocoon velocity width integrated over the length of the stream is independent of orbital blurring along the stream and the power spectrum integrates over the width of the stream, sidestepping the geometric details of the streams. Streams develop from star clusters inserted at $\simeq$1 Gyr after the Big Bang and evolved for 13 Gyr to their current orbital positions. Streams in a CDM subhalo population provide the best match to the velocity width, with streams younger than 10 Gyr ruled out as insufficiently hot. The progenitor star cluster masses, which determine the fraction of stars released at late times which comprise the stream core, are found to be $\simeq 8\times 10^4 M_\odot$ for GD-1 and $\simeq 4\times 10^4 M_\odot$ for C-19, although the mass depends on the star cluster half mass radius. Stream heating leads to stream lumpiness which is measurable for the relatively large and clean GD-1 dataset. The stream density power spectrum measured along the length of the DESI GD-1 sample is in good agreement with CDM simulations, with 1.7 to 1.9 times more power than WDM 7 keV and 5.5 keV simulations.

Carlberg, Raymond G. [Toronto U.] (ORCID:000000027↗

FPGA based demodulation of laser induced fluorescence in plasmas

We present a field programmable gate array (FPGA)-based system that counts photons from laser-induced fluorescence (LIF) on a laboratory plasma. This is accomplished with FPGA-based up/down counters that demodulate the data, giving a background-subtracted LIF signal stream that is updated with a new point as each laser amplitude modulation cycle completes. We demonstrate using the FPGA to modulate a laser at 1 MHz and demodulate the resulting LIF data stream. Finally, this data stream is used to calculate an LIF-based measurement sampled at 1 MHz of a plasma ion fluctuation spectrum.

47 OTHER INSTRUMENTATION↗

Note: Optical receiver system for 152-channel magnetoencephalography

An optical receiver system composing 13 serial data restore/synchronizer modules and a single module combiner converted optical 32-bit serial data into 32-bit synchronous parallel data for a computer to acquire 152-channel magnetoencephalography (MEG) signals. A serial data restore/synchronizer module identified 32-bit channel-voltage bits from 48-bit streaming serial data, and then consecutively reproduced 13 times of 32-bit serial data, acting in a synchronous clock. After selecting a single among 13 reproduced data in each module, a module combiner converted it into 32-bit parallel data, which were carried to 32-port digital input board in a computer. When the receiver system together with optical transmitters were applied to 152-channel superconducting quantum interference device sensors, this MEG system maintained a field noise level of 3 fT/√Hz @ 100 Hz at a sample rate of 1 kSample/s per channel.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗

Performance of the Ultra-High Rate Germanium (UHRGe) System

This report describes the final performance achieved with the detector system developed for the Ultra High Rate Germanium (UHRGe) project. The system performance has been evaluated at low, moderate and high rates and includes the performance of real-time analysis algorithms running in the FPGA of the data acquisition system. This performance is compared to that of offline analyses of streaming waveform data collected with the same data acquisition system the performance of a commercial Multi-Channel Analyzer designed for high-resolution spectroscopy applications, the Canberra LYNX.

47 OTHER INSTRUMENTATION↗

Bio-energy feedstock yields and their water quality benefits in Mississippi

Cellulosic and agricultural bio-energy crops can, under careful management, be harvested as feedstock for bio-fuels production and provide environmental benefits. However, it is required to quantify their relative advantages in feedstock production and water quality. The primary objective of this research was to evaluate potential feedstock yield and water quality benefit scenarios of bioenergy crops: Miscanthus (Miscanthus-giganteus), Switchgrass (Panicum virgatum), Johnsongrass (Sorghum halepense), Alfalfa (Medicago sativa L.), Soybean {Glycine max (L.) Merr.}, and Corn (Lea mays) in the Upper Pearl River watershed (UPRW), Mississippi using a Soil and Water Assessment Tool (SWAT). The SWAT model was calibrated (January 1981 to December 1994) and validated (January 1995 to September 2008) using monthly measured stream flow data. The calibrated and validated model determined good to very good performance for stream flow prediction (R2 and E from 0.60 to 0.86). The RMSE values (from 14 m3 s-1 to 37 m3 s-1) were estimated at similar levels of errors during model calibration and validation. The long-term average annual potential feedstock yield as an alternative energy source was determined the greatest when growing Miscanthus grass (373,849 Mg) as followed by Alfalfa (206,077 Mg), Switchgrass (132,077 Mg), Johnsongrass (47,576 Mg), Soybean (37,814 Mg), and Corn (22,069 Mg) in the pastureland and cropland of the watershed. Model results determined that average annual sediment yield from the Miscanthus grass scenario determined the least (1.16 Mg/ha) and corn scenario the greatest (12.04 Mg/ha). The SWAT model simulated results suggested that growing Miscanthus grass in the UPRW would have the greatest potential feedstock yield and water quality benefits.

Biofuels↗

Towards adaptive, streaming analysis of x-ray tomography data

Temporal and spatial resolution of chemical imaging methodologies such as x-ray tomography are rapidly increasing, leading to more complex experimental procedures and fast growing data volumes. Automated analysis pipelines and big data analytics are becoming essential to effectively evaluate the results of such experiments. Offering those data techniques in an adaptive, streaming environment can further substantially improve the scientific discovery process, by enabling experimental control and steering based on the evaluation of emerging phenomena as they are observed by the experiment. Pacific Northwest National Laboratory (PNNL)’ Chemical Imaging Initiative (CII - http://imaging.pnnl.gov/ ) has worked since 2011 towards developing a framework that allows users to rapidly compose and customize high throughput experimental analysis pipelines for multiple instrument types. The framework, named ‘Rapid Experimental Analysis’ (REXAN) Framework [1], is based on the idea of reusable component libraries and utilizes the PNNL developed collaborative data management and analysis environment ‘Velo’, to provide a user friendly analysis and data management environment for experimental facilities. This article will, discuss the capabilities established for X-Ray tomography, discuss lessons learned, and provide an overview of our more recent work in the Analysis in Motion Initiative (AIM - http://aim.pnnl.gov/ ) at PNNL to provide REXAN capabilities in a streaming environment.

analysis framework↗

Random Forest Regressor-Based Approach for Detecting Fault Location and Duration in Power Systems

Power system failures or outages due to short-circuits or “faults” can result in long service interruptions leading to significant socio-economic consequences. It is critical for electrical utilities to quickly ascertain fault characteristics, including location, type, and duration, to reduce the service time of an outage. Existing fault detection mechanisms (relays and digital fault recorders) are slow to communicate the fault characteristics upstream to the substations and control centers for action to be taken quickly. Fortunately, due to availability of high-resolution phasor measurement units (PMUs), more event-driven solutions can be captured in real time. In this paper, we propose a data-driven approach for determining fault characteristics using samples of fault trajectories. A random forest regressor (RFR)-based model is used to detect real-time fault location and its duration simultaneously. This model is based on combining multiple uncorrelated trees with state-of-the-art boosting and aggregating techniques in order to obtain robust generalizations and greater accuracy without overfitting or underfitting. Four cases were studied to evaluate the performance of RFR: 1. Detecting fault location (case 1), 2. Predicting fault duration (case 2), 3. Handling missing data (case 3), and 4. Identifying fault location and length in a real-time streaming environment (case 4). A comparative analysis was conducted between the RFR algorithm and state-of-the-art models, including deep neural network, Hoeffding tree, neural network, support vector machine, decision tree, naive Bayesian, and K-nearest neighborhood. Experiments revealed that RFR consistently outperformed the other models in detection accuracy, prediction error, and processing time.

24 POWER TRANSMISSION AND DISTRIBUTION↗

Multi-Level Anomaly Detection on Time-Varying Graph Data

This work presents a novel modeling and analysis framework for graph sequences which addresses the challenge of detecting and contextualizing anomalies in labelled, streaming graph data. We introduce a generalization of the BTER model of Seshadhri et al. by adding flexibility to community structure, and use this model to perform multi-scale graph anomaly detection. Specifically, probability models describing coarse subgraphs are built by aggregating probabilities at finer levels, and these closely related hierarchical models simultaneously detect deviations from expectation. This technique provides insight into a graph's structure and internal context that may shed light on a detected event. Additionally, this multi-scale analysis facilitates intuitive visualizations by allowing users to narrow focus from an anomalous graph to particular subgraphs or nodes causing the anomaly. For evaluation, two hierarchical anomaly detectors are tested against a baseline Gaussian method on a series of sampled graphs. We demonstrate that our graph statistics-based approach outperforms both a distribution-based detector and the baseline in a labeled setting with community structure, and it accurately detects anomalies in synthetic and real-world datasets at the node, subgraph, and graph levels. To illustrate the accessibility of information made possible via this technique, the anomaly detector and an associated interactive visualization tool are tested on NCAA football data, where teams and conferences that moved within the league are identified with perfect recall, and precision greater than 0.786.

anomaly detection↗

Streaming Readout of the CLAS12 Forward Tagger Using TriDAS and JANA2

An effort is underway to develop streaming readout data acquisition system for the CLAS12 detector in Jefferson Lab’s experimental Hall-B. Successful beam tests were performed in the spring and summer of 2020 using a 10GeV electron beam from Jefferson Lab’s CEBAF accelerator. The prototype system combined elements of the TriDAS and CODA data acquisition systems with the JANA2 analysis/reconstruction framework. This successfully merged components that included an FPGA stream source, a distributed hit processing system, and software plugins that allowed offline analysis written in C++ to be used for online event filtering. Details of the system design and performance are presented.

Ameli, Fabrizio↗

Automated System-wide Event Detection and Classification Using Machine Learning on Synchrophasor Data

As the number of phasor measurement units (PMUs) deployed in a power system increases, and their data volume streamed to the control canter intensifies, operators are facing challenges related to the analysis of such data, which need to be observed and responded to as the measurements are displayed in the Control Room. Humans are generally unable to process such large amount of data efficiently and rapidly. There is an apparent need for automated ways to analyze the data, extract actionable information about occurrence of specific events, and characterize the events quickly and cost effectively. This paper discusses the use of machine learning (ML) to facilitate such tasks by providing automated, highly computationally efficient, and cost-effective ways of extracting actionable information from synchrophasor big data in real-time. We developed Big Data Smart (BDSmart) ML-based prototype tool for the Control Room use that automatically analyses data properties from synchrophasor system measurements taken across the three grid Interconnections in the USA (Western, Eastern and ERCOT). The data collected from several hundreds of PMUs located across the Interconnections over a period of two years have been made available for our extensive study. As a result, we were able to identify a number of big data properties that influence how ML methodology is applied to select, develop, train and test the data models that can eventually be used for the tool implementation. The resulting set of candidate algorithms spans unsupervised, supervised, semi-supervised and transfer-learning approaches. Many ML techniques, such as decision trees, multinomial logistic regression, feed-forward neural networks, K-nearest neighbor, multiclass support vector machine, and single and multi-channel convolutional neural networks, are implemented, and their performance is examined. We offer the results from testing the data models. The novelty of our study is in the approaches for bad data detection and mitigation, selection of a simplified feature for event detection, and data label improvements. As a result, we came up with a list of recommendations for the utilities on how to improve the PMU recording practices to cater to the future ML applications aimed at automating the analysis of synchrophasor data.

Synchrophasors, Machine Learning, System-wide Even↗