SEARCH · Engineering Papers

Results for “streaming data”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 469 records · Page 26

All sources NASA NTRS DOE OSTI

DOE OSTI2024

HERMES

HERMES - High-speed Event Retrieval and Management for Enhanced Spectral imaging code. This code is meant to unpack and process neutron imaging data from the TPX3Cam made by Amsterdam Scientific Instruments. The TPX3Cam utilizes the Timepix3 chip in a single photon counting mode of image acquisition. The photon counting data streaming off the TPX3Cam needs to be process and analyzed in order to create final images. Therefore we are developing both python and cpp codes to allow for users of the TPX3Cam to efficiently analyze data and create images.

Long, Alexander↗

DOE OSTI2021

Data Reduction for Science: Brochure from the Advanced Scientific Computing Research Workshop

Data reduction for science holds promise for addressing the challenges of moving, storing, and processing massive data sets produced by the scientific community. Pursuing the PRDs outlined here will enable advances in data streaming, fast feedback and/or autonomous control of experiments, and faster time to scientific insight. These advances will result in a significant improvement in the ability to transport, store, process and interpret experimental, observational, and computational data.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2022

Status of the data acquisition, trigger, and slow control systems of the Mu2e experiment at Fermilab

The Mu2e experiment at the Fermilab will search for a coherent neutrinoless conversion of a muon into an electron in the field of an aluminum nucleus with a sensitivity improvement by a factor of 10,000 over existing limits. In this work, the Mu2e Trigger and Data Acquisition System (TDAQ) uses otsdaq framework as the online Data Acquisition System (DAQ) solution. Developed at Fermilab, otsdaq integrates several framework components — an artdaq-based DAQ, an art-based event processing, and an EPICS-based detector control system (DCS), and provides a uniform multi-user interface to its components through a web browser. Data streams from the Mu2e tracker and calorimeter are handled by the artdaq-based DAQ and processed by a one-level software trigger implemented within the art framework. Events accepted by the trigger have their data combined, post-trigger, with the separately read out data from the Mu2e Cosmic Ray Veto system. Foundation of the Mu2e DCS, EPICS – an Experimental Physics and Industrial Control System – is an open-source platform for monitoring, controlling, alarming, and archiving. A prototype of the TDAQ and the DCS systems has been built and tested over the last three years at Fermilab’s Feynman Computing Center, and now the production system installation is underway. This work presents their status and focus on the installation plans and procedures for racks, workstations, network switches, gateway computers, DAQ hardware, slow controls implementation, and testing.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗

DOE OSTI2025

Unsupervised anomaly clustering via offset alignment in multivariate grid sensing data

Modern industries increasingly rely on multi-sensor technologies to acquire complex, high-dimensional data streams, enabling advanced monitoring and control systems. One critical application is online anomaly detection in electrical smart grids, where multivariate and multimodal sensing technologies play a vital role. However, detecting anomalies in such time-series data is challenging due to their inherent temporal dependencies and stochastic behavior. Traditional approaches based on supervised and semi-supervised learning methods depend on labeled datasets, which are often unavailable in real-world scenarios. While unsupervised methods have emerged as promising alternatives, these methods are highly susceptible to noise and outliers commonly present in sensing applications. Furthermore, deep learning-based anomaly detection methods, despite their performance, are often criticized for their black-box nature, limiting their applicability in safety-critical and online environments where interpretability and explainability are paramount. In this work, we propose an unsupervised anomaly clustering method leveraging a cyclic alignment-based offset detection algorithm for multivariate time-series signals. The proposed method is applied to multivariate data collected from vibrational, voltage, and magnetic field sensors deployed in a local grid substation. Our results demonstrate the robustness of the algorithm in accurately clustering various anomalies/events across different sensing modalities. Additionally, we compare the effectiveness of the proposed approach against a simple pattern-based anomaly detection method, which performs well for univariate data but fails to generalize to multivariate and multimodal time-series data.

Mukherjee, Subrata [ORNL] (ORCID:0000000309930338)↗

DOE OSTI2024

Slow control and TDAQ systems installation and tests in the Mu2e experiment

The Mu2e experiment at Fermilab will attempt to detect a coherent neutrinoless conversion of a muon into an electron in the field of an aluminum nucleus, with a sensitivity that is 10,000 times greater than existing limits. The Mu2e trigger and data acquisition system (TDAQ) uses the otsdaqframework as its online Data Acquisition System (DAQ) solution. Developed at Fermilab, otsdaq integrates several components, such as an artdaq-based DAQ, an art-based event processing, and an EPICS-based detector control system (DCS), and provides a uniform multi-user interface toits components through a web browser. The data streams from the Mu2e tracker and calorimeter are handled by the artdaq-based DAQ and processed by a one-level software trigger implemented within the art framework. Events accepted by the trigger have their data combined, post-trigger, with the separately read-out data from the Mu2e Cosmic Ray Veto system. The foundation of Mu2e DCS, EPICS, an Experimental Physics and Industrial Control System, is an open-source platform for monitoring, controlling, alarming, and archiving. Over the last three years, a prototype ofthe TDAQ and DCS systems has been built and tested at Fermilab’s Feynman Computing Center.Currently, the production system installation is underway. At the end, this work presents a brief update on the installation of racks and DAQ hardware.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗

NASA NTRS1987

Program for Automated Real-Time Monitoring

Process-control decisions made by applying rules to process data streams. Expert System Executive computer program demonstrates feasibility of automated real-time monitoring. Using Executives knowledge of system formally coded as set of rules specifying actions taken under various conditions. Program makes inferences about observed data based on rules. Written in C language.

Goodrum, Glenn R.↗

DOE OSTI2023

Operational Focused Data Analytics for Optimizing Radiation Portal Monitor-Based Nuclear Smuggling Detection Systems at Global Ports of Entry

The National Nuclear Security Administration’s Office of Nuclear Smuggling Detection and Deterrence has deployed a fleet of radiation portal monitors (RPMs) across the world at global ports of entry including seaports, airports, and land border crossings. These RPMs are integrated into radiation detection systems (RDS) that also include fixed cameras, optical character recognition (OCR) systems, primary scanning systems (e.g., X-ray or gamma-ray), and secondary scanning systems (e.g., spectroscopic radiation portal monitors, portable radiation detection systems). The data from these sensing technologies is collected at the Central Alarm Station (CAS) where servers and computers reside to control and operate the system. Operators utilize the data collected by the CAS and declared cargo information to make decisions on how to respond to an alarm.This work explores the use of CAS-located data, looking at both the sensor data streams and operator inputs, to perform analysis which supports customs and border protection agencies to improve training capability and operational effectiveness. We focus on analyzing site level effectiveness and behavior by rolling up CAS-located data collected from individual occurrences. To-date, more than 15 sites (e.g., seaports, airports, border crossings) have been analyzed in this manner with the goal of understanding system operations to verify effectiveness and recommend potential improvements. This work first aims to provide background information on relevant CAS-located data sources and our current operational system analytics process including example results. After summarizing our current analytic techniques, we discuss how the future data analytics systems can provide key benefits to improving operational performance while minimizing the burden these detection systems place on operators.

Kuhn, Michael↗

NASA NTRS1975

Impact of remote sensing upon the planning, management, and development of water resources

Principal water resources users were surveyed to determine the impact of remote data streams on hydrologic computer models. Analysis of responses demonstrated that: most water resources effort suitable to remote sensing inputs is conducted through federal agencies or through federally stimulated research; and, most hydrologic models suitable to remote sensing data are federally developed. Computer usage by major water resources users was analyzed to determine the trends of usage and costs for the principal hydrologic users/models. The laws and empirical relationships governing the growth of the data processing loads were described and applied to project the future data loads. Data loads for ERTS CCT image processing were computed and projected through the 1985 era.

Castruccio, P. A.↗

NASA NTRS1990

A simple low-cost array on the lunar near-side for the early lunar expeditions

A concept of a simple, low-cost array on the lunar near-side is discussed. The concept includes 19 stations laid in a T, with three arms each 36 km long, and separated by 6 km. Each station consists of two 10-m-long orthogonal dipoles, each feeding a receiver of a commercial pocket digital shortwave radio type, and a 3-m spring-loaded mast with a small UHF array at the top for receiving the timing signals and tuning commmands and for sending out the digitized data streams. Each station has 0.25 square meters of solar arrays and 20 kg of lithium batteries. At the array center there is either a digital correlator for processing the signals from each antenna pair to compact the data into a 32-kb/sec channel or a 10-cm telescope for sending back raw data to earth.

Kuiper, T. B. H.↗

DOE OSTI2021

Hysteresis Patterns of Watershed Nitrogen Retention and Loss Over the Past 50 years in United States Hydrological Basins

Patterns of watershed nitrogen (N) retention and loss are shaped by how watershed biogeochemical processes retain, biogeochemically transform, and lose incoming atmospheric deposition of N. Loss patterns represented by concentration, discharge, and their associated stream exports are important indicators of integrated watershed N retention behaviors. We examined continental United States (CONUS) scale N deposition (e.g., wet and dry atmospheric deposition), vegetation trends, and stream trends as potential indicators of watershed N-saturation and retention conditions, and how watershed N retention and losses vary over space and time. By synthesizing changes and modalities in watershed nitrogen loss patterns based on stream data from 2200 U.S. watersheds over a 50 years record, our work revealed two patterns of watershed N-retention and loss. One was a hysteresis pattern that reflects the integrated influence of hydrology, atmospheric inputs, land-use, stream temperature, elevation, and vegetation. The other pattern was a one-way transition to a new state. We found that regions with increasing atmospheric deposition and increasing vegetation health/biomass patterns have the highest N-retention capacity, become increasingly N-saturated over time, and are associated with the strongest declines in stream N exports—a pattern, that is, consistent across all land cover categories. We provide a conceptual model, validated at an unprecedented scale across the CONUS that links instream nitrogen signals to upstream mechanistic landscape processes. Our work can aid in the future interpretation of in-stream concentrations of DOC and DIN as indicators of watershed N-retention status and integrators of watershed hydrobiogeochemical processes.

58 GEOSCIENCES↗

DOE OSTI2023

Quillinan, et al 2018 DOE Geothermal Technology Office REE Report for NEWTS Database and Case Studies

Produced water data processed into the NEWTS data format for easy input into aqueous chemistry modeling software, including oil & gas and coal bed methane produced waters, and geothermal waters. Includes information on rare earth elements (REEs) and critical minerals (CMs). Case studies are included to demonstrate solved streams using aqueous chemistry software. An input template is provided for modeling stream data in OLI Studio. Original data from: Quillinan, Scott, Nye, Charles, Engle, Mark, Bartos, Timothy T., Neupane, Ghanashyam, Brant, Jonathan, Bagdonas, Davin, McLing, Travis, McLaughlin, J. Fred, Phillips, Erin, Hallberg, Laura L., Shahabadi, Mahdi, and Johnson, Matthew. Assessing rare earth element concentrations in geothermal and oil and gas produced waters: A potential domestic source of strategic mineral commodities (Final Report). United States: N. p., 2018. Web. doi:10.2172/1509037.

Aqueous Chemistry↗

NASA NTRS1984

Revised Radiometric Calibration Technique for LANDSAT-4 Thematic Mapper Data by the Canada Centre for Remote Sensing

A technique for the radiometric correction of LANDSAT-4 Thematic Mapper data was proposed by the Canada Center for Remote Sensing. Subsequent detailed observations of raw image data, raw radiometric calibration data and background measurements extracted from the raw data stream on High Density Tape highlighted major shortcomings in the proposed method which if left uncorrected, can cause severe radiometric striping in the output product. Results are presented which correlate measurements of the DC background with variations in both image data background and calibration samples. The effect on both raw data and on data corrected using the earlier proposed technique is explained, and the correction required for these factors as a function of individual scan line number for each detector is described. It is shown how the revised technique can be incorporated into an operational environment.

Murphy, J.↗

NASA NTRS2001

Level 1 Processing of MODIS Direct Broadcast Data at the GSFC DAAC

The GSFC DAAC is working to test and package the MODIS Level 1 Processing software for Aqua Direct Broadcast data. This entails the same code base, but different lookup tables for Aqua and Terra. However, the most significant change is the use of ancillary attitude and ephemeris files instead of orbit/attitude information within the science data stream (as with Terra). In addition, we are working on Linux: ports of the algorithms, which could eventually enable processing on PC clusters. Finally, the GSFC DAAC is also working with the GSFC Direct Readout laboratory to ingest Level 0 data from the GSFC DB antenna into the main DAAC, enabling level 1 production in near real time in support of applications users, such as the Synergy project. The mechanism developed for this could conceivably be extended to other participating stations.

Lynnes, Christopher↗

NASA NTRS2017

Evaluation of the Ozone Fields in NASA's MERRA-2 Reanalysis

The assimilated ozone product from the Modern-Era Retrospective Analysis for Research and Applications, Version 2 (MERRA-2), produced at NASAs Global Modeling and Assimilation Office (GMAO) is summarized. The reanalysis begins in 1980 with the use of retrieved partial-column ozone concentrations from a series of Solar Backscatter Ultraviolet Radiometer (SBUV) instruments on NASA and NOAA spacecraft. Beginning in October 2004, retrieved ozone profiles from the Microwave Limb Sounder (MLS) and total column ozone from the Ozone Monitoring Instrument (OMI) on NASAs EOS Aura satellite are assimilated. While this change in data streams does lead to a discontinuity in the assimilated ozone fields in MERRA-2, making it not useful for studies in decadal (secular) trends in ozone, this choice was made to prioritize demonstrating the value NASAs high-quality research data in the reanalysis context. The MERRA-2 ozone is compared with independent satellite and ozonesonde data, focusing on the representation of the spatial and temporal variability of stratospheric and upper-tropospheric ozone. The comparisons show agreement within 10 (standard deviation of the difference) between MERRA-2 profiles and independent satellite data in most of the stratosphere. The agreement improves after 2004, when EOS Aura data are assimilated. The standard deviation of the differences between the lower-stratospheric and upper-tropospheric MERRA-2 ozone and ozonesondes is 11.2 and 24.5, respectively, with correlations of 0.8 and above. This is indicative of a realistic representation of the UTLS ozone variability in MERRA-2. After 2004, the upper tropospheric ozone in MERRA-2 shows a low bias compared to the sondes, but the covariance with independent observations is improved compared to earlier years. Case studies demonstrate the integrity of MERRA-2 analyses in representing important features such as tropopause folds.

Ozone↗

NASA NTRS2023

Interchangeable Use of GNSS and Seismic Data for Rapid Earthquake Characterization: 2021 Chignik Earthquake, Alaska

Earthquake magnitude estimation using peak ground velocities (PGV) derived from 13 Global Navigation Satellite Systems (GNSS) data has shown promise for rapid 14 characterization of damaging earthquakes. Here we examine the feasibility of using 15 GNSS-derived velocity waveforms as interchangeable data for ground motion estimation 16 and other products that typically rely on strong-motion seismic records. Our study 17 compares PGVs derived from high-rate GNSS to those computed from high-rate seismic 18 records (strong-motion and velocity), at co- and closely-located stations. The recent 2021 Manuscript Click here to access/download;Manuscript;Manuscript_Final.docx 2 19 Mw 8.2 Chignik earthquake in Alaska that was recorded on co-located GNSS and strong20 motion sensors provides the perfect opportunity to compare the two data streams and 21 their application in rapid response. The Chignik velocity records appear almost identical 22 at co-located GNSS and strong-motion stations when observed at frequencies < 0.25 Hz. 23 GNSS and strong-motion derived velocity data are further employed to generate rapid 24 estimates of PGV-derived moment magnitudes for the earthquake. The moment 25 magnitude estimates from GNSS and joint GNSS/seismic data are within ~ ±0.4 26 magnitude units (Fang et al., 2020) of the final magnitude (Mw 8.2). ShakeMaps 27 generated for the 2021 Chignik earthquake using GNSS and seismic PGVs show notable 28 agreement between them, and show negligible shifts in PGV contours when co-/closely 29 located GNSS and seismic stations are substituted for one another. Therefore, we posit 30 that GNSS is a powerful alternative or addition to seismic data and vice versa.

Earthquake rapid response↗

DOE OSTI2024

An Accurate and Automated Convective Vortex Detection Method for Long-Duration Infrasound Microbarometer Data

Heating of the surficial layer of the atmosphere often generates convective vortices, known as “dust devils” when they entrain visible debris. Convective vortices are common on both Earth and Mars, where they affect the climate via dust loading, contribute to wind erosion, impact the efficiency of photovoltaic systems, and potentially result in injury and property damage. However, long-duration terrestrial convective vortex activity records are rare. We have developed a high-precision and high-recall method to extract convective vortex signatures from infrasound microbarometer data streams. The techniques utilizes a wavelet-based detector to capture potential events and then a template matching system to extract the duration of the vortex. Since permanent and temporary infrasound sensors networks are present throughout the globe (many with open data), our method unlocks a vast new convective vortex dataset without requiring the deployment of specialized instrumentation.

54 ENVIRONMENTAL SCIENCES↗

DOE OSTI2022

INITIAL STUDIES OF CAVITY FAULT PREDICTION AT JEFFERSON LABORATORY

The Continuous Electron Beam Accelerator Facility (CEBAF) at Jefferson Laboratory is a CW recirculating linac that utilizes over 400 superconducting radio-frequency (SRF) cavities to accelerate electrons up to 12 GeV through 5-passes. Recent work has shown that, given RF signals from a cavity during a fault as input, machine learning approaches can accurately classify the fault type. In this paper we report on initial results of predicting a fault onset using only data prior to the failure event. A data set was constructed using time-series data immediately before a fault (’unstable’) and 1.5 seconds prior to a fault (’stable’) gathered from over 5,000 saved fault events. The data was used to train a binary classifier. The results gave key insights into the behavior of several fault types and provided motivation to investigate whether data prior to a failure event could also predict the type of fault. We discuss our method using a sliding window approach and report on initial results. Recent modifications to the low-level RF control system will provide access to streaming signals and we outline a path forward for leveraging deep learning on streaming data

Vidyaratne, Lasitha↗

DOE OSTI2021

INITIAL STUDIES OF CAVITY FAULT PREDICTION AT JEFFERSON LABORATORY

Vidyaratne, Lasitha↗