SEARCH · Engineering Papers

Results for “streaming data”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 505 records · Page 28

All sources NASA NTRS DOE OSTI

NASA NTRS2023

Statistical Decomposition and Machine Learning to Clean In Situ Spaceflight Magnetic Field Measurements

Robust in situ magnetic field measurements are critical to understanding the various mechanisms that couple mass, momentum, and energy throughout our solar system. However, the spacecraft on which magnetometers are often deployed contaminate the magnetic field measurements via onboard subsystems including reaction wheels and magnetorquers. Two magnetometers can be deployed at different distances from the spacecraft to determine an approximation of the interfering field for subsequent removal, but constant data streams from both magnetometers can be impractical due to power and telemetry limitations. Here we propose a method to identify and remove time-varying magnetic interference from sources such as reaction wheels using statistical decomposition and convolutional neural networks, providing high-fidelity magnetic field data even in cases where dual-sensor measurements are not constantly available. For example, a measurement interval from the Parker Solar Probe outboard magnetometer experienced a 95.1% reduction in reaction wheel interference following application of the proposed technique.

magnetic field data↗

DOE OSTI2020

Advanced Laboratory and Field Arrays: Evaluating Sampling Techniques for MHK Biological Monitoring (Task 6)

The overall goal of the task was to identify cost effective biological sampling techniques for MHK environmental monitoring. Protected, demersal, and pelagic fish and selected nektonic invertebrates were surveyed using capture and remote sensing techniques at the PacWave sites. The performance of capture and remote sensing monitoring techniques were evaluated and generic nekton monitoring indices were developed for MHK technologies and sites. In parallel to data collections at the PacWave site, the ability of regression models to characterize, detect, and predict change in acoustic data was evaluated using acoustic data collected in Admiralty Inlet, WA, during a biological monitoring study of the proposed SnoPud tidal turbine project site. Expected outcomes of these efforts included an evaluation of instrumentation and techniques used to monitor biological variability; identification of data streams that can be used to detect and quantify change; and sampling requirements to ensure detection of change in monitored variables.

13 HYDRO ENERGY↗

DOE OSTI2025

COMPASS-FME Terrestrial Ecosystem Manipulation to Probe the Effects of Storm Treatments (TEMPEST) Experiment Level 1 Sensor Data v2-1

This is the version 2-1 Level 1 (L1) data release for COMPASS-FME environmental sensors located at our Terrestrial Ecosystem Manipulation to Probe the Effects of Storm Treatments (TEMPEST) experimental site. This manipulative, ecosystem-scale TEMPEST experiment addresses the potential for freshwater and estuarine-water disturbance events to alter tree function, species composition, and ecosystem processes in a deciduous coastal forest in MD, USA. The experiment uses a large-unit (2000 m2), un-replicated experimental design, with three 50 m × 40 m plots serving as control, freshwater, and estuarine-water treatments. L1 data are close to raw, but are units-transformed and have out-of-instrument-bounds, out-of-service, and outlier flags added. Duplicates and missing data are removed but otherwise these data are not filtered, and have not been subject to any additional algorithmic or human QA/QC. Any scientific analyses of L1 data should be performed with care. **This dataset will be updated quarterly with new data for the duration of the project** This dataset includes: - An overall dataset README file that describes the current version, gives citation and contact information, etc. - Site- and year-specific folders, each holding variable-specific CSV (comma separated value) data files for each site and plot in that year. - Metadata files within each site-year folder provide full information on data units, expected ranges, contact information, detailed flood times, as well as a general description of the site. - Environmental sensor types that appear in the data files include weather (ClimaVUE50, CS, RM Young, and LI instruments in the graphs below); soil conditions (TEROS12); soil redox state (Redox); groundwater variables (AquaTROLL200 and AquaTROLL600); open water sondes (Exo); tree sap velocity (Sapflow); and system voltage and state (Datalogger). Data are normally logged every 15 minutes. Please see v2-1 TEMPEST L1 Sensor Package Quick Start.pdf for detailed information on data package structure, temporal coverage, and versioning. The TEMPEST flood events occurred on the following dates. They lasted for ~10 hours each day and delivered ~80,000 gallons to each plot; many data streams are available at 1 or 5 minute frequency during these periods. * Tests: Aug 25 (fresh plot) and Sep 9 (salt plot), 2021 * TEMPEST 1: June 22, 2022 * TEMPEST 2: June 6-7, 2023 * TEMPEST 3: June 11-13, 2024 This dataset was updated 2026-03-12: (i) data now go through 2025-12-31 (previous end was 2025-06-30) and (ii) dataset and file names updated to “…v2-1” (previously was “v2-0”).

54 ENVIRONMENTAL SCIENCES↗

NASA NTRS1979

Carrier phase recovery performance for PN-spread TDRSS link with radio-frequency interference

The carrier-phase recovery performance of a Costas loop for a nonlinear satellite channel with uplink and downlink noise is analyzed, and the extension of these results to the case where uplink radio-frequency interference (RFI) is present is considered. The signal format here is binary phase-shift-keyed (BPSK) or unbalanced quaternary phase-shift-keyed (UQPSK), the signals being either the non-return-to-zero (NRZ) or biphase (Manchester) type. With UQPSK, at least one data stream must be PN-spread. The channel comprises additive white Gaussian noise (WGN), a wideband filter, a bandpass nonlinearity and more additive WGN. The RFI being considered is pulsed continuous-wave, pulsed WGN, or a combination of the two. It is noted that the Tracking and Data Relay Satellite System (TDRSS) will be subjected to RFI from certain locations, especially in the S-band of frequencies. The characteristics of this link are generalized to form the assumptions underlying this analysis. The analytical results given include the S-curve of the equivalent loop and the phase-error variance of the linearized loop.

Mckenzie, T. M.↗

DOE OSTI2021

Mitigating Smart Meter Asynchrony Error Via Multi-Objective Low Rank Matrix Recovery

Not provided.

Engineering↗

DOE OSTI2022

Towards a Stronger Foundation: Digitizing Commercial Buildings with Brick to Enable Portable Advanced Applications

Most large commercial buildings have digital controls for their heating, ventilation, and air-conditioning (HVAC) and lighting systems with the potential to implement advanced control strategies and data analytics. However, advanced control strategies and data analytics are rarely deployed at scale due to non-standard naming conventions and heterogenous building configurations. Semantic metadata standards, like Brick, show promise to proliferate these applications across many buildings, but they have not been widely adopted by industry due to barriers such as perceived risk and unfamiliarity with the technology. This paper describes the workflow we established and evaluated while using it to develop over ten Brick models of existing buildings. Through this process, we observed that digitizing existing commercial buildings is a cost and labor-intensive effort in which understanding the buildings’ data streams is the major bottleneck. Yet, we conclude this investment is worthwhile since various use case applications such as fault detection and diagnostics, thermal comfort analysis, and HVAC control optimization can utilize the same Brick model. The paper also explores the challenges and lessons learned we encountered while creating these data models, such as: 1) difficulties in finding metadata descriptions and relationships for existing buildings; 2) handling missing concepts in the schema needed to model a building; 3) lack of guidance on how to structure the data model or how much detail to include; 4) unfamiliarity with technologies, which makes the learning curve steep for applications developers. Finally, we also describe future directions for semantic metadata research and development to make such transformative technologies more accessible to practitioners.

Roa, Carlos Duarte↗

NASA NTRS2014

A User Guide for Smoothing Air Traffic Radar Data

Matlab software was written to provide smoothing of radar tracking data to simulate ADS-B (Automatic Dependent Surveillance-Broadcast) data in order to test a tactical conflict probe. The probe, called TSAFE (Tactical Separation-Assured Flight Environment), is designed to handle air-traffic conflicts left undetected or unresolved when loss-of-separation is predicted to occur within approximately two minutes. The data stream that is down-linked from an aircraft equipped with an ADS-B system would include accurate GPS-derived position and velocity information at sample rates of 1 Hz. Nation-wide ADS-B equipage (mandated by 2020) should improve surveillance accuracy and TSAFE performance. Currently, position data are provided by Center radar (nominal 12-sec samples) and Terminal radar (nominal 4.8-sec samples). Aircraft ground speed and ground track are estimated using real-time filtering, causing lags up to 60 sec, compromising performance of a tactical resolution tool. Offline smoothing of radar data reduces wild-point errors, provides a sample rate as high as 1 Hz, and yields more accurate and lag-free estimates of ground speed, ground track, and climb rate. Until full ADS-B implementation is available, smoothed radar data should provide reasonable track estimates for testing TSAFE in an ADS-B-like environment. An example illustrates the smoothing of radar data and shows a comparison of smoothed-radar and ADS-B tracking. This document is intended to serve as a guide for using the smoothing software.

Center/Tracon Radar↗

DOE OSTI2021

Data-Driven Adaptive Damping Controller for Wind Power Plants with Doubly-Fed Induction Generators

This paper presents an adaptive damping controller for wind power plants in which the turbines are equipped with doubly-fed induction generators. The controller is designed to respond to an input control signal that is triggered according to the system operating conditions. A processing unit continuously estimates the electromechanical modes of oscillation based on real-time streaming data acquired from a phasor measurement unit that is strategically positioned on the grid. The decision to trigger (or not trigger) the control signal is automatic, based on the relative damping of the dominant mode. The modes are estimated using the dynamic mode decomposition algorithm with time-delay embedding. Numerical simulations performed on the two-area system demonstrate that the proposed controller enhances the rotor angle stability for both small-signal and large disturbances, and is adaptive to changing grid conditions.

damping controller↗

DOE OSTI2021

Data-Driven Adaptive Damping Controller for Wind Power Plants with Doubly-Fed Induction Generators: Preprint

61 RADIATION PROTECTION AND DOSIMETRY↗

DOE OSTI2025

A co-registered in-situ and ex-situ dataset from wire arc additive manufacturing process

Recent progress in sensing techniques and data analytics tools have significantly accelerated the development of Wire Arc Additive Manufacturing (WAAM) systems. This data-centric approach emphasizes leveraging sensor data available throughout the production process to optimize performance. Integration of extensive data analysis provides opportunities for improving precision, reducing waste, and enhancing the quality of produced parts. This method relies on AI/ML models and optimization techniques, which are developed using the data collected from various sources, including in-situ sensors, ex-situ imaging, and manufacturing process parameters. The quality and diversity of this data, along with the alignment between different data streams (achieved through spatiotemporal registration) are critical for the successful development of AI/ML and optimization models. In this work, we present a spatiotemporally registered dataset generated during the WAAM process of deposition of a rectangular block. The dataset includes a comprehensive description of the deposition process, process parameters, welding characteristics and acoustic data collected in-situ, and X-Ray Computed Tomography data of the build.

42 ENGINEERING↗

DOE OSTI2025

Classifying thermodynamic cloud phase using machine learning models

Vertically resolved thermodynamic cloud-phase classifications are essential for studies of atmospheric cloud and precipitation processes. The Department of Energy (DOE) Atmospheric Radiation Measurement (ARM) Thermodynamic Cloud Phase (THERMOCLDPHASE) value-added product (VAP) uses a multi-sensor approach to classify the thermodynamic cloud phase by combining lidar backscatter and depolarization, radar reflectivity, Doppler velocity, spectral width, microwave-radiometer-derived liquid water path, and radiosonde temperature measurements. The measured pixels are classified as ice, snow, mixed phase, liquid (cloud water), drizzle, rain, and liq_driz (liquid+drizzle). We use this product as the ground truth to train three machine learning (ML) models to predict the thermodynamic cloud phase from multi-sensor remote sensing measurements taken at the ARM North Slope of Alaska (NSA) observatory: a random forest (RF), a multi-layer perceptron (MLP), and a convolutional neural network (CNN) with a U-Net architecture. Evaluations against the outputs of the THERMOCLDPHASE VAP with 1 year of data show that the CNN outperforms the other two models, achieving the highest test accuracy, F1 score, and mean intersection over union (IOU). Analysis of ML confidence scores shows that ice, rain, and snow have higher confidence scores, followed by liquid, while mixed, drizzle, and liq_driz have lower scores. Feature importance analysis reveals that the mean Doppler velocity and vertically resolved temperature are the most influential data streams for ML thermodynamic cloud-phase predictions. Lidar measurements exhibit lower feature importance due to rapid signal attenuation caused by the frequent presence of persistent low-level clouds at the NSA site. The ML models' generalization capacity is further evaluated by applying them at another Arctic ARM site in Norway using data taken during the ARM Cold-Air Outbreaks in the Marine Boundary Layer Experiment (COMBLE) field campaign. The models demonstrated similar performance to that observed at the NSA site. Finally, we evaluate the ML models' response to simulated instrument outages and signal degradation and show that a CNN U-Net model trained with input channel dropouts performs better when input fields are missing.

ARM Aerial Facility↗

NASA NTRS1975

Aerodynamic results of a separation effects test conducted in the AEDC 40 by 40 inch tunnel A facility on the Rockwell International launch configuration 3 (model-OTS) integrated vehicle (IA13), volume 1

Experimental aerodynamic investigations were conducted from July 5 through July 17, 1973, on a 0.01 scale model. The AEDC captive trajectory system was utilized in conjunction with the tunnel primary sector to obtain grid-type data for external tank abort from the orbiter, and for nominal separation of one solid rocket booster from the orbiter-tank combination. Booster separation was investigated with and without separation motors plume simulation. The plumes were generated by eight M sub j = 2.15 nozzles using a 1500 psia cold air supply. Free stream data were obtained for all models (orbiter, tank, orbiter-tank, and right-hand booster) to provide baselines for evaluation of proximity effects.

Campbell, J. H., II↗

NASA NTRS2005

Launch Commit Criteria Monitoring Agent

The Spaceport Processing Systems Branch at NASA Kennedy Space Center has developed and deployed a software agent to monitor the Space Shuttle's ground processing telemetry stream. The application, the Launch Commit Criteria Monitoring Agent, increases situational awareness for system and hardware engineers during Shuttle launch countdown. The agent provides autonomous monitoring of the telemetry stream, automatically alerts system engineers when predefined criteria have been met, identifies limit warnings and violations of launch commit criteria, aids Shuttle engineers through troubleshooting procedures, and provides additional insight to verify appropriate troubleshooting of problems by contractors. The agent has successfully detected launch commit criteria warnings and violations on a simulated playback data stream. Efficiency and safety are improved through increased automation.

Semmel, Glenn S.↗

DOE OSTI2022

100 Gb/s High Throughput Serial Protocol (HTSP) for data acquisition systems with interleaved streaming

Demands on Field-Programmable Gate Array (FPGA) data transport have been increasing over the years as frame sizes and refresh rates increase. As the bandwidths requirements increase the ability to implement data transport protocol layers using "soft" programmable logic becomes harder and start to require harden IP blocks implementation. To reduce the number of physical links and interconnects, it is common for data acquisition systems to require interleaving of streams on the same link (e.g. streaming data and streaming register access). Further, this paper presents a way to leverage existing FPGA harden IP blocks to achieve a robust, low latency 100 Gb/s point-to-point link with minimal programmable logic overhead geared towards the needs of data acquisition systems with interleaved streaming requirements.

47 OTHER INSTRUMENTATION↗

DOE OSTI2023

HopPyBar

HopPyBar is a python program to import, analyze, and export split-Hopkinson pressure bar (SHPB, also known as Kolsky bar) data. Traditional analysis offers a black box approach, where input data is converted to analyzed output by performing a series of calculations without user involvement. This program serves as a developmental platform to "white box" the data analysis process. Data streams can be captured (in-situ) to enable advanced or unconventional analyses, statistics, and comparisons. Additionally, the program is geared towards the standardized forms of input and output used at LANL to streamline analysis, but the open nature of the program makes additional input/output schemes straightforward to add. General workflow will import SHPB data in one of a number of formats, identify relevant portions of data signals, and convert to stress-strain-strain rate to show material behavior as a function of dynamic testing.

Morrow, Benjamin↗

DOE OSTI2024

A Co-Registered In-Situ and Ex-Situ Dataset of Electrical, Acoustic, and CT Characteristics from Wire Arc Additive Manufacturing Process

Recent progress in sensing techniques and data analytics tools have significantly accelerated the development of Wire Arc Additive Manufacturing (WAAM) systems. This data centric approach emphasizes leveraging available data throughout the production process to optimize performance. Integration of extensive data analysis provides the opportunity to improve precision, reduce waste, and enhance the quality of produced parts. This method relies on AI/ML models and optimization techniques, which are developed using the data collected from various sources, including in-situ sensors, ex-situ imaging, and manufacturing process parameters. The quality and diversity of this data, along with the alignment between different data streams (achieved through spatiotemporal registration) are critical for the successful development of AI/ML and optimization models. In this work, we present a spatiotemporally registered dataset generated during the WAAM process of deposition of a rectangular block. The dataset includes the comprehensive description of deposition process, process parameters, in-situ collected welding characteristics, acoustic data, and X-Ray Computed Tomography analysis data for the build. Dataset A Co-Registered In-Situ and Ex-Situ Dataset of Electrical, Acoustic, and CT Characteristics from Wire Arc Additive Manufacturing Process has arisen under UT-Battelle, LLC’s Prime Contract No. DE-AC05-00OR22725 with the U.S. Department of Energy (DOE) to manage and operate the Oak Ridge National Laboratory. UT-Battelle, LLC will not assert any rights under United States law or under the Prime Contract it has in the dataset against any user of the dataset, including any copyrights or patent rights. UT-Battelle, LLC requests that attribution to the dataset is provided as academically appropriate.

42 ENGINEERING↗

DOE OSTI2022

Effectiveness of Privacy Techniques in Smart Metering Systems

Smart grid technologies enable timely energy billing for residential homes. The ability to react to energy demands during peak hours allows energy providers to conserve power and operate efficiently. However, these data streams are also susceptible to privacy attacks within the energy company and from outside hackers. We implemented four different privacy models: k-anonymous, l-diversity, t-closeness, and ε-differential privacy. We demonstrate the models’ effectiveness using a real-world dataset composed of 15 different residential households with energy consumption data spanning over a year.

Peralta-Peterson, Martin↗

NASA NTRSDate not supplied

AN INTRODUCTION TO THE GEONEX LEVEL-1G PRODUCTS: TOP-OF-ATMOSPHERE REFLECTANCE AND BRIGHTNESS TEMPERATURE

This paper introduces the GeoNEX (Geostationary-NASA Earth eXchange) Level-1G products of top-of-atmosphere (TOA) reflectance and brightness temperature. The products use data streams from the latest geostationary (GEO) sensors including the GOES-16/17 ABI and the Himawari-8/9 AHI. The GeoNEX processing pipeline starts by converting digital numbers to physical quantities with the latest radiometric calibration information. It integrates algorithms to automatically detect and remove residual geolocation errors, to estimate the pixel-wise data-acquisition time, and to accurately calculate the solar illumination angles for each pixel in the domain at every time step. The outputs are reprojected to a globally tiled common grid in geographic coordinates designed to facilitate inter-comparisons and/or synergies between the GeoNEX products and existing Earth observation datasets from polar-orbiting satellites. Therefore, the GeoNEX L1G products provide accurate and consistent TOA reflectance and brightness temperature datasets for scientific analyses and downstream product development.

Geostationary satellite, GOES-16, Himawari-8, NASA↗