SEARCH · Engineering Papers

Results for “streaming data”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 541 records · Page 30

All sources NASA NTRS DOE OSTI

DOE OSTI2022

Inference-Optimized AI and High Performance Computing for Gravitational Wave Detection at Scale

We introduce an ensemble of artificial intelligence models for gravitational wave detection that we trained in the Summit supercomputer using 32 nodes, equivalent to 192 NVIDIA V100 GPUs, within 2 h. Once fully trained, we optimized these models for accelerated inference using NVIDIA TensorRT. We deployed our inference-optimized AI ensemble in the ThetaGPU supercomputer at Argonne Leadership Computer Facility to conduct distributed inference. Using the entire ThetaGPU supercomputer, consisting of 20 nodes each of which has 8 NVIDIA A100 Tensor Core GPUs and 2 AMD Rome CPUs, our NVIDIA TensorRT-optimized AI ensemble processed an entire month of advanced LIGO data (including Hanford and Livingston data streams) within 50 s. Our inference-optimized AI ensemble retains the same sensitivity of traditional AI models, namely, it identifies all known binary black hole mergers previously identified in this advanced LIGO dataset and reports no misclassifications, while also providing a 3X inference speedup compared to traditional artificial intelligence models. We used time slides to quantify the performance of our AI ensemble to process up to 5 years worth of advanced LIGO data. In this synthetically enhanced dataset, our AI ensemble reports an average of one misclassification for every month of searched advanced LIGO data. We also present the receiver operating characteristic curve of our AI ensemble using this 5 year long advanced LIGO dataset. This approach provides the required tools to conduct accelerated, AI-driven gravitational wave detection at scale.

97 MATHEMATICS AND COMPUTING↗

NASA NTRS1973

Transmission of radiometer data from the Synchronous Meteorological Satellite

The Synchronous Meteorological Satellite uses a spin scanner radiometer which generates eight visual signals and two infrared signals. These signals are multiplexed and converted into a 28-Mbps data stream. This signal is transmitted to ground by quadriphase modulation at 1686.1 MHz. On the ground, the digital signal is reconstructed to an analog signal. To conserve bandwidth, an analog-to-digital converter with a nonlinear transfer function was used for the visual signals. The size of the quantization step was made proportional to the noise output of the scanner photomultiplier tube which increases as the square root of incident light. The radiometer data transmission link was simulated on a digital computer to determine the transfer function. Some results of the simulation are shown.

Davies, R. S.↗

NASA NTRS1999

Retrieval Assimilation and Modeling of Atmospheric Water Vapor from Ground- and Space-Based GPS Networks: Investigation of the Global and Regional Hydrological Cycles

Uncertainty over the response of the atmospheric hydrological cycle (particularly the distribution of water vapor and cloudiness) to anthropogenic forcing is a primary source of doubt in current estimates of global climate sensitivity, which raises severe difficulties in evaluating its likely societal impact. Fortunately, a variety of advanced techniques and sensors are beginning to shed new light on the atmospheric hydrological cycle. One of the most promising makes use of the sensitivity of the Global Positioning System (GPS) to the thermodynamic state, and in particular the water vapor content, of the atmosphere through which the radio signals propagate. Our strategy to derive the maximum benefit for hydrological studies from the rapidly increasing GPS data stream will proceed in three stages: (1) systematically analyze and archive quality-controlled retrievals using state-of-the-art techniques; (2) employ both currently available and innovative assimilation procedures to incorporate these determinations into advanced regional and global atmospheric models and assess their effects; and (3) apply the results to investigate selected scientific issues of relevance to regional and global hydrological studies. An archive of GPS-based estimation of total zenith delay (TZD) data and water vapor where applicable has been established with expanded automated quality control. The accuracy of the GPS estimates is being monitored; the investigation of systematic errors is ongoing using comparisons with water vapor radiometers. Meteorological packages have been implemented. The accuracy and utilization of the TZD estimates has been improved by implementing a troposphere gradient model. GPS-based gradients have been validated as real atmospheric moisture gradients, establishing a link between the estimated gradients and the passage of weather fronts. We have developed a generalized ray tracing inversion scheme that can be used to analyze occultation data acquired from space- or land-based receivers. The National Center for Atmospheric Research mesoscale model (version MM5) has been adapted for Southern California, and assimilation studies are underway. Additional information is contained in the original.

Dickey, Jean O.↗

DOE OSTI2020

Noise-Based Selection of Robust Inherited Model for Accurate Continual Learning

There is a growing demand for an intelligent system to continually learn knowledge from a data stream. Continual learning requires both the preservation of previous knowledge (i.e., avoiding catastrophic forgetting) and the acquisition of new knowledge. Different from previous works that focus only on model adaptation (e.g., regularization, network expansion, memory rehearsal, etc.), we propose a novel training scheme named acquisitive learning (AL), which emphasizes both the knowledge inheritance and knowledge acquisition. AL starts from an elaborately selected model with pre-trained knowledge (the inherited model) and then adapts it to new data using segmented training. The selection is achieved by injecting random noise to various inherited models for better model robustness, which promises higher accuracy in further knowledge acquisition. The approach is validated by the visualization of the loss landscape and quantitative roughness measurement. The combination of the selective inherited model and knowledge acquisition reduces catastrophic forgetting by 10X on the CIFAR-100 dataset.

Du, Xiaocong↗

DOE OSTI2023

Model Agnostic Bayesian Framework for Online Anomaly/Event Detection in PMU Data

Phasor measurement units (PMU) are integral to the modernization and automation plan of the electric power industry. A PMU data signature contains system-level events (e.g., faults, generation/load change, etc.) and any measurement/device-related errors. Therefore, the reliable and resilient operation of power systems is equivalent to the quality of the PMU data and the situation awareness provided by its data signature. Despite recent progress, current state-of-the-art methods are not fool-proof and have certain limitations tracing an error/abnormality to sensor sub-components and grid systems. This is because of technical challenges imposed by the scarcity of the labeled information, loss of data quality, and non-stationarity of data. In this paper, we consider the online PMU data stream as an output of a stochastic process and pose the anomaly/event detection as a changepoint detection problem dealing with detecting parameter changes in the underlying stochastic processes. The proposed model-agnostic framework relies on: (a) feature extraction utilizing the minimum volume enclosing ellipsoids (MVEE) method from raw PMU observations and (b) a Bayesian framework of changepoint detection. The validity of the proposed methodology is discussed through numerical experiments on real-world utility-scale PMU data.

Hossain, Ramij Raja↗

DOE OSTI2026

Multi-modal dynamic radiography using short-pulse laser-generated probe beams

Radiography is an important tool for the interrogation of dynamic experiments in the fields of dynamic properties of materials, and in condensed matter, high explosive, and high-energy-density physics. Multi-modal radiography advances the hypothesis that combining the information delivered by multiple radiographic modalities can lead to more constrained (improved) “reconstruction” of the scene than can be obtained from a single probe. We identify four modalities: multi-probe, time sequence, multi-view, and multi-messenger. Multi-probe radiography is a promising candidate for a next-generation dynamic radiographic facility. High-energy X-rays are the most frequently used probe for dynamic radiography, although recent developments show the utility of proton (pRad), electron (eRad), and neutron probe beams. Because each probing species interacts with material in the radiographic scene through quantitatively different mechanisms, each returns independent information about the scene, which can add extra constraints to the reconstruction process. How to conduct detailed, quantitative “co-analysis” of multiple data streams remains an area of active research. Multi-beam, short-pulse, laser-generated probes offer sufficient dose, an appropriate spectrum, and appropriate spatio-temporal resolution to produce high-quality dynamic radiographs. This paper reports on technology development to advance the state of the art of multi-modal/multi-probe radiography and the pursuit of both deterministic and inferential (AI/ML assisted) co-analysis methodologies to produce more constrained reconstructions from multi-modal data.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗

DOE OSTI2022

1.2.4.404 - Data-Driven Approach for Hydropower Plant Controller Prototyping Using Remote Hardware in the Loop (DR-HIL)

Real-time prototyping of hydropower plant controls is important for reducing the cost and the risk of field deployment. This project will 1) collect design and operational data from actual hydro plants and 2) use a physics-informed machine learning approach for real-time emulation of hydropower plants, including hydro turbine and hydrodynamics. The data-driven models will be interfaced with digital real-time simulation at NREL’s Flatirons campus for hardware-in-the-loop (HIL) testing of the governor hardware device or controller-HIL (CHIL). The proposed approach will also establish the connectivity based remote CHIL testing capability using real-time data streams from an actual hydro plant. This integrated hydro-plant emulation with CHIL will be used to prototype hydro-governor controls and eventually provide an opportunity to test hydropower integrated with various technologies (e.g. conventional and renewable generation, energy conversion, etc.) as HIL.

controls prototyping↗

DOE OSTI2023

Signal Processing Based Method for Real-Time Anomaly Detection in High-Performance Computing

Performance anomalies can manifest as irregular execution times or abnormal execution events for many reasons, including network congestion and resource contention. Detecting such anomalies in real-time by analyzing the details of performance traces at scale is impractical due to the sheer volume of data High-Performance Computing (HPC) applications produce. In this paper, we propose formulating HPC performance anomaly detection as a signal-processing problem where anomalies can be treated as noise. We evaluate our proposed method in comparison with two other commonly used anomaly detection techniques of varying complexity based on their detection accuracy and scalability. Since real-time in-situ anomaly detection at a large scale requires lightweight methods that can handle a large volume of streaming data, we find that our proposed method provides the best trade-off. We then implement the proposed method in Chimbuko, the first online, distributed, and scalable workflow-level performance trace analysis framework. We compare our proposed signal-based anomaly detection algorithm with two other methods using a function of their accuracy, F1 score, and detection overhead. Our experiments demonstrate that our proposed approach achieves a 99% improvement for the benchmark datasets and a 93% improvement with Chimbuko traces.

99 GENERAL AND MISCELLANEOUS↗

NASA NTRS2002

Implementation of a 622 Mbps Digital Modem

This paper presents the implementation and initial test results of an Orthogonal Frequency Division Multiplexing (OFDM) digital modem (modulator and demodulator) with an aggregate information throughput of 622 megabits per second (Mbps). The OFDM waveform is constructed by dividing an incoming data stream into four channels, each channel using either a 16-ary Quadrature Amplitude Modulation (16QAM) scheme or an 8-Phase Shift Keying (8PSK) scheme. The generation and detection of the composite waveform are performed using Discrete Fourier Transform (DFT) and polyphase filtering, to digitally stack and band-limit the individual carriers respectively. The four-channel OFDM approach enables the implementation of a modem that can be both power and bandwidth efficient, with sufficient parallelism to meet higher data rate goals. As a result, the OFDM modem requires only a 240 MHz bandwidth to transmit 622 Mbps. Hardware and simulation results in the form of spectrum diagrams and bit-error-rate (BER) curves are also presented in this paper.

Kifle, Muli↗

NASA NTRS2007

Adaptive Modeling of the International Space Station Electrical Power System

Software simulations provide NASA engineers the ability to experiment with spacecraft systems in a computer-imitated environment. Engineers currently develop software models that encapsulate spacecraft system behavior. These models can be inaccurate due to invalid assumptions, erroneous operation, or system evolution. Increasing accuracy requires manual calibration and domain-specific knowledge. This thesis presents a method for automatically learning system models without any assumptions regarding system behavior. Data stream mining techniques are applied to learn models for critical portions of the International Space Station (ISS) Electrical Power System (EPS). We also explore a knowledge fusion approach that uses traditional engineered EPS models to supplement the learned models. We observed that these engineered EPS models provide useful background knowledge to reduce predictive error spikes when confronted with making predictions in situations that are quite different from the training scenarios used when learning the model. Evaluations using ISS sensor data and existing EPS models demonstrate the success of the adaptive approach. Our experimental results show that adaptive modeling provides reductions in model error anywhere from 80% to 96% over these existing models. Final discussions include impending use of adaptive modeling technology for ISS mission operations and the need for adaptive modeling in future NASA lunar and Martian exploration.

Thomas, Justin Ray↗

DOE OSTI2021

Strategies for on-chip digital data compression for X-ray pixel detectors

Here, the continued desire for X-ray pixel detectors with higher frame rates will stress the ability of application-specific integrated circuit (ASIC) designers to provide sufficient off-chip bandwidth to reach continuous frame rates in the 1 MHz regime. To move from the current 10 kHz to the 1 MHz frame rate regime, ASIC designers will continue to pack as many power-hungry high-speed transceivers at the periphery of the ASIC as possible. In this paper, however, we present new strategies to make the most efficient use of the off-chip bandwidth by utilizing data compression schemes for X-ray photon-counting and charge-integrating pixel detectors. In particular, we describe a novel in-pixel compression scheme that converts from analog to digital converter units to encoded photon counts near the photon Poisson noise level and achieves a compression ratio of >1.5x independent of the dataset. In addition, we describe a simple yet efficient zero-suppression compression scheme called "zeromask" (ZM) located at the ASIC's edge before streaming data off the ASIC chip. ZM achieves average compression ratios of >4x, >7x, and >8x for high-energy X-ray diffraction, ptychography, and X-ray photon correlation spectroscopy datasets, respectively. We present the conceptual designs, register-transfer level block diagrams, and the physical ASIC implementation of these compression schemes in 65 nm CMOS. When combined, these two digital compression schemes could increase the effective off-chip bandwidth by a factor of 6-12x.

47 OTHER INSTRUMENTATION↗

NASA NTRS2014

The Phoenix TECP Relative Humidity Sensor: Revised Results

The original calibration function of the RH sensor on the Phoenix mission's Thermal and Electrical Conductivity Sensor (TECP), has been revised to correct the erroneously-published original calibration equation, to demonstrate the value of this unique data set, and to improve characterization of H2O exchange between the martian regolith and atmosphere. TECP returned two data streams, the temperature of the electronics analog board (Tb) and the digital 12-bit output of the RH sensor (DN), both of which are required to uniquely specify the H2O abundance. Because the original flight instrument calibration was performed against a pair of hygrometers that measured frost point (Tf), the revised calibration equation is also cast in terms of frost point. The choice of functional form for the calibration function is minimally constrained. A series of profiles across the calibration data cloud at constant DN and Tb does not reveal any evidence of a complex functional form. Therefore, a series of polynomials in both DN and Tb was investigated, along with several non-linear functions of DN and Tb.

Phoenix mission↗

DOE OSTI2022

Combinatorial Evaluation of Physical Feature Engineering, Classical Machine Learning, and Deep Learning Models for Synchrophasor Data at Scale

A major objective of the project was to train and evaluate the effectiveness of multiple event and anomaly detection, identification and classification deep temporal learning models for processing of real-time phasor measurement unit (PMU) data streams. A vast dataset, consisting of two years of phasor measurements from all three U.S. Interconnections, was curated and released by the Department of Energy (DOE) through Pacific Northwest National Laboratory (PNNL). The dataset also included an event log that provided event times and types (e.g. generator trips, line trips, planned service events, transformer operations, etc.). Our analysis of this dataset addressed six (6) of the eleven (11) research priorities identified in Funding Opportunity Announcement (FOA) DE-FOA-0001861 “Big Data Analysis of Synchrophasor Data” (FOA 1861). Rather than being limited to pre-determined specific algorithms, this project relied on the uniquely structured, highly performant underlying time series database capabilities of the PredictiveGrid platform to assess the vast dataset utilizing a wide variety of algorithms.

24 POWER TRANSMISSION AND DISTRIBUTION↗

NASA NTRS2015

Mission Control Operations: Employing a New High Performance Design for Communications Links Supporting Exploration Programs

The planetary exploration programs demand a totally new examination of data multiplexing, digital communications protocols and data transmission principles for both ground and spacecraft operations. Highly adaptive communications devices on‐board and on the ground must provide the greatest possible transmitted data density between deployed crew personnel, spacecraft and ground control teams. Regarding these requirements, this proposal borrows from research into quantum mechanical computing by applying the concept of a qubit, a single bit that represents 16 states, to radio frequency (RF) communications link design for exploration programs. This concept of placing multiple character values into a single data bit can easily make the evolutionary steps needed to meet exploration mission demands. To move the qubit from the quantum mechanical research laboratory into long distance RF data transmission, this proposal utilizes polarization modulation of the RF carrier signal to represent numbers from zero to fifteen. It introduces the concept of a binary‐to‐hexadecimal converter that quickly chops any data stream into 16‐bit words and connects variously polarized feedhorns to a single‐frequency radio transmitter. Further, the concept relies on development of a receiver that uses low‐noise amplifiers and an antenna array to quickly assess carrier polarity and perform hexadecimal to binary conversion. Early testbed experiments using the International Space Station (ISS) as an operations laboratory can be implemented to provide the most cost‐effective return for research investment. The improvement in signal‐to‐noise ratio while supporting greater baseband data rates that could be achieved through this concept justifies its consideration for long‐distance exploration programs.

Jackson, Dan E., Jr.↗

NASA NTRS1980

Wind tunnel investigation of the Titan Forward Skirt compartment vent from a free-stream Mach number of 0.80 to 1.96

A test was conducted to determine the flow characteristics of the Titan forward skirt compartment vent over a free stream Mach number range of 0.80 to 1.96. The vent was mounted in a flat plate and the plate was flush mounted to the tunnel side wall with coinciding center lines. Air was discharged from a duct, located on the tunnel side wall behind the plate, through a canted aft 30 deg honeycomb vent into the free stream. Data for the analysis of the Titan forward skirt compartment venting during ascent through the atmosphere are provided. Full scale simulated flight hardware, such as the honeycomb vent, duct corrugations and field joint ring were used. Boundary layer thicknesses were used to vary boundary height. The highest vent discharge coefficient for any given Mach number and vent pressure ratio generally occurred at the maximum displacement thickness. With no vent flow the static pressure in the vent region was generally less than the free stream static pressure. With vent flow, the static pressures upstream of the vent increased, and those downstream of the vent decreased.

Johns, A. L.↗

DOE OSTI2020

Planck intermediate results

We present the NPIPE processing pipeline, which produces calibrated frequency maps in temperature and polarization from data from the Planck Low Frequency Instrument (LFI) and High Frequency Instrument (HFI) using high-performance computers. NPIPE represents a natural evolution of previous Planck analysis efforts, and combines some of the most powerful features of the separate LFI and HFI analysis pipelines. For example, following the LFI 2018 processing procedure, NPIPE uses foreground polarization priors during the calibration stage in order to break scanning-induced degeneracies. Similarly, NPIPE employs the HFI 2018 time-domain processing methodology to correct for bandpass mismatch at all frequencies. In addition, NPIPE introduces several improvements, including, but not limited to: inclusion of the 8% of data collected during repointing manoeuvres; smoothing of the LFI reference load data streams; in-flight estimation of detector polarization parameters; and construction of maximally independent detector-set split maps. For component-separation purposes, important improvements include: maps that retain the CMB Solar dipole, allowing for high-precision relative calibration in higher-level analyses; well-defined single-detector maps, allowing for robust CO extraction; and HFI temperature maps between 217 and 857 GHz that are binned into 0'.9 pixels (Nside = 4096), ensuring that the full angular information in the data is represented in the maps even at the highest Planck resolutions. The net effect of these improvements is lower levels of noise and systematics in both frequency and component maps at essentially all angular scales, as well as notably improved internal consistency between the various frequency channels. Based on the NPIPE maps, we present the first estimate of the Solar dipole determined through component separation across all nine Planck frequencies. The amplitude is (3366.6 ± 2.7) μK, consistent with, albeit slightly higher than, earlier estimates. From the large-scale polarization data, we derive an updated estimate of the optical depth of reionization of τ = 0.051 ± 0.006, which appears robust with respect to data and sky cuts. There are 600 complete signal, noise and systematics simulations of the full-frequency and detector-set maps. As a Planck first, these simulations include full time-domain processing of the beam-convolved CMB anisotropies. The release of NPIPE maps and simulations is accompanied with a complete suite of raw and processed time-ordered data and the software, scripts, auxiliary data, and parameter files needed to improve further on the analysis and to run matching simulations.

79 ASTRONOMY AND ASTROPHYSICS↗

NASA NTRS2021

Concurrent Runtime Verification of Data Rich Events

This paper presents the open source runtime verification tool MESA (MEssage-based System Analysis), implemented in Scala, which supports concurrent monitors using the Actor model. Furthermore, the tool supports indexing (slicing) on the data values occurring in data-carrying events, for each individual monitor. The tool is generic in the sense that any monitoring system can be used for creating monitors. In this paper, we use the internal Scala DSL Daut for programming such in data parameterized state machines and temporal logic. To illustrate MESA/Daut, we present a case study that monitors flights from live U.S. airspace data streams, verifying that they conform to planned routes. With base in the case study, we then perform an extensive empirical study of the potential benefits from monitoring slices of a single property in concurrently executing actors. Due to the overhead of scheduling “small” actors (one for each slice or a small number of slices), it is not obvious that concurrent execution of such is beneficial. However, as a main result, we demonstrate that concurrent monitoring of slices to handle data-carrying events can provide considerable speed gains.

finite state machines↗

NASA NTRS1994

Galileo mission planning for Low Gain Antenna based operations

The Galileo mission operations concept is undergoing substantial redesign, necessitated by the deployment failure of the High Gain Antenna, while the spacecraft is on its way to Jupiter. The new design applies state-of-the-art technology and processes to increase the telemetry rate available through the Low Gain Antenna and to increase the information density of the telemetry. This paper describes the mission planning process being developed as part of this redesign. Principal topics include a brief description of the new mission concept and anticipated science return (these have been covered more extensively in earlier papers), identification of key drivers on the mission planning process, a description of the process and its implementation schedule, a discussion of the application of automated mission planning tool to the process, and a status report on mission planning work to date. Galileo enhancements include extensive reprogramming of on-board computers and substantial hard ware and software upgrades for the Deep Space Network (DSN). The principal mode of operation will be onboard recording of science data followed by extended playback periods. A variety of techniques will be used to compress and edit the data both before recording and during playback. A highly-compressed real-time science data stream will also be important. The telemetry rate will be increased using advanced coding techniques and advanced receivers. Galileo mission planning for orbital operations now involves partitioning of several scarce resources. Particularly difficult are division of the telemetry among the many users (eleven instruments, radio science, engineering monitoring, and navigation) and allocation of space on the tape recorder at each of the ten satellite encounters. The planning process is complicated by uncertainty in forecast performance of the DSN modifications and the non-deterministic nature of the new data compression schemes. Key mission planning steps include quantifying resource or capabilities to be allocated, prioritizing science observations and estimating resource needs for each, working inter-and intra-orbit trades of these resources among the Project elements, and planning real-time science activity. The first major mission planning activity, a high level, orbit-by-orbit allocation of resources among science objectives, has already been completed; and results are illustrated in the paper. To make efficient use of limited resources, Galileo mission planning will rely on automated mission planning tools capable of dealing with interactions among time-varying downlink capability, real-time science and engineering data transmission, and playback of recorded data. A new generic mission planning tool is being adapted for this purpose.

Gershman, R.↗