Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “data streams”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 775 records · Page 43

On learning what to learn: Heterogeneous observations of dynamics and establishing possibly causal relations among them

Abstract Before we attempt to (approximately) learn a function between two sets of observables of a physical process, we must first decide what the inputs and outputs of the desired function are going to be. Here we demonstrate two distinct, data-driven ways of first deciding “the right quantities” to relate through such a function, and then proceeding to learn it. This is accomplished by first processing simultaneous heterogeneous data streams (ensembles of time series) from observations of a physical system: records of multiple observation processes of the system. We determine (i) what subsets of observables are common between the observation processes (and therefore observable from each other, relatable through a function); and (ii) what information is unrelated to these common observables, therefore particular to each observation process, and not contributing to the desired function. Any data-driven technique can subsequently be used to learn the input–output relation—from k-nearest neighbors and Geometric Harmonics to Gaussian Processes and Neural Networks. Two particular “twists” of the approach are discussed. The first has to do with the identifiability of particular quantities of interest from the measurements. We now construct mappings from a single set of observations from one process to entire level sets of measurements of the second process, consistent with this single set. The second attempts to relate our framework to a form of causality: if one of the observation processes measures “now,” while the second observation process measures “in the future,” the function to be learned among what is common across observation processes constitutes a dynamical model for the system evolution.

Sroczynski, David W.↗

ROTATION MEASURE SYNTHESIS OF GALACTIC POLARIZED EMISSION WITH THE DRAO 26-m TELESCOPE

Radio polarimetry at decimeter wavelengths is the principal source of information on the Galactic magnetic field. The diffuse polarized emission is strongly influenced by Faraday rotation in the magneto-ionic medium and rotation measure (RM) is the prime quantity of interest, implying that all Stokes parameters must be measured over wide frequency bands with many frequency channels. The Dominion Radio Astrophysical Observatory (DRAO) 26 m Telescope has been equipped with a wide-band feed, a polarization transducer to deliver both hands of circular polarization, and a receiver, all operating from 1277 to 1762 MHz. Half-power beamwidth is between 40 and 30 arcmin. A digital Field-Programmable Gate Array spectrometer, based on commercially available components, produces all Stokes parameters in 2048 frequency channels over a 485-MHz bandwidth. Signals are digitized to 8 bits and a Fast Fourier Transform is applied to each data stream. Stokes parameters are then generated in each frequency channel. This instrument is in use at DRAO for a Northern sky polarization survey. Observations consist of scans up and down the Meridian at a drive rate of {approx}0.{sup 0}9 per minute to give complete coverage of the sky between declinations -30 deg. and 90 deg. This paper presents a complete description of the receiver and data acquisition system. Only a small fraction of the frequency band of operation is allocated for radio astronomy, and about 20% of the data are lost to interference. The first 8% of data from the survey are used for a proof-of-concept study, which has led to the first application of RM-Synthesis to the diffuse Galactic emission obtained with a single-antenna telescope. We find RM values for the diffuse emission as high as {approx}{+-}100 rad m{sup -2}, much higher than recorded in earlier work.

79 ASTRONOMY AND ASTROPHYSICS↗

Robust Event Classification Using Imperfect Real-world PMU Data

Here, this paper studies robust event classification using imperfect real-world phasor measurement unit (PMU) data. By analyzing the real-world PMU data, we find it is challenging to directly use this dataset for event classifiers due to the low data quality observed in PMU measurements and event logs. To address these challenges, we develop a novel machine learning framework for training robust event classifiers, which consists of three main steps: data preprocessing, fine-grained event data extraction, and feature engineering. Specifically, the data preprocessing step addresses the data quality issues of PMU measurements (e.g., bad data and missing data); in the fine-grained event data extraction step, a model-free event detection method is developed to accurately localize the events from the inaccurate event timestamps in the event logs; and the feature engineering step constructs the event features based on the patterns of different event types, in order to improve the performance and the interpretability of the event classifiers. Based on the proposed framework, we develop a workflow for event classification using the real-world PMU data streaming into the system in real time. Using the proposed framework, robust event classifiers can be efficiently trained based on many off-the-shelf lightweight machine learning models. Numerical experiments using the real-world dataset from the Western Interconnection of the U.S power transmission grid show that the event classifiers trained under the proposed framework can achieve high classification accuracy while being robust against low-quality data.

96 KNOWLEDGE MANAGEMENT AND PRESERVATION↗

Integrated Simulation Development and Decision Support Tool-Set for Utility Market and Distributed Solar Power Generation

This project will enable utilities to develop long-term strategic plans that integrate high levels of renewable energy generation, and to better plan power system operations under high renewable penetration. The program developed forecast data streams for decision support and effective integration of centralized and distributed solar power generation in utility operations. This toolset focused on real time simulation of distributed power generation within utility grids with the emphasis on potential applications in day ahead (market) and real time (reliability) utility operations. The project team developed and demonstrated methodologies for quantifying the impact of distributed solar generation on core utility operations, identified protocols for internal data communication requirements, and worked with utility personnel to adapt the new distributed generation (DG) forecasts seamlessly within existing Load and Generation procedures through a sophisticated DMS. This project supported the objectives of the SunShot Initiative and SUNRISE by enabling core utility operations to enhance their simulation capability to analyze and prepare for the impacts of high penetrations of solar on the power grid. The impact of high penetration solar PV on utility operations is not only limited to control centers, but across many core operations. Benefits of an enhanced DMS using state-of-the-art solar forecast data were demonstrated within this project and have had an immediate direct operational cost savings for Energy Marketing for Day Ahead generation commitments, Real Time Operations, Load Forecasting (at an aggregate system level for Day Ahead), Demand Response, Long term Planning (asset management), Distribution Operations, and core ancillary services as required for balancing and reliability. This provided power system operators with the necessary tools and processes to operate the grid in a reliable manner under high renewable penetration.

14 SOLAR ENERGY↗

Final Technical Report for Contract No. DE-EE0006332, "Integrated Simulation Development and Decision Support Tool-Set for Utility Market and Distributed Solar Power Generation"

This project will enable utilities to develop long-term strategic plans that integrate high levels of renewable energy generation, and to better plan power system operations under high renewable penetration. The program developed forecast data streams for decision support and effective integration of centralized and distributed solar power generation in utility operations. This toolset focused on real time simulation of distributed power generation within utility grids with the emphasis on potential applications in day ahead (market) and real time (reliability) utility operations. The project team developed and demonstrated methodologies for quantifying the impact of distributed solar generation on core utility operations, identified protocols for internal data communication requirements, and worked with utility personnel to adapt the new distributed generation (DG) forecasts seamlessly within existing Load and Generation procedures through a sophisticated DMS. This project supported the objectives of the SunShot Initiative and SUNRISE by enabling core utility operations to enhance their simulation capability to analyze and prepare for the impacts of high penetrations of solar on the power grid. The impact of high penetration solar PV on utility operations is not only limited to control centers, but across many core operations. Benefits of an enhanced DMS using state-of-the-art solar forecast data were demonstrated within this project and have had an immediate direct operational cost savings for Energy Marketing for Day Ahead generation commitments, Real Time Operations, Load Forecasting (at an aggregate system level for Day Ahead), Demand Response, Long term Planning (asset management), Distribution Operations, and core ancillary services as required for balancing and reliability. This provided power system operators with the necessary tools and processes to operate the grid in a reliable manner under high renewable penetration.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗

Classification of Nuclear Reactor Operations Using Spatial Importance and Multisensor Networks

Distributed multisensor networks record multiple data streams that can be used as inputs to machine learning models designed to classify operations relevant to proliferation at nuclear reactors. The goal of this work is to demonstrate methods to assess the importance of each node (a single multisensor) and region (a group of proximate multisensors) to machine learning model performance in a reactor monitoring scenario. This, in turn, provides insight into model behavior, a critical requirement of data-driven applications in nuclear security. Using data collected at the High Flux Isotope Reactor at Oak Ridge National Laboratory via a network of Merlyn multisensors, two different models were trained to classify the reactor’s operational state: a hidden Markov model (HMM), which is simpler and more transparent, and a feed-forward neural network, which is less inherently interpretable. Traditional wrapper methods for feature importance were extended to identify nodes and regions in the multisensor network with strong positive and negative impacts on the classification problem. These spatial-importance algorithms were evaluated on the two different classifiers. The classification accuracy was then improved relative to baseline models via feature selection from 0.583 to 0.839 and from 0.811 ± 0.005 to 0.884 ± 0.004 for the HMM and feed-forward neural network, respectively. While some differences in node and region importance were observed when using different classifiers and wrapper methods, the nodes near the facility’s cooling tower were consistently identified as important—a conclusion further supported by studies on feature importance in decision trees. Node and region importance methods are model-agnostic, inform feature selection for improved model performance, and can provide insight into opaque classification models in the nuclear security domain.

Tibbetts, Jake↗

Linking big models to big data: efficient ecosystem model calibration through Bayesian model emulation

Data-model integration plays a critical role in assessing and improving our capacity to predict ecosystem dynamics. Similarly, the ability to attach quantitative statements of uncertainty around model forecasts is crucial for model assessment and interpretation and for setting field research priorities. Bayesian methods provide a rigorous data assimilation framework for these applications, especially for problems with multiple data constraints. However, the Markov chain Monte Carlo (MCMC) techniques underlying most Bayesian calibration can be prohibitive for computationally demanding models and large datasets. We employ an alternative method, Bayesian model emulation of sufficient statistics, that can approximate the full joint posterior density, is more amenable to parallelization, and provides an estimate of parameter sensitivity. Analysis involved informative priors constructed from a meta-analysis of the primary literature and specification of both model and data uncertainties, and it introduced novel approaches to autocorrelation corrections on multiple data streams and emulating the sufficient statistics surface. We report the integration of this method within an ecological workflow management software, Predictive Ecosystem Analyzer (PEcAn), and its application and validation with two process-based terrestrial ecosystem models: SIPNET and ED2. In a test against a synthetic dataset, the emulator was able to retrieve the true parameter values. A comparison of the emulator approach to standard brute-force MCMC involving multiple data constraints showed that the emulator method was able to constrain the faster and simpler SIPNET model's parameters with comparable performance to the brute-force approach but reduced computation time by more than 2 orders of magnitude. The emulator was then applied to calibration of the ED2 model, whose complexity precludes standard (brute-force) Bayesian data assimilation techniques. Both models are constrained after assimilation of the observational data with the emulator method, reducing the uncertainty around their predictions. Performance metrics showed increased agreement between model predictions and data. Our study furthers efforts toward reducing model uncertainties, showing that the emulator method makes it possible to efficiently calibrate complex models.

58 GEOSCIENCES↗

Land Surface Reflectances from Geostationary Sensors

GEONEX is a processing pipeline that produces a suite of satellite land surface products using data streams from the latest geostationary (GEO) sensors including the GOES016/ABI and the Himawari-8/AHI. The suite, created collaboratively by scientists from NASA and NOAA, includes top-of-atmosphere (TOA) reflectances, land surface reflectances (LSRs), vegetation indices, LAI/fPAR, and other downstream products. As a key component of the GEONEX product processing, we have adapted the Multi-Angle Implementation of Atmospheric Correction (MAIAC) algorithm to produce LSRs from the TOA data. Because the algorithm depends on building "stacks" of images, we first run internal geo-registration checks to ensure geo-spatial accuracy and consistency of the input (L1B) data before transferring them from the geostationary projection into a tile system in geographic grids. Scan-time is inferred from metadata and applied to calculate the sun-sensor angles for each grid cell. The MAIAC algorithm is run to detect clouds/shadows, estimate aerosol optical thickness (AOT), perform atmospheric corrections, and generate LSRs. We have processed 18-months (from 2016/04 onward) of AHI data over East Asia and Oceania at a 10-minute time step and 10-months (from 2018/01 onward) of ABI data over North and South Americas at a 15-minute time step. As a verification measure, we compare the GEONEX (AHI/ABI) surface reflectances with the standard MODIS products (MOD09GA) and the MODIS MAIAC products over pixels that have similar sun-view geometries. The results indicate general linear relationships between GEONEX and corresponding MODIS LSRs. In particular, the RMSEs between GEONEX and MOD09 data are comparable to those between MOD09 and MODIS MAIAC products, suggesting that the uncertainties of GEONEX LSRs fall into an acceptable range. However, direct comparisons of LSRs over pixels with different sun-view angles are not as straightforward and require more modeling efforts to correct the directional effects. Evaluation of such angular influences on the downstream products (e.g., vegetation indices) is also under investigation.

Geostationary satellite; Remote Sensing; Atmospher↗

Heterogeneity and Chemical Reactivity of the Remote Troposphere Defined By Aircraft Measurements - Corrected

The NASA Atmospheric Tomography (ATom) mission built a photochemical climatology of air parcels based on in situ measurements with the NASA DC-8 aircraft along objectively planned profiling transects through the middle of the Pacific and Atlantic oceans. In this paper we present and analyze a data set of 10 s (2 km) merged and gap-filled observations of the key reactive species driving the chemical budgets of O3 and CH4 (O3, CH4, CO, H2O, HCHO, H2O2, CH3OOH, C2H6, higher alkanes, alkenes, aromatics, NOx, HNO3, HNO4, peroxyacetyl nitrate, and other organic nitrates), consisting of 146 494 distinct air parcels from ATom deployments 1 through 4. Six models calculated the O3 and CH4 photochemical tendencies from this modeling data stream for ATom 1. We find that 80 %–90 % of the total reactivity lies in the top 50 % of the parcels and 25 %–35 % in the top 10 %, supporting previous model-only studies that tropospheric chemistry is driven by a fraction of all the air. Surprisingly, the probability densities of species and reactivities averaged on a model scale (100 km) differ only slightly from the 2 km ATom 10 s data, indicating that much of the heterogeneity in tropospheric chemistry can be captured with current global chemistry models. Comparing the ATom reactivities over the tropical oceans with climatological statistics from six global chemistry models, we find generally good agreement with the reactivity rates for O3 and CH4. Models distinctly underestimate O3 production below 2 km relative to the mid-troposphere, and this can be traced to lower NOx levels than observed. Attaching photochemical reactivities to measurements of chemical species allows for a richer, yet more constrained-to-what-matters, set of metrics for model evaluation.

Tropospheric Chemistry↗

Inference-Optimized AI and High Performance Computing for Gravitational Wave Detection at Scale

We introduce an ensemble of artificial intelligence models for gravitational wave detection that we trained in the Summit supercomputer using 32 nodes, equivalent to 192 NVIDIA V100 GPUs, within 2 h. Once fully trained, we optimized these models for accelerated inference using NVIDIA TensorRT. We deployed our inference-optimized AI ensemble in the ThetaGPU supercomputer at Argonne Leadership Computer Facility to conduct distributed inference. Using the entire ThetaGPU supercomputer, consisting of 20 nodes each of which has 8 NVIDIA A100 Tensor Core GPUs and 2 AMD Rome CPUs, our NVIDIA TensorRT-optimized AI ensemble processed an entire month of advanced LIGO data (including Hanford and Livingston data streams) within 50 s. Our inference-optimized AI ensemble retains the same sensitivity of traditional AI models, namely, it identifies all known binary black hole mergers previously identified in this advanced LIGO dataset and reports no misclassifications, while also providing a 3X inference speedup compared to traditional artificial intelligence models. We used time slides to quantify the performance of our AI ensemble to process up to 5 years worth of advanced LIGO data. In this synthetically enhanced dataset, our AI ensemble reports an average of one misclassification for every month of searched advanced LIGO data. We also present the receiver operating characteristic curve of our AI ensemble using this 5 year long advanced LIGO dataset. This approach provides the required tools to conduct accelerated, AI-driven gravitational wave detection at scale.

97 MATHEMATICS AND COMPUTING↗

Real-time plasma control based on the ISTTOK tomography diagnostic

The presently available processing power in generic processing units (GPUs) combined with state-of-the-art programmable logic devices benefits the implementation of complex, real-time driven, data processing algorithms for plasma diagnostics. A tomographic reconstruction diagnostic has been developed for the ISTTOK tokamak, based on three linear pinhole cameras each with ten lines of sight. The plasma emissivity in a poloidal cross section is computed locally on a submillisecond time scale, using a Fourier-Bessel algorithm, allowing the use of the output signals for active plasma position control. The data acquisition and reconstruction (DAR) system is based on ATCA technology and consists of one acquisition board with integrated field programmable gate array (FPGA) capabilities and a dual-core Pentium module running real-time application interface (RTAI) Linux. In this paper, the DAR real-time firmware/software implementation is presented, based on (i) front-end digital processing in the FPGA; (ii) a device driver specially developed for the board which enables streaming data acquisition to the host GPU; and (iii) a fast reconstruction algorithm running in Linux RTAI. This system behaves as a module of the central ISTTOK control and data acquisition system (FIRESIGNAL). Preliminary results of the above experimental setup are presented and a performance benchmarking against the magnetic coil diagnostic is shown.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗

Transmission of radiometer data from the Synchronous Meteorological Satellite

The Synchronous Meteorological Satellite uses a spin scanner radiometer which generates eight visual signals and two infrared signals. These signals are multiplexed and converted into a 28-Mbps data stream. This signal is transmitted to ground by quadriphase modulation at 1686.1 MHz. On the ground, the digital signal is reconstructed to an analog signal. To conserve bandwidth, an analog-to-digital converter with a nonlinear transfer function was used for the visual signals. The size of the quantization step was made proportional to the noise output of the scanner photomultiplier tube which increases as the square root of incident light. The radiometer data transmission link was simulated on a digital computer to determine the transfer function. Some results of the simulation are shown.

Davies, R. S.↗

Complete information acquisition in scanning probe microscopy

In the last three decades, scanning probe microscopy (SPM) has emerged as a primary tool for exploring and controlling the nanoworld. A critical part of the SPM measurements is the information transfer from the tip-surface junction to a macroscopic measurement system. This process reduces the many degrees of freedom of a vibrating cantilever to relatively few parameters recorded as images. Similarly, the details of dynamic cantilever response at sub-microsecond time scales of transients, higher-order eigenmodes and harmonics are averaged out by transitioning to millisecond time scale of pixel acquisition. Hence, the amount of information available to the external observer is severely limited, and its selection is biased by the chosen data processing method. Here, we report a fundamentally new approach for SPM imaging based on information theory-type analysis of the data stream from the detector. This approach allows full exploration of complex tip-surface interactions, spatial mapping of multidimensional variability of material s properties and their mutual interactions, and SPM imaging at the information channel capacity limit.

77 NANOSCIENCE AND NANOTECHNOLOGY↗

Retrieval Assimilation and Modeling of Atmospheric Water Vapor from Ground- and Space-Based GPS Networks: Investigation of the Global and Regional Hydrological Cycles

Uncertainty over the response of the atmospheric hydrological cycle (particularly the distribution of water vapor and cloudiness) to anthropogenic forcing is a primary source of doubt in current estimates of global climate sensitivity, which raises severe difficulties in evaluating its likely societal impact. Fortunately, a variety of advanced techniques and sensors are beginning to shed new light on the atmospheric hydrological cycle. One of the most promising makes use of the sensitivity of the Global Positioning System (GPS) to the thermodynamic state, and in particular the water vapor content, of the atmosphere through which the radio signals propagate. Our strategy to derive the maximum benefit for hydrological studies from the rapidly increasing GPS data stream will proceed in three stages: (1) systematically analyze and archive quality-controlled retrievals using state-of-the-art techniques; (2) employ both currently available and innovative assimilation procedures to incorporate these determinations into advanced regional and global atmospheric models and assess their effects; and (3) apply the results to investigate selected scientific issues of relevance to regional and global hydrological studies. An archive of GPS-based estimation of total zenith delay (TZD) data and water vapor where applicable has been established with expanded automated quality control. The accuracy of the GPS estimates is being monitored; the investigation of systematic errors is ongoing using comparisons with water vapor radiometers. Meteorological packages have been implemented. The accuracy and utilization of the TZD estimates has been improved by implementing a troposphere gradient model. GPS-based gradients have been validated as real atmospheric moisture gradients, establishing a link between the estimated gradients and the passage of weather fronts. We have developed a generalized ray tracing inversion scheme that can be used to analyze occultation data acquired from space- or land-based receivers. The National Center for Atmospheric Research mesoscale model (version MM5) has been adapted for Southern California, and assimilation studies are underway. Additional information is contained in the original.

Dickey, Jean O.↗

Strategies to minimize the influence of instrumental bias in neutron scattering

The influence of instrumental bias on neutron scattering measurements that involve a change between two experimental states is studied analytically and computationally. Different sources of bias were applied to an unperturbed (perfect) data stream. The influence of bias on the measurement was quantified and means to mitigate it is demonstrated. We find frequent changes of state are better than fewer, and evenly alternating between changes of state is often unwise.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗

Noise-Based Selection of Robust Inherited Model for Accurate Continual Learning

There is a growing demand for an intelligent system to continually learn knowledge from a data stream. Continual learning requires both the preservation of previous knowledge (i.e., avoiding catastrophic forgetting) and the acquisition of new knowledge. Different from previous works that focus only on model adaptation (e.g., regularization, network expansion, memory rehearsal, etc.), we propose a novel training scheme named acquisitive learning (AL), which emphasizes both the knowledge inheritance and knowledge acquisition. AL starts from an elaborately selected model with pre-trained knowledge (the inherited model) and then adapts it to new data using segmented training. The selection is achieved by injecting random noise to various inherited models for better model robustness, which promises higher accuracy in further knowledge acquisition. The approach is validated by the visualization of the loss landscape and quantitative roughness measurement. The combination of the selective inherited model and knowledge acquisition reduces catastrophic forgetting by 10X on the CIFAR-100 dataset.

Du, Xiaocong↗

Model Agnostic Bayesian Framework for Online Anomaly/Event Detection in PMU Data

Phasor measurement units (PMU) are integral to the modernization and automation plan of the electric power industry. A PMU data signature contains system-level events (e.g., faults, generation/load change, etc.) and any measurement/device-related errors. Therefore, the reliable and resilient operation of power systems is equivalent to the quality of the PMU data and the situation awareness provided by its data signature. Despite recent progress, current state-of-the-art methods are not fool-proof and have certain limitations tracing an error/abnormality to sensor sub-components and grid systems. This is because of technical challenges imposed by the scarcity of the labeled information, loss of data quality, and non-stationarity of data. In this paper, we consider the online PMU data stream as an output of a stochastic process and pose the anomaly/event detection as a changepoint detection problem dealing with detecting parameter changes in the underlying stochastic processes. The proposed model-agnostic framework relies on: (a) feature extraction utilizing the minimum volume enclosing ellipsoids (MVEE) method from raw PMU observations and (b) a Bayesian framework of changepoint detection. The validity of the proposed methodology is discussed through numerical experiments on real-world utility-scale PMU data.

Hossain, Ramij Raja↗

Multi-modal dynamic radiography using short-pulse laser-generated probe beams

Radiography is an important tool for the interrogation of dynamic experiments in the fields of dynamic properties of materials, and in condensed matter, high explosive, and high-energy-density physics. Multi-modal radiography advances the hypothesis that combining the information delivered by multiple radiographic modalities can lead to more constrained (improved) “reconstruction” of the scene than can be obtained from a single probe. We identify four modalities: multi-probe, time sequence, multi-view, and multi-messenger. Multi-probe radiography is a promising candidate for a next-generation dynamic radiographic facility. High-energy X-rays are the most frequently used probe for dynamic radiography, although recent developments show the utility of proton (pRad), electron (eRad), and neutron probe beams. Because each probing species interacts with material in the radiographic scene through quantitatively different mechanisms, each returns independent information about the scene, which can add extra constraints to the reconstruction process. How to conduct detailed, quantitative “co-analysis” of multiple data streams remains an area of active research. Multi-beam, short-pulse, laser-generated probes offer sufficient dose, an appropriate spectrum, and appropriate spatio-temporal resolution to produce high-quality dynamic radiographs. This paper reports on technology development to advance the state of the art of multi-modal/multi-probe radiography and the pursuit of both deterministic and inferential (AI/ML assisted) co-analysis methodologies to produce more constrained reconstructions from multi-modal data.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗