Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “data streams”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 757 records · Page 42

Data Science and the Knowledge Discovery Adventure

This talk will cover the important steps involved in the data science and knowledge discovery process: • Initial fact gathering (interview domain experts, review reports, articles, state-of-the-art) • Identify the problem (prediction, classification, statistical analysis, etc.) • Survey supporting data sources • Understand the data (numerical, categorical, text, sampling rate, data quality issues, etc.) • Selecting relevant features and sources • Acquire the data (set up agreements with the data stewards, APIs to download, etc.) • Merge data sources (temporal, spatial, common key, other ontologies...) • Feature Engineering (non linear domain knowledge or physics-based relationships) • Build data processing pipeline (may need to tap into data stream, develop parallel processing algorithm, federated learning etc.) • Build model and test (tune hyper-parameters, cross validation.) • Analyze/Validate results (do the results make sense. Does it answer the original question). • Deploy/Publish (Monitor and assess benefits)

Data science↗

Online Distribution System State Estimation via Stochastic Gradient Algorithm

Distribution network operation is becoming more challenging because of the growing integration of intermittent and volatile distributed energy resources (DERs). This motivates the development of new distribution system state estimation (DSSE) paradigms that can operate at fast timescale based on real-time data stream of asynchronous measurements enabled by modern information and communications technology. To solve the real-time DSSE with asynchronous measurements effectively and accurately, this paper formulates a weighted least squares DSSE problem and proposes an online stochastic gradient algorithm to solve it. The performance of the proposed scheme is analytically guaranteed and is numerically corroborated with realistic data on IEEE 123-bus feeder.

distribution system state estimation↗

A burst compression and expansion technique for variable-rate users in satellite-switched TDMA networks

A burst compression and expansion technique is described for asynchronously interconnecting variable-data-rate users with cost-efficient ground terminals in a satellite-switched, time-division-multiple-access (SS/TDMA) network. Compression and expansion buffers in each ground terminal convert between lower rate, asynchronous, continuous-user data streams and higher-rate TDMA bursts synchronized with the satellite-switched timing. The technique described uses a first-in, first-out (FIFO) memory approach which enables the use of inexpensive clock sources by both the users and the ground terminals and obviates the need for elaborate user clock synchronization processes. A continous range of data rates from kilobits per second to that approaching the modulator burst rate (hundreds of megabits per second) can be accommodated. The technique was developed for use in the NASA Lewis Research Center System Integration, Test, and Evaluation (SITE) facility. Some key features of the technique have also been implemented in the gound terminals developed at NASA Lewis for use in on-orbit evaluation of the Advanced Communications Technology Satellite (ACTS) high burst rate (HBR) system.

Budinger, James M.↗

A burst compression and expansion technique for variable-rate users in satellite-switched TDMA networks

A burst compression and expansion technique is described for asynchronously interconnecting variable-data-rate users with cost-efficient ground terminals in a satellite-switched, time-division-multiple-access (SS/TDMA) network. Compression and expansion buffers in each ground terminal convert between lower rate, asynchronous, continuous-user data streams and higher-rate TDMA bursts synchronized with the satellite-switched timing. The technique described uses a first-in, first-out (FIFO) memory approach which enables the use of inexpensive clock sources by both the users and the ground terminals and obviates the need for elaborate user clock synchronization processes. A continuous range of data rates from kilobits per second to that approaching the modulator burst rate (hundreds of megabits per second) can be accommodated. The technique was developed for use in the NASA Lewis Research Center System Integration, Test, and Evaluation (SITE) facility. Some key features of the technique have also been implemented in the ground terminals developed at NASA Lewis for use in on-orbit evaluation of the Advanced Communications Technology Satellite (ACTS) high burst rate (HBR) system.

Budinger, James M.↗

Text Stream Trend Analysis using Multiscale Visual Analytics with Applications to Social Media Systems

In this paper, we introduce a new visual analytics system, called Matisse, that allows exploration of global trends in textual information streams with specific application to social media platforms. Despite the potential for real-time situational awareness using these services, interactive analysis of such semi-structured textual information is a challenge due to the high-throughput and high-velocity properties. Matisse addresses these challenges through the following contributions: (1) robust stream data management, (2) automated sen- timent/emotion analytics, (3) inferential temporal, geospatial, and term-frequency visualizations, and (4) a flexible drill-down interaction scheme that progresses from macroscale to microscale views. In addition to describing these contributions, our work-in-progress paper concludes with a practical case study focused on the analysis of Twitter 1% sample stream information captured during the week of the Boston Marathon bombings.

interaction↗

MSE based drilling optimization using neural network simulaton

The method disclosed receives a data stream from an MWD system and determines the response of a specific energy (SE) relationship and a rate of penetration (ROP) relationship respectively to variables controllable by the operator, in order to enable operation at a lowest SE, or a highest Rate-of-Penetration (ROP) to SE ratio. The method utilizes artificial neural networks trained by MWD data to deduce a depth-of-cut and torque based on relationships manifesting between the various data points collected, and an SE equation and a predicted ROP is evaluated over a series of probable operating points. The method continuously gathers and analyzes MWD data during the drilling operation and allows an operator to manage the controllable parameters such that operation at the lowest SE or highest ROP or ROP to SE ratio can be achieved during the drilling operation.

Zhang, Wu↗

System and method for forward error correction

A system and method are provided for transferring a packet across a data link. The packet may include a stream of data symbols which is delimited by one or more framing symbols. Corruptions of the framing symbol which result in valid data symbols may be mapped to invalid symbols. If it is desired to transfer one of the valid data symbols that has been mapped to an invalid symbol, the data symbol may be replaced with an unused symbol. At the receiving end, these unused symbols are replaced with the corresponding valid data symbols. The data stream of the packet may be encoded with forward error correction information to detect and correct errors in the data stream.

Cole, Robert M.↗

System and method for transferring data on a data link

A system and method are provided for transferring a packet across a data link. The packet may include a stream of data symbols which is delimited by one or more framing symbols. Corruptions of the framing symbol which result in valid data symbols may be mapped to invalid symbols. If it is desired to transfer one of the valid data symbols that has been mapped to an invalid symbol, the data symbol may be replaced with an unused symbol. At the receiving end, these unused symbols are replaced with the corresponding valid data symbols. The data stream of the packet may be encoded with forward error correction information to detect and correct errors in the data stream.

Cole, Robert M.↗

CORE (Common Operating Response Environment) Software Technology Suite

Agencies that oversee complex, multi-stakeholder programs need efficient, secure ways to link people and knowledge within and across organizations. The Common Operating Response Environment (CORE), a software suite developed by PNNL researchers does just that. The CORE tool—which is customizable for a multitude of uses—facilitates situational awareness by integrating diverse data streams without the need to reformat them, summarizing that information, and providing users with the information they need to rapidly understand and appropriately respond to situations. It is mobile device-ready, has a straightforward interface for ease of use across organizations and skill sets, and is incredibly configurable to the needs of each specific user, whether they require data summaries for high-level decision makers or tactical maps, operational data, or weather information for responders in the field. Information can be input into CORE and queried in a variety of ways—using customized forms, reports, visuals, or other organizational templates—according to the needs of each user’s organization, teams, and business processes. CORE data forms, for instance, could be accessed and used in real-time to capture information about vessels being inspected for nuclear material.

42 ENGINEERING↗

Situ

Rapidly discovering novel and sophisticated cyber attacks and providing situation aware-ness to analysts are unsolved problems in cyber security. We have developed a platform that scores events in real-time based on probabilistic models to define how typical an event is. This anomaly detection approach is based on unsupervised, probabilistic modeling of data at multiple scales. Scoring events based on multiple scales allows the system to 1) score anomalousness at different levels to detect important events that would otherwise be hidden, and 2) explain to users why an event is anomalous, not just that it is. The system was designed to address several challenges: 1) scaling to very high volume, heterogeneous, streaming data, and 2) minimizing the time from observation to discovery to understanding. The prototype has the real-time framework for pushing scored events to a web-based visualization.

Source record↗

Combined impact of lead, cadmium, polychlorinated biphenyls and non-chemical risk factors on blood pressure in NHANES

High blood pressure is associated with exposure to multiple chemical and non-chemical risk factors, but epidemiological analyses to date have not assessed the combined effects of both chemical and non-chemical stressors on human populations in the context of cumulative risk assessment. We developed a novel modeling approach to evaluate the combined impact of lead, cadmium, polychlorinated biphenyls (PCBs), and multiple non-chemical risk factors on four blood pressure measures using data for adults aged ≥20 years from the National Health and Nutrition Examination Survey (1999–2008). We developed predictive models for chemical and other stressors. Structural equation models were applied to account for complex associations among predictors of stressors as well as blood pressure. Models showed that blood lead, serum PCBs, and established non-chemical stressors were significantly associated with blood pressure. Lead was the chemical stressor most predictive of diastolic blood pressure and mean arterial pressure, while PCBs had a greater influence on systolic blood pressure and pulse pressure, and blood cadmium was not a significant predictor of blood pressure. The simultaneously fit exposure models explained 34%, 43% and 52% of the variance for lead, cadmium and PCBs, respectively. The structural equation models were developed using predictors available from public data streams (e.g., U.S. Census), which would allow the models to be applied to any U.S. population exposed to these multiple stressors in order to identify high risk subpopulations, direct intervention strategies, and inform public policy. - Highlights: • We evaluated joint impact of chemical and non-chemical stressors on blood pressure. • We built predictive models for lead, cadmium and polychlorinated biphenyls (PCBs). • Our approach allows joint evaluation of predictors from population-specific data. • Lead, PCBs and established non-chemical stressors were related to blood pressure. • Framework allows cumulative risk assessment in specific geographic settings.

54 ENVIRONMENTAL SCIENCES↗

Automatic Traffic Queue-End Identification using Location-Based Waze User Reports

Traffic queues, especially queues caused by non-recurrent events such as incidents, are unexpected to high-speed drivers approaching the end of queue (EOQ) and become safety concerns. Though the topic has been extensively studied, the identification of EOQ has been limited by the spatial-temporal resolution of traditional data sources. This study explores the potential of location-based crowdsourced data, specifically Waze user reports. It presents a dynamic clustering algorithm that can group the location-based reports in real time and identify the spatial-temporal extent of congestion as well as the EOQ. The algorithm is a spatial-temporal extension of the density-based spatial clustering of applications with noise (DBSCAN) algorithm for real-time streaming data with an adaptive threshold selection procedure. Here, the proposed method was tested with 34 traffic congestion cases in the Knoxville, Tennessee area of the United States. It is demonstrated that the algorithm can effectively detect spatial-temporal extent of congestion based on Waze report clusters and identify EOQ in real-time. The Waze report-based detection are compared to the detection based on roadside sensor data. The results are promising: The EOQ identification time of Waze is similar to the EOQ detection time of traffic sensor data, with only 1.1 min difference on average. In addition, Waze generates 1.9 EOQ detection points every mile, compared to 1.8 detection points generated by traffic sensor data, suggesting the two data sources are comparable in respect of reporting frequency. The results indicate that Waze is a valuable complementary source for EOQ detection where no traffic sensors are installed.

99 GENERAL AND MISCELLANEOUS↗

X-composer: enabling cross-environments in-situ workflows between HPC and cloud

As large-scale scientific simulations and big data analyses become more popular, it is increasingly more expensive to store huge amounts of raw simulation results to perform post-analysis. To minimize the expensive data I/O, "in-situ" analysis is a promising approach, where data analysis applications analyze the simulation generated data on the fly without storing it first. However, it is challenging to organize, transform, and transport data at scales between two semantically different ecosystems due to the distinct software and hardware difference. To tackle these challenges, we design and implement the X-Composer framework. X-Composer connects cross-ecosystem applications to form an "in-situ" scientific workflow, and provides a unified approach and recipe for supporting such hybrid in-situ workflows on distributed heterogeneous resources. X-Composer reorganizes simulation data as continuous data streams and feeds them seamlessly into the Cloud-based stream processing services to minimize I/O overheads. For evaluation, we use X-Composer to set up and execute a cross-ecosystem workflow, which consists of a parallel Computational Fluid Dynamics simulation running on HPC, and a distributed Dynamic Mode Decomposition analysis application running on Cloud. Our experimental results show that X-Composer can seamlessly couple HPC and Big Data jobs in their own native environments, achieve good scalability, and provide high-fidelity analytics for ongoing simulations in real-time.

Wang, Dali↗

SEVEN-YEAR WILKINSON MICROWAVE ANISOTROPY PROBE (WMAP ) OBSERVATIONS: GALACTIC FOREGROUND EMISSION

We present updated estimates of Galactic foreground emission using seven years of WMAP data. Using the power spectrum of differences between multi-frequency template-cleaned maps, we find no evidence for foreground contamination outside of the updated (KQ85y7) foreground mask. We place a 15 {mu}K upper bound on rms foreground contamination in the cleaned maps used for cosmological analysis. Further, the cleaning process requires only three power-law foregrounds outside of the mask. We find no evidence for polarized foregrounds beyond those from soft (steep-spectrum) synchrotron and thermal dust emission; in particular we find no indication in the polarization data of an extra 'haze' of hard synchrotron emission from energetic electrons near the Galactic center. We provide an updated map of the cosmic microwave background (CMB) using the internal linear combination method, updated foreground masks, and updates to point source catalogs using two different techniques. With additional years of data, we now detect 471 point sources using a five-band technique and 417 sources using a three-band CMB-free technique. In total there are 62 newly detected point sources, a 12% increase over the five-year release. Also new are tests of the Markov chain Monte Carlo foreground fitting procedure against systematics in the time-stream data, and tests against the observed beam asymmetry. Within a few degrees of the Galactic plane, the behavior in total intensity of low-frequency foregrounds is complicated and not completely understood. WMAP data show a rapidly steepening spectrum from 20 to 40 GHz, which may be due to emission from spinning dust grains, steepening synchrotron, or other effects. Comparisons are made to a 1 deg 408 MHz map (Haslam et al.) and the 11 deg ARCADE 2 data (Singal et al.). We find that spinning dust or steepening synchrotron models fit the combination of WMAP and 408 MHz data equally well. ARCADE data appear inconsistent with the steepening synchrotron model and consistent with the spinning dust model, though some discrepancies remain regarding the relative strength of spinning dust emission. More high-resolution data in the 10-40 GHz range would shed much light on these issues.

79 ASTRONOMY AND ASTROPHYSICS↗

Sleuthing the MSL EDL Performance from an X Band Carrier Perspective

During the Entry, Descent, and Landing (EDL) of NASA's Mars Science Laboratory (MSL), or Curiosity, rover to Gale Crater on Mars on August 6, 2012 UTC, the rover transmitted an X-band signal composed of carrier and tone frequencies and a UHF signal modulated with an 8kbps data stream. During EDL, the spacecraft's orientation is determined by its guidance and mechanical subsystems to ensure that the vehicle land safely at its destination. Although orientation to maximize telecom performance is not possible, antennas are especially designed and mounted to provide the best possible line of sight to Earth and to the Mars orbiters supporting MSL's landing. The tones and data transmitted over these links are selected carefully to reflect the most essential parameters of the vehicle's state and the performance of the EDL subsystems for post-EDL reconstruction should no further data transmission from the vehicle be possible. This paper addresses the configuration of the X band receive system used at NASA / JPL's Deep Space Network (DSN) to capture the signal spectrum of MSL's X band carrier and tone signal, examines the MSL vehicle state information obtained from the X band carrier signal only and contrasts the Doppler-derived information against the post-EDL known vehicle state. The paper begins with a description of the MSL EDL sequence of events and discusses the impact of the EDL maneuvers such as guided entry, parachute deploy, and powered descent on the frequency observables expected at the DSN. The range of Doppler dynamics possible is derived from extensive 6 Degrees-Of-Freedom (6 DOF) vehicle state calculations performed by MSL's EDL simulation team. The configuration of the DSN's receive system, using the Radio Science Receivers (RSR) to perform open-loop recording for both for nominal and off-nominal EDL scenarios, is detailed. Expected signal carrier power-to-noise levels during EDL are shown and their impact on signal detection is considered. Particular attention is given to the selection of the appropriate RSR processing bandwidths and to its configuration for real-time signal detection. The X-band carrier frequency obtained through post-processing of the open-loop recorded spectrum is given. Detection of spacecraft status and completion of key vehicle events through their Doppler signature is discussed and illustrated. This Doppler-derived information is compared against the very accurate vehicle data obtained post-EDL via MSL's UHF radio subsystem. The paper concludes with a discussion on the advantages and disadvantages of transmitting the X-band carrier and tone signal in the general context of EDL communications and lessons learned for future missions with EDL sequences are given

Mars Science Laboratory (MSL)↗

Misuse Detection for a Generalized SFR Test Reactor

Sodium-cooled Fast Reactors (SFRs) present unique challenges for international safeguards. SFRs possess neutron physics characteristics that if configured appropriately could produce more fissile material than consumed. An adversary state may choose to build an SFR, justified by a lack of domestic natural uranium and limited access or interest to procuring uranium from international markets. Once constructed, the state may choose to misuse the SFR for the purpose of diverting fissile plutonium from declared operation. This work shows that a demonstration SFR does not need to be configured as a plutonium breeder to create one Significant Quantity (SQ) of plutonium in a short amount of time (e.g., one to few years). However, such an extreme case of misuse would change the core reactivity in such a way as to be easily indicated by deviations of control rod position compared to declared operation. In this work a contrived SFR demonstration reactor was modeled for the purpose of exploring proliferation scenarios and how such misuse could be detected using the SFR's Reactor Data Acquisition System (RDAS). Typically, the International Atomic Energy Agency (IAEA) does not have access to the control rod position, power, thermal, pressure sensing and indicating systems of nuclear power plants. However, this work shows that such data streams can be compared against a parallel detailed simulation model (a Digital Twin) to detect possible misuse.

98 NUCLEAR DISARMAMENT, SAFEGUARDS, AND PHYSICAL P↗

Multifractal characterizations of nonstationarity and intermittency in geophysical fields: Observed, retrieved, or simulated

Geophysical data rarely show any smoothness at any scale, and this often makes comparison with theoretical model output difficult. However, highly fluctuating signals and fractal structures are typical of open dissipative systems with nonlinear dynamics, the focus of most geophysical research. High levels of variability are excited over a large range of scales by the combined actions of external forcing and internal instability. At very small scales we expect geophysical fields to be smooth, but these are rarely resolved with available instrumentation or simulation tools; nondifferentiable and even discontinuous models are therefore in order. We need methods of statistically analyzing geophysical data, whether measured in situ, remotely sensed or even generated by a computer model, that are adapted to these characteristics. An important preliminary task is to define statistically stationary features in generally nonstationary signals. We first discuss a simple criterion for stationarity in finite data streams that exhibit power law energy spectra and then, guided by developments in turbulence studies, we advocate the use of two ways of analyzing the scale dependence of statistical information: singular measures and qth order structure functions. In nonstationary situations, the approach based on singular measures seeks power law behavior in integrals over all possible scales of a nonnegative stationary field derived from the data, leading to a characterization of the intermittency in this (gradient-related) field. In contrast, the approach based on structure functions uses the signal itself, seeking power laws for the statistical moments of absolute increments over arbitrarily large scales, leading to a characterization of the prevailing nonstationarity in both quantitative and qualitative terms. We explain graphically, step by step, both multifractal statistics which are largely complementary to each other. The geometrical manifestations of nonstationarity and intermittency, 'roughness' and 'sparseness', respectively, are illustrated and the associated analytical (differentiability and continuity) properties are discussed. As an example, the two techniques are applied to a series of recent measurements of liquid water distributions inside marine stratocumulus decks; these are found to be multifractal over scales ranging from approximately 60 m to approximately 60 km. Finally, we define the 'mean multifractal plane' and show it to be a simple yet comprehensive tool with many applications including data intercomparison, (dynamical or stochastic) model and retrieval validations.

Davis, Anthony↗

Time series methods for the analysis of soundscapes and other cyclical ecological data

Biodiversity monitoring has entered an era of ‘big data’, exemplified by a near-continuous collection of sounds, images, chemical and other signals from organisms in diverse ecosystems. Such data streams have the potential to help identify new threats, assess the effectiveness of conservation interventions, as well as generate new ecological insights. However, appropriate analytical methods are often still missing, particularly with respect to characterizing cyclical temporal patterns. Here, we present a framework for characterizing and analysing ecological responses that represent nonstationary, complex temporal patterns and demonstrate the value of using Fourier transforms to decorrelate continuous data points. In our example, we use a framework based on three approaches (spectral analysis, magnitude squared coherence, and principal component analysis) to characterize differences in tropical forest soundscapes within and across sites and seasons in Gabon. By reconstructing the underlying, cyclic behaviour of the soundscape for each site, we show how one can identify circadian patterns in acoustic activity. Soundscapes in the dry season had a complex diel cycle, requiring multiple harmonics to represent daily variation, while in the wet season there was less variance attributable to the daily cyclic patterns. Our framework can be applied to most continuous, or near-continuous ecological data collected at a fine temporal resolution, allowing ecologists to explore patterns of temporal autocorrelation at multiple levels for biologically meaningful trends. Such methods will become indispensable as biological big data are used to understand the impact of anthropogenic pressures on biodiversity and to inform efforts to mitigate them.

54 ENVIRONMENTAL SCIENCES↗