Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “data streams”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 559 records · Page 31

Integrated photonic encoder for low power and high-speed image processing

Abstract Modern lens designs are capable of resolving greater than 10 gigapixels, while advances in camera frame-rate and hyperspectral imaging have made data acquisition rates of Terapixel/second a real possibility. The main bottlenecks preventing such high data-rate systems are power consumption and data storage. In this work, we show that analog photonic encoders could address this challenge, enabling high-speed image compression using orders-of-magnitude lower power than digital electronics. Our approach relies on a silicon-photonics front-end to compress raw image data, foregoing energy-intensive image conditioning and reducing data storage requirements. The compression scheme uses a passive disordered photonic structure to perform kernel-type random projections of the raw image data with minimal power consumption and low latency. A back-end neural network can then reconstruct the original images with structural similarity exceeding 90%. This scheme has the potential to process data streams exceeding Terapixel/second using less than 100 fJ/pixel, providing a path to ultra-high-resolution data and image acquisition systems.

47 OTHER INSTRUMENTATION↗

Pattern Discovery in Internet Background Radiation

Internet Background Radiation (IBR) is observed in empty network address spaces. No traffic should arrive there, but it does in overwhelming quantities, gathering evidences of attacks, malwares and misconfigurations. The study of IBR helps to detect spreading network problems, common vulnerabilities and attack trends. However, network traffic data evolves quickly and is of high volume and diversity, i.e., an outstanding big data challenge. When used to assist network security, it also requires the online classification of dynamic streaming data. In this paper, we introduce an AGgregation & Mode (AGM) vector to represent network traffic. The AGM format characterizes IP hosts by extracting aggregated and mode values of IP header fields, and without inspecting payloads. We performed clustering and statistical analysis to explore six months of IBR from 2012 with the AGM mapping. The discovered patterns allow building a classification of IBR, which identifies phenomena that have been actively polluting the Internet for years. The AGM representation is light and tailored for monitoring and pattern discovery. We show that AGM vectors are suitable to analyze large volumes of network traffic: they capture permanent operations, such as long term scanning, as well as bursty events from targeted attacks and short term incidents.

Iglesias, Felix↗

Online Voltage Event Detection Using Synchrophasor Data with Structured Sparsity-Inducing Norms

This paper develops an accurate and computationally efficient data-driven framework to detect voltage events from PMU data streams. It develops an innovative Proximal Bilateral Random Projection (PBRP) algorithm to quickly decompose the PMU data matrix into a low-rank matrix, a row-sparse event-pattern matrix and a noise matrix. Here, the row-sparse pattern matrix significantly distinguishes events from normal behavior. These matrices are then fed into a clustering algorithm to separate voltage events from normal operating conditions. Large-scale numerical study results on real-world PMU data show that the proposed algorithm is computationally more efficient and achieves higher F scores than state-of-the-art benchmarks.

24 POWER TRANSMISSION AND DISTRIBUTION↗

Use of Integrated Instrumentation to Detect and Classify Targets in Shallow Water

The detection and classification of targets in shallow water environments (e.g., depth < 20 m) is of increasing interest, both in security applications and more general environmental studies. Here, we report the ability of a cabled integrated instrumentation system to detect and classify targets in a shallow (15 m deep), narrow (150 m wide) channel influenced by moderate tidal currents (peak currents < 2 m/s). The instrumentation system incorporates a passive acoustic array, multibeam sonar, acoustic camera, stereo-optical camera, and acoustic Doppler current profiler. The system effectiveness is benchmarked using autonomous “cooperative” targets (e.g., drifters with known position and acoustic characteristics), divers, and opportunistic observations of fish and marine mammals. By fusing data streams from multiple sensors, detection and classification rates are improved, particularly for smaller targets. Approaches to data management (data rates exceed 80 MB/s), real-time classification, and de-noising of acoustic imagery are also discussed.

16 TIDAL AND WAVE POWER↗

The CompTox Chemistry Dashboard: a community data resource for environmental chemistry

Despite an abundance of online databases providing access to chemical data, there is increasing demand for highquality, structure-curated, open data to meet the various needs of the environmental sciences and computational toxicology communities. The U.S. Environmental Protection Agency’s (EPA) web-based CompTox Chemistry Dashboard is addressing these needs by integrating diverse types of relevant domain data through a cheminformatics layer, built upon a database of curated substances linked to chemical structures. These data include physicochemical, environmental fate and transport, exposure, usage, in vivo toxicity, and in vitro bioassay data, surfaced through an integration hub with link-outs to additional EPA data and public domain online resources. Batch searching allows for direct chemical identifer (ID) mapping and downloading of multiple data streams in several diferent formats. This facilitates fast access to available structure, property, toxicity, and bioassay data for collections of chemicals (hundreds to thousands at a time). Advanced search capabilities are available to support, for example, non-targeted analysis and identifcation of chemicals using mass spectrometry. The contents of the chemistry database, presently containing ~ 760,000 substances, are available as public domain data for download. The chemistry content underpinning the Dashboard has been aggregated over the past 15 years by both manual and auto-curation techniques within EPA’s DSSTox project. DSSTox chemical content is subject to strict quality controls to enforce consistency among chemical substance-structure identifers, as well as list curation review to ensure accurate linkages of DSSTox substances to chemical lists and associated data. The Dashboard, publicly launched in April 2016, has expanded considerably in content and user trafc over the past year. It is continuously evolving with the growth of DSSTox into high-interest or data-rich domains of interest to EPA, such as chemicals on the Toxic Substances Control Act listing, while providing the user community with a fexible and dynamic web-based platform for integration, processing, visualization and delivery of data and resources. The Dashboard provides support for a broad array of research and regulatory programs across the worldwide community of toxicologists and environmental scientists.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

"Sintering" Models and In-Situ Experiments: Data Assimilation for Microstructure Prediction in SLS Additive Manufacturing of Nylon Components

Selective laser sintering methods are workhorses for additively manufacturing polymer-based components. The ease of rapid prototyping also means it is easy to produce illicit components. It is necessary to have a data-calibrated in-situ physical model of the build process in order to predict expected and defective microstructure characteristics that inform component provenance. Toward this end, sintering models are calibrated and characteristics such as component defects are explored. This is accomplished by assimilating multiple data streams, imaging analysis, and computational model predictions in an adaptive Bayesian parameter estimation algorithm. From these data sources, along with a phase-field model, bulk porosity distributions are inferred. Model parameters are constrained to physically-relevant search directions by sensitivity analysis, and then matched to predictions using adaptive sampling. Using this feedback loop, data-constrained estimates of sintering model parameters along with uncertainty bounds are obtained.

3D printing, Additive Manufacturing, polymer, sint↗

JSME scatterometer data processing

A software system was developed which processes digitized scatterometer data from the 13.3 GHz, 1.6 GHz and 400 MHz scatterometer systems. In addition to this, the hardware capability has been developed to recover the raw analog radar signals and the aircraft parameters from an ADAS data stream in a digital format for processing by the software package. Software for the preparation of data reports and chart presentation of scattering coefficients time histories has also been developed. This report documents the development of the software, describes key components of the processing system and presents examples of the processed data and procedure for software operation.

Source record↗

The Massively Parallel Processor and its applications

A long-term experimental development program conducted at Goddard Space Flight Center to implement an ultrahigh-speed data processing system known as the Massively Parallel Processor (MPP) is described. The MPP is a single instruction multiple data stream computer designed to perform logical, integer, and floating point arithmetic operations on variable word length data. Information is presented on system architecture, the system configuration, the array unit architecture, individual processing units, and expected operating rates for several image processing applications (including the processing of Landsat data).

Strong, J. P.↗

Optical Communications Link to Airborne Transceiver

An optical link from Earth to an aircraft demonstrates the ability to establish a link from a ground platform to a transceiver moving overhead. An airplane has a challenging disturbance environment including airframe vibrations and occasional abrupt changes in attitude during flight. These disturbances make it difficult to maintain pointing lock in an optical transceiver in an airplane. Acquisition can also be challenging. In the case of the aircraft link, the ground station initially has no precise knowledge of the aircraft s location. An airborne pointing system has been designed, built, and demonstrated using direct-drive brushless DC motors for passive isolation of pointing disturbances and for high-bandwidth control feedback. The airborne transceiver uses a GPS-INS system to determine the aircraft s position and attitude, and to then illuminate the ground station initially for acquisition. The ground transceiver participates in link-pointing acquisition by first using a wide-field camera to detect initial illumination from the airborne beacon, and to perform coarse pointing. It then transfers control to a high-precision pointing detector. Using this scheme, live video was successfully streamed from the ground to the aircraft at 270 Mb/s while simultaneously downlinking a 50 kb/s data stream from the aircraft to the ground.

Regehr, Martin W.↗

Semi-Supervised Eigenbasis Novelty Detection

Recent discoveries in high-time-resolution radio astronomy data have focused attention on a new class of events. Fast transients are rare pulses of radio frequency energy lasting from microseconds to seconds that might be produced by a variety of exotic astrophysical phenomena. For example, X-ray bursts, neutron stars, and active galactic nuclei are all possible sources of short-duration, transient radio signals. It is difficult to anticipate where such signals might appear, and they are most commonly discovered through analysis of high-time- resolution data that had been collected for other purposes. Transients are often faint and difficult to detect, so improved detection algorithms can directly benefit the science yield of all such commensal monitoring. A new detection algorithm learns a low-dimensional linear manifold for describing the normal data. High reconstruction error indicates a novel signal that does not match the patterns of normal data. One unsupervised portion of the manifold model adapts its representation in response to recent data. A second supervised portion of the model is made of a basis trained in advance using labeled examples of RFI; this prevents false positives due to these events. For a linear model, an orthonormalization operation is used to combine these bases prior to the anomaly detection decision. Another novel aspect of the approach lies in combining basis vectors learned in an unsupervised, online fashion from the data stream with supervised basis vectors learned in advance from known examples of false alarms. Adaptive, data-driven detection is achieved that is also informed by existing domain knowledge about signals that may be statistically anomalous, but are not interesting and should therefore be ignored. The method was evaluated using data from the Parkes Multibeam Survey. This data set was originally collected to search for pulsars, which are astronomical sources that emit radio pulses at regular periods. However, several non-pulsar anomalies have recently been discovered in this dataset, making it a compelling test case. By explicitly filtering known false alarm patterns, the approach yields significantly better performance than current transient detection methods.

Wagstaff, Kiri L.↗

Artemis I Liftoff Measurements Issues and Corrections

The SLS rocket was instrumented to collect liftoff environment data during Artemis I launch. The instruments were microphones and pressure transducers. Most data were acceptable for analysis, but required corrections for port resonance, mean signal drift, data system delays, and data spikes. The data were all transmitted via telemetry and in this process some of the measurements experienced unexpectedly long gaps in the data stream. A time validated reconstruction algorithm failed when encountering the long gaps and required use of data from a simpler reconstruction process, with manually inserted data delay corrections. This paper discusses the issues and the corrections that were necessary for the launch data to determine the true physics of the acoustics occurring during launch.

environments↗

Artemis I Liftoff Measurements Issues and Corrections

The SLS rocket was instrumented to collect liftoff environment data during Artemis I launch. The instruments were microphones and pressure transducers. Most data were acceptable for analysis, but required corrections for port resonance, mean signal drift, data system delays, and data spikes. The data were all transmitted via telemetry and in this process some of the measurements experienced unexpectedly long gaps in the data stream. A time validated reconstruction algorithm failed when encountering the long gaps and required use of data from a simpler reconstruction process, with manually inserted data delay corrections. This paper discusses the issues and the corrections that were necessary for the launch data to determine the true physics of the acoustics occurring during launch.

environments↗

Enabling API Access to the Space Weather Services at the Community Coordinated Modeling Center

Over the span of 20 years, the Community Coordinated Modeling Center (CCMC, https://ccmc.gsfc.nasa.gov) has been leading a number of community-driven services and applications that provide a free and open access to the cutting-edge space weather and Heliophysics models through a simple web-based interface. CCMC also oversees an open archive of user model simulations and related metadata, maintains space weather-related data streams, curates validation and event datasets, and more. To maximize utility of its complementary services and data holdings, CCMC has been gradually building up an ad-hoc set of interfaces and specifications that facilitate coupling and interconnection within the organization, while also simplifying management and monitoring of the data. As the models continue to grow in maturity and complexity, CCMC is looking to reduce the complexity for its end users by making public some of the internal APIs as well as implementing dedicated interfaces as required by the community. In this presentation, we will overview the current run services at CCMC and will describe our current and near-future efforts in providing interfaces to these services.

space weather↗

Volcano infrasound: progress and future directions

Over the past two decades (2000–2020), volcano infrasound (acoustic waves with frequencies less than 20 Hz propagating in the atmosphere) has evolved from an area of academic research to a useful monitoring tool. As a result, infrasound is routinely used by volcano observatories around the world to detect, locate, and characterize volcanic activity. It is particularly useful in confirming subaerial activity and monitoring remote eruptions, and it has shown promise in forecasting paroxysmal activity at open-vent systems. Fundamental research on volcano infrasound is providing substantial new insights on eruption dynamics and volcanic processes and will continue to do so over the next decade. The increased availability of infrasound sensors will expand observations of varied eruption styles, and the associated increase in data volume will make machine learning workflows more feasible. More sophisticated modeling will be applied to examine infrasound source and propagation effects from local to global distances, leading to improved infrasound-derived estimates of eruption properties. Future work will use infrasound to detect, locate, and characterize moving flows, such as pyroclastic density currents, lahars, rockfalls, lava flows, and avalanches. Infrasound observations will be further integrated with other data streams, such as seismic, ground- and satellite-based thermal and visual imagery, geodetic, lightning, and gas data. The volcano infrasound community should continue efforts to make data and codes accessible and to improve diversity, equity, and inclusion in the field. In summary, the next decade of volcano infrasound research will continue to advance our understanding of complex volcano processes through increased data availability, sensor technologies, enhanced modeling capabilities, and novel data analysis methods that will improve hazard detection and mitigation.

58 GEOSCIENCES↗

Interactive Data Visualization

Interactive data visualization leverages human visual perception and cognition to improve the accuracy and effectiveness of data analysis. When combined with automated data analytics, data visualization systems orchestrate the strengths of humans with the computational power of machines to solve problems neither approach can manage in isolation. In the intelligent transportation system domain, such systems are necessary to support decision making in large and complex data streams. In this chapter, we provide an introduction to several key topics related to the design of data visualization systems. In addition to an overview of key techniques and strategies, we will describe practical design principles. The chapter is concluded with a detailed case study involving the design of a multivariate visualization tool.

information visualization↗

Applications of visualization technology in the structural sciences

The structural sciences are undergoing a transformation driven by advancements in visualization technologies that aid researchers in understanding and communicating experimental data from complex molecular systems. New applications of integrative structural biological and biophysical approaches add a wide variety of complementary information from a broad range of scientific disciplines. These approaches extend structural biophysical methodologies to enable research by the incorporation of a variety of data streams and utilization of tools like molecular graphics, virtual reality, and machine learning. To redefine how structural data—particularly from cryo-electron microscopy and x-ray crystallography—are fed forward for scientific exploration and communication, the advances in tools for data visualization and interpretation have been critical. By bringing molecular systems into an interactive three-dimensional space, these novel technologies enhance research workflows, facilitate structure-based drug design, and create engaging educational experiences. Taken together, these visualization innovations are essential tools for advancing the field by making concepts more accessible and compelling.

Eng, Edward T. [New York Structural Biology Center↗

Streaming Analytics Machine (SAM) v. 0.0.1

This is software developed in the course of my dissertation work, which was funded by Sandia. The software includes a library (the Streaming Analytics Machine or SAM) for expressing machine learning and subgraph matching over streaming data sources. It also includes a utility that will convert from a high-level language, the Streaming Analytics Language (SAL) to SAM. SAND2019-4322 M

Goodman, Eric↗

Classification and Localization of Fracture-Hit Events in Low-Frequency Distributed Acoustic Sensing Strain Rate with Convolutional Neural Networks

Summary Distributed acoustic sensing (DAS) has been used in the oil and gas industry as an advanced technology for surveillance and diagnostics. Operators use DAS to monitor hydraulic fracturing activities, examine well stimulation efficacy, and estimate complex fracture system geometries. Particularly, low-frequency DAS can detect geomechanical events such as fracture hits because hydraulic fractures propagate and create strain rate variations in the rock. Analysis of DAS data today is mostly done post-job and subject to interpretation methods. However, the continuous and dense data stream generated live by DAS poses the opportunity for more efficient and accurate real-time data-driven analysis. The objective of this study is to develop a machine learning-based workflow that can identify and locate fracture-hit events in simulated strain rate responses correlated with low-frequency DAS data. In this paper, “fracture hit” refers to a hydraulic fracture originating from a stimulated well intersecting an offset well. We start with building a single fracture propagation model to produce strain rate patterns observed at a hypothetical monitoring well. This model is used to generate two sets of strain rate responses with one set containing fracture-hit events. The labeled synthetic data are then used to train a custom convolutional neural network (CNN) model for identifying the presence of fracture-hit events. The same model is trained again for locating the event with the output layer of the model replaced with linear units. We achieved near-perfect predictions for both event classification and localization. These promising results prove the feasibility of using CNN for real-time event detection from fiber-optic sensing data. Additionally, we use edge detection techniques to recognize fracture-hit event patterns in strain rate images. The fracture-hit location can be identified using recognized pixels in the image. The accuracy of edge detection-based location identification is also plausible, but edge detection is dependent on the assumption of pattern shape and image quality, hence it is less robust compared to CNN models. This comparison further supports the need for CNN applications in image-based real-time fiber-optic sensing event detection.

Engineering↗