Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “data streams”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 253 records · Page 14

Why wait? Let's start computing while data is still on the wire

In this era of Big Data, computing useful information from data is becoming increasingly complicated, particularly due to the ever increasing volumes of data that need to travel over the network to data centers to be stored and processed, all highly expensive operations in the long haul. In this paper we suggest that we can do computing and analysis of data "on the wire," i.e., while data is still in transit. The nature of these computations include analysis, visualization, pattern recognition, and prediction, or forecasting, on the streaming data. We follow a service function chaining architecture to implement this, assuming that the data packets arrive within a single network administrative domain. As a demonstration of this new computing paradigm, we present three examples. Firstly, we demonstrate pattern recognition and data visualization on streaming forex data, which can be used for lucrative trading in the forex market. In our second example, we analyze and learn user buying patterns from clickstream data streaming from multiple websites. Finally, we monitor solar sensors for a zero reading while the packets are still on their way to the data center, to schedule any maintenance and requisite repairs with no time delay.

97 MATHEMATICS AND COMPUTING↗

Adaptive Waveform Correlation Detectors for Arrays: Algorithms for Autonomous Calibration

Waveform correlation detectors compare a signal template with successive windows of a continuous data stream and report a detection when the correlation coefficient, or some comparable detection statistic, exceeds a specified threshold. Since correlation detectors exploit the fine structure of the full waveform, they are exquisitely sensitive when compared to power (STA/LTA) detectors. The drawback of correlation detectors is that they require complete knowledge of the signal to be detected, which limits such methods to instances of seismicity in which a very similar signal has already been observed by every station used. Such instances include earthquake swarms, aftershock sequences, repeating industrial seismicity, and many other forms of controlled explosions. The reduction in the detection threshold is even greater when the techniques are applied to arrays since stacking can be performed on the individual channel correlation traces to achieve significant array gain. In previous years we have characterized the decrease in detection threshold afforded by correlation detection across an array or network when observations of a previous event provide an adequate template for signals from subsequent events located near the calibration event. Last year we examined two related issues: (1) the size of the source region calibration footprint afforded by a master event, and (2) the use of temporally incoherent detectors designed to detect the gross envelope structure of the signal to extend the footprint. In Case 1, results from the PETROBAR-1 marine refraction profile indicated that array correlation gain was usable at inter-source separations out to one or two wavelengths. In Case 2, we found that incoherent detectors developed from a magnitude 6 event near Svalbard were successful at detecting aftershocks where correlation detectors derived from individual aftershocks were not. Incoherent detectors might provide 'seed' events for correlation detectors that then could extend detection to lower magnitudes. This year we addressed a problem long known to limit the acceptance of correlation detectors in practice: the labor intensive development of templates. For example, existing design methods cannot keep pace with rapidly unfolding aftershock sequences. We successfully built and tested an object-oriented framework (as described in our 2005 proposal) for autonomous calibration of waveform correlation detectors for an array. The framework contains a dynamic list of detectors of several types operating on a continuous array data stream. The list has permanent detectors: beam forming power (STA/LTA) detectors which serve the purpose of detecting signals not yet characterized with a waveform template. The framework also contains an arbitrary number of subspace detectors which are launched automatically using the waveforms from validated power detections as templates. The implementation is very efficient such that the computational cost of adding subspace detectors was low. The framework contains a supervisor that oversees the validation of power detections, and periodically halts the processing to revise the portfolio of detectors. The process of revision consists of collecting the waveforms from all detections, performing cross-correlations pairwise among all waveforms, clustering the detections using correlations as a distance measure, then creating a new subspace detector from each cluster. The collection of new subspace detectors replaces the existing portfolio and processing of the data stream resumes. This elaborate scheme was implemented to prevent proliferation of closely-related subspace detectors. The method performed very well on several simple sequences: 2005 'drumbeat' events observed locally at Mt. St. Helens, and the 2003 Orinda, CA aftershock sequence. Our principal test entailed detection of the aftershocks of the San Simeon earthquake using the NVAR array; in this case, the system automatically detected and categorized approximately 2/3 of the events above magnitude 2.8.

58 GEOSCIENCES↗

LHC physics dataset for unsupervised New Physics detection at 40 MHz

In the particle detectors at the Large Hadron Collider, hundreds of millions of proton-proton collisions are produced every second. If one could store the whole data stream produced in these collisions, tens of terabytes of data would be written to disk every second. The general-purpose experiments ATLAS and CMS reduce this overwhelming data volume to a sustainable level, by deciding in real-time whether each collision event should be kept for further analysis or be discarded. We introduce a dataset of proton collision events that emulates a typical data stream collected by such a real-time processing system, pre-filtered by requiring the presence of at least one electron or muon. This dataset could be used to develop novel event selection strategies and assess their sensitivity to new phenomena. In particular, we intend to stimulate a community-based effort towards the design of novel algorithms for performing unsupervised new physics detection, customized to fit the bandwidth, latency and computational resource constraints of the real-time event selection system of a typical particle detector.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗

Selecting Essential Information for Biosurveillance - A Multi-Criteria Decision Analysis

This paper proposes the use of Multi-Attribute Utility Theory to address the issue of identifying and selecting essential information for inclusion into a biosurveillance system or process. Additionally, we developed a decision support framework that can facilitate identifying data streams for use in biosurveillance systems or processes and demonstrated utility by applying the framework to the problem of evaluating data streams for use in an global infectious disease surveillance system.

59 BASIC BIOLOGICAL SCIENCES↗

Transitioning NPOESS Data to Weather Offices: The SPoRT Paradigm with EOS Data

Real-time satellite information provides one of many data sources used by NWS weather forecast offices (WFOs) to diagnose current weather conditions and to assist in short-term forecast preparation. While GOES satellite data provides relatively coarse spatial resolution coverage of the continental U.S. on a 10-15 minute repeat cycle, polar orbiting imagery has the potential to provide snapshots of weather conditions at high-resolution in many spectral channels. Additionally, polar orbiting sounding data can provide additional information on the thermodynamic structure of the atmosphere in data sparse regions of at asynoptic observation times. The NASA Short-term Prediction Research and Transition (SPoRT) project has demonstrated the utility of polar orbiting MODIS and AIRS data on the Terra and Aqua satellites to improve weather diagnostics and short-term forecasting on the regional and local scales. SPoRT scientists work directly forecasters at selected WFOS in the Southern Region (SR) to help them ingest these unique data streams into their AWIPS system, understand how to use the data (through on-site and distance learn techniques), and demonstrate the utility of these products to address significant forecast problems. This process also prepares forecasters for the use of similar observational capabilities from NPOESS operational sensors. NPOESS environmental data records (EDRs) from the Visible 1 Infrared Imager I Radiometer Suite (VIIRS), the Cross-track Infrared Sounder (CrlS) and Advanced Technology Microwave Sounder (ATMS) instruments and additional value-added products produced by NESDIS will be available in near real-time and made available to WFOs to extend their use of NASA EOS data into the NPOESS era. These new data streams will be integrated into the NWs's new AWIPS II decision support tools. The AWIPS I1 system to be unveiled in WFOs in 2009 will be a JAVA-based decision support system which preserves the functionality of the existing systems and offers unique development opportunities for new data sources and applications in the Service Orientated Architecture ISOA) environment. This paper will highlight some of the SPoRT activities leading to the integration of VllRS and CrIS/ATMS data into the display capabilities of these new systems to support short-term forecasting problems at WFOs.

Jedlovec, Gary↗

Atmospheric Radiation Measurement Program Climate Research Facility Operations Quarterly Report - October 1 - December 31, 2008

Description. Individual raw data streams from instrumentation at the Atmospheric Radiation Measurement (ARM) Program Climate Research Facility (ACRF) fixed and mobile sites are collected and sent to the Data Management Facility (DMF) at Pacific Northwest National Laboratory (PNNL) for processing in near real-time. Raw and processed data are then sent daily to the ACRF Archive, where they are made available to users. For each instrument, we calculate the ratio of the actual number of data records received daily at the Archive to the expected number of data records. The results are tabulated by (1) individual data stream, site, and month for the current year and (2) site and fiscal year (FY) dating back to 1998.

54 ENVIRONMENTAL SCIENCES↗

Stream splitting moving target defense

Systems and methods for utilizing stream splitting Moving Target Defense (MTD) to provide enhanced computer system communication system security by splitting a data stream in to a plurality of paths is described. In some implementations, Stream splitting MTD, involves splitting a single data stream (e.g., TCP stream) into a plurality of discrete units, then sending and receiving those discrete units from and to different (ideally geographically disparate) receiving servers, with the stream being reassembled on the receiving end. The plurality of discrete units of data include resequencing data. The size of each discrete unit may vary depending on the specific implementation, even down to small unit sizes (e.g., a single packet).

Evans, Nathaniel↗

Atmospheric Radiation Measurement Program Climate Research Facility Operations Quarterly Report January 1 - March 31, 2009

Individual raw data streams from instrumentation at the Atmospheric Radiation Measurement (ARM) Program Climate Research Facility (ACRF) fixed and mobile sites are collected and sent to the Data Management Facility (DMF) at Pacific Northwest National Laboratory (PNNL) for processing in near real-time. Raw and processed data are then sent daily to the ACRF Archive, where they are made available to users. For each instrument, we calculate the ratio of the actual number of data records received daily at the Archive to the expected number of data records. The results are tabulated by (1) individual data stream, site, and month for the current year and (2) site and fiscal year (FY) dating back to 1998.

54 ENVIRONMENTAL SCIENCES↗

High data volume and transfer rate techniques used at NASA's image processing facility

Data storage and transfer operations at a new image processing facility are described. The equipment includes high density digital magnetic tape drives and specially designed controllers to provide an interface between the tape drives and computerized image processing systems. The controller performs the functions necessary to convert the continuous serial data stream from the tape drive to a word-parallel blocked data stream which then goes to the computer-based system. With regard to the tape packing density, 1.8 times 10 to the tenth data bits are stored on a reel of one-inch tape. System components and their operation are surveyed, and studies on advanced storage techniques are summarized.

Heffner, P.↗

The Life Cycle Application of Intelligent Software Modeling for the First Materials Science Research Rack

Marshall Space Flight Center (MSFC) has been funding development of intelligent software models to benefit payload ground operations for nearly a decade. Experience gained from simulator development and real-time monitoring and control is being applied to engineering design, testing, and operation of the First Material Science Research Rack (MSRR-1). MSRR-1 is the first rack in a suite of three racks comprising the Materials Science Research Facility (MSRF) which will operate on the International Space Station (ISS). The MSRF will accommodate advanced microgravity investigations in areas such as the fields of solidification of metals and alloys, thermo-physical properties of polymers, crystal growth studies of semiconductor materials, and research in ceramics and glasses. The MSRR-1 is a joint venture between NASA and the European Space Agency (ESA) to study the behavior of different materials during high temperature processing in a low gravity environment. The planned MSRR-1 mission duration is five (5) years on-orbit and the total design life is ten (IO) years. The MSRR-1 launch is scheduled on the third Utilization Flight (UF-3) to ISS, currently in February of 2003). The objective of MSRR-1 is to provide an early capability on the ISS to conduct material science, materials technology, and space product research investigations in microgravity. It will provide a modular, multi-user facility for microgravity research in materials crystal growth and solidification. An intelligent software model of MSRR-1 is under development and will serve multiple purposes to support the engineering analysis, testing, training, and operational phases of the MSRR-1 life cycle development. The G2 real-time expert system software environment developed by Gensym Corporation was selected as the intelligent system shell for this development work based on past experience gained and the effectiveness of the programming environment. Our approach of multi- uses of the simulation model and its intuitive graphics capabilities is providing a concurrent engineering environment for rapid prototyping and development. Operational schematics of the MSRR-1 electrical, thermal control, vacuum access, and gas supply systems, and furnace inserts are represented graphically in the environment. Logic to represent first order engineering calculations is coded into the knowledge base to simulate the operational behavior of the MSRR-1 systems. An example of engineering data provided includes electrical currents, voltages, operational power, temperatures, thermal fluid flow rates. pressures, and component status indications. These type of data are calculated and displayed at appropriate instrumentation points, and the schematics are animated to reflect the simulated operational status of the MSRR-1. The software control functions are also simulated to represent appropriate operational behavior based on automated control and response to commands received by the crew or ground controllers. The first benefit of this simulation environment is being realized in the high fidelity engineering analysis results from the electrical power system G2 model. Secondly, the MSRR-1 simulation model will be embedded with a hardware mock-up of the MSRR-1 to provide crew training on MSRR-1 integrated payload operations. G2 gateway code will output the simulated instrumentation values, termed as telemetry, in a flight-like data stream so that the crew has realistic and accurate simulated MSRR-1 data on the flight displays which will be designed for crew use. The simulation will also respond appropriately to crew or ground initiated commands, which will be part of normal facility operations. A third use of the G2 model is being planned; the MSRR-1 simulation will be integrated with additional software code as part of the test configuration of the primary onboard computer, or Master Controller, for MSRR-1. We will take advantage of the G2 capability to simulate the flight like data stream to test flight software responses and behavior. A fourth use of the G2 model will be to train the Ground Support Personnel that will monitor the MSRR-1 systems and payloads while they are operating aboard the ISS. The intuitive, schematic based environment will provide an excellent foundation for personnel to understand the integrated configuration and operation of the MSRR-1, and the anticipated telemetry feedback based on operational modes of the equipment. Expert monitoring features will be enhanced to provide a smart monitoring environment for the operators. These features include: (1) Animated, intuitive schematic-based displays which reflect telemetry values, (1) Real-time plotting of simulated or incoming sensor values, (3) High/Low exception monitoring for analog data, (4) Expected state monitoring for discrete data, (5) Data trending, (6) Automated malfunction procedure execution to diagnose problems, (7) Look ahead capability to planned MSRR-1 activities in the onboard timeline. And finally, the logic to calculate telemetry values will be deactivated, and the same environment will interface to the incoming data for the real-time telemetry stream to schematically represent the onboard hardware configuration. G2 will be the foundation for the real-time monitoring and control environment. In summary, our MSRR-1 simulation model spans many elements of the life cycle development of this project: Engineering Analysis, Test and Checkout, Training of Crew and Ground Personnel, and Real-time monitoring and control. By utilizing the unique features afforded by an expert system development environment, we have been able to synergize a powerful tool capable of addressing our project needs at every phase of project development.

Rice, Amanda↗

Evaluation of existing and proposed computer architectures for future ground-based systems

Parallel processing architectures and techniques used in current supercomputers are described and projections are made of future advances. Presently, the von Neumann sequential processing pattern has been accelerated by having separate I/O processors, interleaved memories, wide memories, independent functional units and pipelining. Recent supercomputers have featured single-input, multiple data stream architectures, which have different processors for performing various operations (vector or pipeline processors). Multiple input, multiple data stream machines have also been developed. Data flow techniques, wherein program instructions are activated only when data are available, are expected to play a large role in future supercomputers, along with increased parallel processor arrays. The enhanced operational speeds are essential for adequately treating data from future spacecraft remote sensing instruments such as the Thematic Mapper.

Schulbach, C.↗

WHONDRS Surface Water Geochemistry and Organic Matter Characterization Data from Streams Distributed across Latin America

This dataset supports a broader study examining global transferability of stream biogeochemistry and was generated in collaboration with the MicroSudAqua (µSudAqua) network (https://microsudaqua.netlify.app/en/). The dataset provides surface water geochemistry (dissolved organic carbon, total dissolved nitrogen, cations) and organic matter characterization (FTICR-MS) from streams in Argentina, Brazil, Chile, and Colombia. Samples were collected across stream orders (1st to 6th order) within five basins. Related data were collected and will be published separately in collaboration with the µSudAqua network. For details on how to navigate data packages generated by this project, see https://data.ess-dive.lbl.gov/portals/PNNLRiverCorridorSFA/About. In addition to this readme, this data package also includes a file-level metadata (FLMD) file that describes each file and a data dictionary (DD) that describes all column/row headers and variable definitions. This dataset is comprised of (1) a folder of field photos; (2) a folder of surface water sample data, (3) a folder of raw Fourier transform ion cyclotron resonance mass spectrometry (FTICR-MS) data; (4) file-level metadata; (5) data dictionary; (6) field metadata; (7) readme; (8) international generic sample number (IGSN) mapping file; and (9) field protocol. The sample data subfolder contains (1) dissolved organic carbon (DOC, measured as non-purgeable organic carbon, NPOC) data and averages; (2) total dissolved nitrogen data and averages; (3) anions and averages; (4) methods codes; (5) FTICR-MS methods; and (15) a subfolder of 9.4 Tesla (9.4T) FTICR-MS data. This folder contains the processed data and three subfolders, one containing the .xml files, one containing the water CoreMS output files, and the other containing instructions and scripts for processing the files in CoreMS (https://github.com/EMSL-Computing/CoreMS). All files are .csv, .pdf, .R, .xml, .d, .html, .Rmd, .py, .cal, .json, .jpg, .jpeg, .png, .mov, or .mp4.

Anions↗

Machine Learning Digital Twin for Lithium Ion Battery State of Health Predictions

A digital twin system has been established to model the long term degradation of the state of health of lithium ion batteries. Two data streams result from the computational model of the system and from the physical experiment measurements. A machine learning pipeline has been developed at the nexus of these data streams. Leveraging the unique data sources in multiple transfer learning approaches has lead to the development of multiple cell specific machine learning digital twins. Our discussion on this first of its kind technology will cover challenges in data ingestion, scalability, architecture and orchestration, and research findings.

25 - ENERGY STORAGE↗

Development of an e-VLBI Data Transport Software Suite with VDIF

We have developed a software library (KVTP-lib) for VLBI data transmission over the network with the VDIF (VLBI Data Interchange Format), which is the newly proposed standard VLBI data format designed for electronic data transfer over the network. The software package keeps the application layer (VDIF frame) and the transmission layer separate, so that each layer can be developed efficiently. The real-time VLBI data transmission tool sudp-send is an application tool based on the KVTP-lib library. sudp-send captures the VLBI data stream from the VSI-H interface with the K5/VSI PC-board and writes the data to file in standard Linux file format or transmits it to the network using the simple- UDP (SUDP) protocol. Another tool, sudp-recv , receives the data stream from the network and writes the data to file in a specific VLBI format (K5/VSSP, VDIF, or Mark 5B). This software system has been implemented on the Wettzell Tsukuba baseline; evaluation before operational employment is under way.

Sekido, Mamoru↗

Aerodynamic results of a separation effects test on a 0.01-scale model (52-OTS) of integrated SSV in the AEDC/VKF 40-by-40 inch supersonic wind tunnel A, volume 1

Experimental aerodynamic investigations were conducted, during the period July 18-19, 1974, in the AEDC/VKF Tunnel A facility on a 0.01-scale model (52-OTS) of the integrated space shuttle vehicle, including only one SRB. The purpose of the investigation was to obtain data for close-in proximity (SRB to orbiter/tank) effects with the orbiter/tank combination at relatively high alpha and beta attitudes, and with the SRB separation motors off. The AEDC Captive Trajectory System (CTS), which supported the SRB, was used in conjunction with the tunnel primary sector (supporting the orbiter/tank) to obtain grid type separation effects data. The one symmetrical SRB model was used interchangeably to obtain both right-hand and left-hand SRB data. Free-stream data were also obtained for the orbiter/tank and for the SRB. This data was used to provide baselines for proximity effects. The entire investigation was conducted at a free-stream Mach number of 4.5 with unit Reynolds number ranging from 4.0 to 6.5 million per foot.

Campbell, J. H., II↗

Supporting Space Weather with the Geospace Dynamics Constellation

The Geospace Dynamics Constellation (GDC) mission, planned to launch at the end of the decade, is a six-satellite constellation that will fly through the ionosphere and thermosphere at around 400 km. While GDC is a science mission, its comprehensive instrumentation will not only contribute to our understanding of space weather phenomena in the ionosphere-thermosphere system, but will also provide valuable, space weather-relevant data. Data from GDC will be made available at low latency via real-time space weather data stream. We are working with operational partners to identify space weather data products and coordinate reception of the space weather beacon data. Alongside preparations for real-time GDC data streams, we are working to identify current model capabilities and needs, to ensure that space weather models that can make use of GDC measurements are moving along the Research-toOperations pipeline. Before GDC launch, Observing System Simulation Experiments (OSSEs) carried out with synthetic GDC data can be used to demonstrate the capabilities of models and to predict the impact of GDC data. Following launch, Observing System Experiments (OSEs) will demonstrate the impact of GDC space weather data. In preparation for GDC, the ITM space weather community should establish baseline metrics for space weather parameters of scientific and operational interest. These metrics, tracked over time before and after the launch of GDC, will allow us to track advancements in forecasting, nowcasting, and hindcasting of the ITM system and to trace the impact of scientific progress from space weather research into operations. By demonstrating the impact of real-time GDC data, specific data needs can be identified and prioritized for long term investment on future observing systems.

Katherine Garcia-Sage↗

OCEANIDS: Autonomous Data Acquisition, Management and Distribution System

OCEANIDS is a clearinghouse for mission essential and near-real-time satellite data streams. This viewgraph presentation describes this mission, and includes the following topics: 1) OCEANIDS Motivation; 2) High-Level Architecture; 3) OCEANIDS Features; 4) OCEANIDS GUI: Nodes; 5) OCEANIDS GUI: Cluster; 6) Data Streams; 7) Statistics; and 8) GHRSST-PP.

automated monitoring↗

G-mode magnetic force microscopy: Separating magnetic and electrostatic interactions using big data analytics

In this work, we develop a full information capture approach for Magnetic Force Microscopy (MFM), referred to as generalized mode (G-Mode) MFM. G-Mode MFM acquires and stores the full data stream from the photodetector, captured at sampling rates approaching the intrinsic photodiode limit. The data can be subsequently compressed, denoised, and analyzed, without information loss. Here, G-Mode MFM is implemented and compared to the traditional heterodyne-based MFM on model systems, including domain structures in ferromagnetic Yttrium Iron Garnet and the electronically and magnetically inhomogeneous high entropy alloy, CoFeMnNiSn. We investigate the use of information theory to mine the G-Mode MFM data and demonstrate its usefulness for extracting information which may be hidden in traditional MFM modes, including signatures of nonlinearities and mode-coupling phenomena. Finally, we demonstrate detection and separation of magnetic and electrostatic tip-sample interactions from a single G-Mode image, by analyzing the entire frequency response of the cantilever. G-Mode MFM is immediately implementable on any atomic force microscopy platform and as such is expected to be a useful technique for probing spatiotemporal cantilever dynamics and mapping material properties, as well as their mutual interactions.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗