Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “data streams”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 181 records · Page 10

Implementation of Dynamic Extensible Adaptive Locally Exchangeable Measures (IDEALEM) v 0.1

Handling large streaming data is essential for various applications such as network traffic analysis, social networks, energy cost trends, and environment modeling. However, it is in general intractable to store, compute, search, and retrieve large streaming data. This software addresses a fundamental issue, which is to reduce the size of large streaming data and still obtain accurate statistical analysis. As an example, when a high-speed network such as 100 Gbps network is monitored, the collected measurement data rapidly grows so that polynomial time algorithms (e.g., Gaussian processes) become intractable. One possible solution to reduce the storage of vast amounts of measured data is to store a random sample, such as one out of 1000 network packets. However, such static sampling methods (linear sampling) have drawbacks: (1) it is not scalable for high-rate streaming data, and (2) there is no guarantee of reflecting the underlying distribution. In this software, we implemented a dynamic sampling algorithm, based on the recent technology from the relational dynamic bayesian online locally exchangeable measures, that reduces the storage of data records in a large scale, and still provides accurate analysis of large streaming data. The software can be used for both online and offline data records.

Sim, Alex↗

Architecture independent environment for developing engineering software on MIMD computers

Engineers are constantly faced with solving problems of increasing complexity and detail. Multiple Instruction stream Multiple Data stream (MIMD) computers have been developed to overcome the performance limitations of serial computers. The hardware architectures of MIMD computers vary considerably and are much more sophisticated than serial computers. Developing large scale software for a variety of MIMD computers is difficult and expensive. There is a need to provide tools that facilitate programming these machines. First, the issues that must be considered to develop those tools are examined. The two main areas of concern were architecture independence and data management. Architecture independent software facilitates software portability and improves the longevity and utility of the software product. It provides some form of insurance for the investment of time and effort that goes into developing the software. The management of data is a crucial aspect of solving large engineering problems. It must be considered in light of the new hardware organizations that are available. Second, the functional design and implementation of a software environment that facilitates developing architecture independent software for large engineering applications are described. The topics of discussion include: a description of the model that supports the development of architecture independent software; identifying and exploiting concurrency within the application program; data coherence; engineering data base and memory management.

Valimohamed, Karim A.↗

Contextually aware roadside radiation measurement testbed

Here we demonstrate a contextually aware multimodal roadside radiation measurement detection testbed for traffic monitoring applications in nuclear nonproliferation. Many variables in traffic such as vehicle or cargo size, mass, speed, shape, and distance of closest approach can have significant impacts on the radiation measured from a vehicle-transported radiation source. These factors can lead to uncertainties in the analysis of the radiation source, especially for lower-strength radiation sources of interest. Our testbed, known as the Multimodal Measurement System (MMS) uses non-radiation sensors including magnetometers, geophones, radiofrequency receivers, cameras, and LiDAR to extract contextual information about vehicles passing by the system. These contextual data can then be fused with data from radiation measurements to increase the system’s sensitivity and accuracy in nuclear threat detection applications. This work describes the instrumentation of the MMS and its data acquisition pipeline. Furthermore, we describe the pre-analysis performed on the raw multimodal data streams for data fusion, and the high-level machine learning analyses for detection and characterization. The variety of sensors within the MMS provides a valuable testbed that can be used to identify the combinations of contextual sensors that provide the greatest improvements to radiation source detection and characterization within the restrictions for various proliferation detection applications. The MMS is also modular so that additional combinations of sensors can be explored in the future.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗

Cyber‐physical system testbed for power system monitoring and wide‐area control verification

The electric power system is intrinsically a cyber‐physical system (CPS) with power flowing inthe physical system and information flowing in the cyber‐network. Testbeds arecrucial for understanding the cyber‐physical interactions and provideenvironments for prototyping novel applications. This study proposes afour‐layer architecture for CPS testbeds with emphases on communication networkemulation and networked physical components. A configurable software‐definednetwork is employed to bridge physical components with wide‐area applicationsfor closed‐loop control. In order to distribute physically coupled devices intomultiple software simulations, this study proposes a data broker setup based ona distributed messaging environment to achieve low‐latency data streaming. Thedecoupled design with data streaming allows for building testbed components asmodules and running them in a distributed manner. Case studies verify the databroker setup for low‐latency sensing and actuation, as well as the communicationemulation setup for the desired network latency. Also illustrated is a replayattack scenario using synchrophasors in the Western Electricity CoordinatingCouncil (WECC) 181‐bus system for demonstrating the closed‐loop cyber‐physicalsimulation capability of the testbed.

Cui, Hantao↗

JANA2 Framework for Event Based and Triggerless Data Processing

Development of the second generation JANA2 multi-threaded event processing framework is ongoing through an LDRD initiative grant at Jefferson Lab. The framework is designed to take full advantage of all cores on modern many-core compute nodes. JANA2 efficiently handles both traditional hardware triggered event data and streaming data in online triggerless environments. Development is being done in conjunction with the Electron Ion Collider development. Anticipated to be the next large scale Nuclear Physics facility constructed. The core framework is written in modern C++ but includes an integrated Python interface. The status of development and summary of the more interesting features are presented.

Lawrence, David↗

Diaspora: Resilience-Enabling Services for Real-Time Distributed Workflows

The need for real-time processing to enable automated decision making and experimental steering has driven a shift from high-performance computing workflows on a centralized system to a distributed approach that integrates remote data sources, edge devices, and diverse compute facilities. Under this paradigm, data can be processed close to the source where it is generated, thus reducing latency and bandwidth usage. System resilience is thus a key challenge, requiring distributed workflows to survive component failures and to meet stringent quality-of-service requirements, which results in the need to mitigate anomalies such as congestion and low availability of resources. To address these challenges, we propose Diaspora, a unified resilience framework that is inspired by event-driven communication patterns used in public clouds. Specifically, we propose an event fabric that extends across sites, facilities, and computations to provide timely, reliable, and accurate information about data, application, and resource status. On top of the event fabric, we build resilience-enabling services that combine QoS-aware data streaming, resilient data views, resilient compute and data resources, and anomaly detection and prediction, all of which collectively enhance workflow resilience for these scientific cases.

Rao, Nageswara↗

Real Time Phasor Analytics (RTPA) and RTPA-SCR System Strength Online Tool

This presentation showcases the Real-Time Phasor Analytics (RTPA) framework for monitoring inertia and assessing system strength in power grids. RTPA is an open-source tool designed to standardize access to data from Power Management Units (PMUs) and Phasor Data Concentrators (PDCs). It facilitates real-time connectivity to multiple PDCs in accordance with the IEEE C37.118-2 standard and supports asynchronous data stream integration. Additionally, RTPA can simulate a PDC server streaming C37.118-2 data and provides Python bindings for seamless interaction with the framework, eliminating the need for direct Rust programming.

24 POWER TRANSMISSION AND DISTRIBUTION↗

Epi Archive: Automated Synthesis of Global Notifiable Disease Data

ObjectiveLANL has built software that automatically collects global notifiable disease data, synthesizes the data, and makes it available to humans and computers within the Biosurveillance Ecosystem (BSVE) as a novel data stream. These data have many applications including improving the prediction and early warning of disease events.IntroductionMost countries do not report national notifiable disease data in a machine-readable format. Data are often in the form of a file that contains text, tables and graphs summarizing weekly or monthly disease counts. This presents a problem when information is needed for more data intensive approaches to epidemiology, biosurveillance and public health.While most nations likely store incident data in a machine-readable format, governments are often hesitant to share data openly for a variety of reasons that include technical, political, economic, and motivational issues1.A survey conducted by LANL of notifiable disease data reporting in over fifty countries identified only a few websites that report data in a machine-readable format. The majority (>70%) produce reports as PDF files on a regular basis. The bulk of the PDF reports present data in a structured tabular format, while some report in natural language.The structure and format of PDF reports change often; this adds to the complexity of identifying and parsing the desired data. Not all websites publish in English, and it is common to find typos and clerical errors.LANL has developed a tool, Epi Archive, to collect global notifiable disease data automatically and continuously and make it uniform and readily accessible.MethodsWe conducted a survey of the national notifiable disease reporting systems notating how the data are reported and in what formats. We determined the minimal metadata that is required to contextualize incident counts properly, as well as optional metadata that is commonly found.The development of software to regularly ingest notifiable disease data and make it available involves three or four main steps: scraping, detecting, parsing and persisting.Scraping: we examine website design and determine reporting mechanisms for each country/website, as well as what varies across the reporting mechanisms. We then designed and wrote code to automate the downloading of the data for each country. We store all artifacts presented as files (PDF, XLSX, etc.) in their original form, along with appropriate metadata for parsing and data provenance.Detecting: This step is required when parsing structured non-machine-readable data such as tabular data in PDF files. We combined the Nurminen methodology of PDF table detection with in-house heuristics to find the desired data within PDF reports2.Parsing: We determined what to extract from each dataset and parsed these data into uniform data structures, correctly accommodating the variations in metadata (e.g., time interval definitions) and the various human languages.Persisting: We store the data in the Epi Archive database and make it available on the internet and through the BSVE. The data is persisted into a structured and normalized SQL database.ResultsThe Epi Archive tool currently contains national and/or subnational notifiable disease data from twenty nations. When a user accesses the Epi Archive site, they are prompted with four fields: country, subregion, disease of interest, and date duration. Upon form submission, a time series is generated from the users’ specifications. The generated graph can then be downloaded into a CSV file if a user is interested in performing personal analysis. Additionally, the data from Epi Archive can be reached through a REST API (Representational State Transfer Application Programming Interface).ConclusionsLANL, as part of a currently funded DTRA effort, is automatically and continually collecting global notifiable disease data. While 20 nations are in production, more are being brought online in the near future. These data are already being utilized and will have many applications including improving the prediction and early warning of disease events.References[1] van Panhuis WG, Paul P, Emerson C, et al. A systematic review of barriers to data sharing in public health. BMC Public Health. 2014. 14:1144. doi:10.1186/1471-2458-14-1144[2] Nurminen, Anssi. "Algorithmic extraction of data in tables in PDF documents." (2013).

Khalsa, Hari S.↗

MIMD computing in the USA - 1984

It is often said that the 1980s are becoming the decade of multiinstruction stream or MIMD computers, while the 1970s could be described as the decade of the SIMD (single instruction stream multiple data stream) computers. The availability of microprocessors and VLSI facilities has led to the proposal and construction of novel computer architectures based on linking many hundreds or even thousands of microprocessors, or specially designed VLSI chips. Some of the larger manufacturers offer computers with a small number of CPUs. Because of the variety of the new developments, it was decided to conduct a survey of proposed and existing MIMD computers in the U.S., taking into account a simple classification of the different devices. Particular attention is given to computers which are designed for numerical work with floating-point numbers and the solution of large problems in physics, chemistry, and engineering.

Hockney, R. W.↗

Chemical network problems solved on NASA/Goddard's massively parallel processor computer

The single instruction stream, multiple data stream Massively Parallel Processor (MPP) unit consists of 16,384 bit serial arithmetic processors configured as a 128 x 128 array whose speed can exceed that of current supercomputers (Cyber 205). The applicability of the MPP for solving reaction network problems is presented and discussed, including the mapping of the calculation to the architecture, and CPU timing comparisons.

Cho, Seog Y.↗

Particle sightings by the infrared telescope on Spacelab 2

Part of the objective of the cooled infrared telescope on Spacelab 2 was to determine the particle environment around the Space Shuttle. The telescope scanned the sky in six wavelength bands ranging from 2 to 100 microns with high time resolution. Dust particles could be identified from their particular signature in the data stream. Particle data from about 4 h early in the mission were analyzed in terms of size, color temperature, velocity, and time. The 1100 particles that were seen were slow moving and ranged in color temperature from 190 to 350 K with a few much hotter. The minimum detectable diameter is between 5 and 13 microns, depending on temperature. The size distribution resembles sample distributions collected at a Shuttle preparation facility. Although particle detection rates varied widely with time, no specific events were identified to be associated with particle production. It was not possible to determine the particle composition, although it was probably not ice.

Simpson, J. P.↗

Lessons learned supporting onboard solid-state recorders

With the advance of semiconductor technology, Solid-State Recorders (SSR) have matured and been accepted as primary onboard data storage devices. Their high reliability, simpler interface and control, and high flexibility have made the SSR's a superb choice in today's spacecraft design. While there are many benefits, the use of SSR's may also add significant complexity to ground data systems. For instance, real-time and playback data may be interleaved into the same data stream, making data sequencing and time ordering difficult. Stored data may be played back out of time order, increasing processing load significantly. Data may also be played back after being sorted by Virtual Channels in the SSR, potentially creating bursts in packet rates that exceed the real-time processing capabilities of the ground systems. This paper presents a summary of lessons learned through the efforts in supporting a number of NASA's missions that employ SSR's. It describes various problems encountered through the design process, and their potential impact on ground system performance, resources, and cost. Recommended approaches to minimizing the impact are demonstrated by examples. The discussion leads to the conclusion that the use of SSR's demands an even higher level of cooperation between spacecraft and ground system designers in order to build the most cost effective end-to-end system.

Shi, Jeff↗

Robust, Brillouin Active Embedded Fiber-Is-The-Sensor System in Smart Composite Structures

Extensive review of our proposed sensing scheme, based mainly on the forward Guided Acoustic Wave Brillouin Scattering (GAWBS) with backward stimulated Brillouin scattering (sBs) as an auxiliary scheme for system fault tolerance has been completed during this project period. This preliminary study is conducted for a number of reasons. The most significant reasons lie in the essential capability of the system to measure temperature and pressure. These two measurands have been proposed to be sensed by sBs in our proposal. Temperature and pressure/strain are important measurands in structural monitoring, so that the effectiveness of sensing by sBs needs to be further examined. It has been pointed out initially that sBs shift will be dependent on temperature and pressure/strain simultaneously. The shift versus temperature or strain is linear. Now, the question is how can these two measurands be separated when sBs is used to sense an environment, in which both temperature and strain are changing simultaneously. Typical sBs shift plotted versus strain and varying temperature is shown in Fig. 1. As is clear, a fiber initially stressed will relax with rising temperature. This is verified by a displacement to the right with rising temperature of the sBs shift vs strain curves in the figure. A way to circumvent this ambiguity is by employing two fibers, one pre-stressed and the other is a free fiber. The latter will measure temperature and subtracting data in the latter fiber from those of the former will give us net strain readings. This is a laborious approach, since it involves the use of two identical fibers, and this is hard to accomplish, especially when many sensors are needed. Additional multiplexing of the data stream for data subtraction becomes a necessity.

Yu, Chung↗

Reduction of ETS-VI Laser Communication Equipment Optical-Downlink Telemetry Collected During GOLD

Free-space laser communications experiments were conducted between the laser communication equipment (LCE) on board the Japanese Engineering Test Satellite VI (ETS-VI) and the ground station located at the Table Mountain Facility (TMF) during late 1995 and early 1996. This article describes the on-line data reduction process used to decode LCE telemetry (called E2) downlinked on the optical carrier during the Ground/Orbiter Lasercomm Demonstration (GOLD) experiments. The LCE has the capability of transmitting real-time sensor and status information at 128 kbps by modulating the onboard diode laser. The optical downlink was detected on the ground, bit synchronized, and the resulting data stream stored on a data recorder. The recorded data were subsequently decoded by on-line data processing that included cross-correlation of the known telemetry data format and the downlink data stream. Signals obtained from the processing can be useful not only in evaluating the characteristics of the LCE but also in understanding uplink and downlink signal quality.

Toyoshima, M.↗

Space-Based Range

Space-Based Range (SBR), previously known as Space-Based Telemetry and Range Safety (STARS), is a multicenter NASA proof-of-concept project to determine if space-based communications using NASA's Tracking and Data Relay Satellite System (TDRSS) can support the Range Safety functions of acquiring tracking data and generating flight termination signals, while also providing broadband Range User data such as voice, video, and vehicle/payload data. There was a successful test of the Range Safety system at Wallops Flight Facility (WFF) on December 20, 2005, on a two-stage Terrier-Orion spin-stabilized sounding rocket. SBR transmitted GPS tracking data and maintained links with two TDRSS satellites simultaneously during the 10-min flight. The payload section deployed a parachute, landed in the Atlantic Ocean about 90 miles downrange from the launch site, and was successfully recovered. During the Terrier-Orion tests flights, more than 99 percent of all forward commands and more than 95 percent of all return frames were successfully received and processed. The time latency necessary for a command to travel from WFF over landlines to White Sands Complex and then to the vehicle via TDRSS, be processed onboard, and then be sent back to WFF was between 1.0 s and 1.1 s. The forward-link margins for TDRS-10 (TDRS East [TDE]) were 11 dB to 12 dB plus or minus 2 dB, and for TDRS-4 (TDRS Spare [TDS]) were 9 dB to 10 dB plus or minus 1.5 dB. The return-link margins for both TDE and TDS were 6 dB to 8 dB plus or minus 3 dB. There were 11 flights on an F-15B at Dryden Flight Research Center (DFRC) between November 2006 and February 2007. The Range User system tested a 184-element TDRSS Ku-band (15 GHz) phased-array antenna with data rates of 5 Mbps and 10 Mbps. This data was a combination of black-and-white cockpit video, Range Safety tracking and transceiver data, and aircraft and antenna controller data streams. IP data formatting was used.

Source record↗

Soft Decision Analyzer and Method

A soft decision analyzer system is operable to interconnect soft decision communication equipment and analyze the operation thereof to detect symbol wise alignment between a test data stream and a reference data stream in a variety of operating conditions.

Steele, Glen F.↗

Soft Decision Analyzer and Method

A soft decision analyzer system is operable to interconnect soft decision communication equipment and analyze the operation thereof to detect symbol wise alignment between a test data stream and a reference data stream in a variety of operating conditions.

Steele, Glen F.↗

Soft decision analyzer and method

A soft decision analyzer system is operable to interconnect soft decision communication equipment and analyze the operation thereof to detect symbol wise alignment between a test data stream and a reference data stream in a variety of operating conditions.

Steele, Glen F.↗