Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “streaming data”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 181 records · Page 10
All sourcesNASA NTRSDOE OSTI
DOE OSTI2023

System and method associated with expedient detection and reconstruction of cyber events in a compact scenario representation using provenance tags and customizable policy

A system associated with detecting a cyber-attack and reconstructing events associated with a cyber-attack campaign, is disclosed. The system performs various operations that include receiving an audit data stream associated with cyber events. The system identifies trustworthiness values in a portion of data associated with the cyber events and assigns provenance tags to the portion of the data based on the identified trustworthiness values. An initial visual representation is generated based on the assigned provenance tags to the portion of the data. The initial visual representation is condensed based on a backward traversal of the initial visual representation in identifying a shortest path from a suspect node to an entry point node. A scenario visual representation is generated that specifies nodes most relevant to the cyber events associated with the cyber-attack based on the identified shortest path.A corresponding method and computer-readable medium are also disclosed.

Source record↗
DOE OSTI2021

AFRL Additive Manufacturing Modeling Series: Challenge 4, In Situ Mechanical Test of an IN625 Sample with Concurrent High-Energy Diffraction Microscopy Characterization

In this work, we describe 3D characterization of an additively manufactured Inconel 625 nickel-base superalloy specimen conducted during a uniaxial tension test using a suite of nondestructive x-ray techniques. High-energy diffraction microscopy in both near- and far-field modalities are employed in situ to track evolution of the material orientation and stress–strain fields at six points during the mechanical test, and these data streams are registered with micro-computed tomography reconstructions which probe the material density. This data volume was matched to a multi-modal serial sectioning characterization of the specimen taken after loading, described in this article’s companion. Twenty-eight grains which were monitored throughout the experiment were selected to form the basis for AFRL AM Modeling Series Challenge 4, Microscale Structure-to-Properties.

36 MATERIALS SCIENCE↗
DOE OSTI2020

Event Definition for the Automated Detection of Nuclear Proliferation Activities

In FY2020, Savannah River National Laboratory (SRNL) in collaboration with the Discovery Analytics Center (DAC) at Virginia Polytechnic Institute and State University (VT) began developing a demonstration prototype system that uses multiple machine learning and data analytic methods on largescale open data sources to identify new, developing, or undeclared nuclear programs. One of the most challenging aspects of applying machine learning techniques to such a problem is the high likelihood of extremely sparse data from disparate sources. To overcome this challenge, the current work will use a strategic combination of supervised, semi-supervised, and unsupervised learning techniques to ingest and fuse data streams to make a forecast of nuclear activities in a targeted geospatial location. Identifying potential data sources and training supervised learning algorithms is dependent upon the development of a robust foundation of targeted event domains that fundamentally define the nuclear activities of interest. This report documents the definition of a hierarchical structure for both nuclear activity and event domains that will be used to guide the research team in development or use of existing semantic dictionaries that are instrumental to searching, parsing, and categorizing events for the forecasting system’s use.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2023

Using Data-Driven Prediction of Downstream 1D River Flow to Overcome the Challenges of Hydrologic River Modeling

Methods for downstream river flow prediction can be categorized into physics-based and empirical approaches. Although based on well-studied physical relationships, physics-based models rely on numerous hydrologic variables characteristic of the specific river system that can be costly to acquire. Moreover, simulation is often computationally intensive. Conversely, empirical models require less information about the system being modeled and can capture a system’s interactions based on a smaller set of observed data. This article introduces two empirical methods to predict downstream hydraulic variables based on observed stream data: a linear programming (LP) model, and a convolutional neural network (CNN). We apply both empirical models within the Colorado River system to a site located on the Green River, downstream of the Yampa River confluence and Flaming Gorge Dam, and compare it to the physics-based model Streamflow Synthesis and Reservoir Regulation (SSARR) currently used by federal agencies. Results show that both proposed models significantly outperform the SSARR model. Moreover, the CNN model outperforms the LP model for hourly predictions whereas both perform similarly for daily predictions. Although less accurate than the CNN model at finer temporal resolution, the LP model is ideal for linear water scheduling tools.

13 HYDRO ENERGY↗
DOE OSTI2022

Sensing Electrical Networks Securely & Economically (SENSE)

The growing adoption of distributed energy resources (DERs) like battery energy storage systems and roof top solar/PV and the rapid penetration of electric vehicles (EVs), the electric grid is undergoing a major transformation with elevated stress on legacy grid assets. Despite a lot of expenditure to address these challenges, both in dollars and manpower, utilities have not been able to receive the value that was promised. The gains have been most visible at the transmission and substation level, especially where the main objective was improving operational and economic efficiency for the utility. Improving visibility and control at a few select points enhances the existing and established paradigm of centralized command and control. With changing load patterns, load types and the overall transition to an “active grid”, the centralized control and coordination paradigm gets challenged. To address the challenges, a new architecture and mechanism is needed, one that supports decentralized control and decision making, extracting value streams at the grid edge, particularly as the changes are fueled by transitions occurring in the distribution system. To address this, a communications and data processing platform, “GAMMA” was developed and demonstrated through the project. At the heart of the platform, are distributed, intelligent edge nodes with sensing and compute capabilities, that can record and analyze information locally. They are embedded in sensors and actuators specific to different distribution system applications. Phase 1 of the project focused on developing novel sensor technology that can be used for monitoring utility pole top distribution transformers. The sensors were designed with the objective of being low-cost, communicating with the GAMMA cloud using novel “delay-tolerant” networking using Bluetooth and a secure mobile application. They were non-intrusive in nature so that they can be installed quickly in the field, resulting in overall low cost of deployment and operations. Following the successful completion of Phase 1, the team manufactured 100 units for a field demonstration in Phase 2. The field demonstration was carried out on two real feeder systems with the local utility partner. In total, 100 sensors were installed and operated over a period of 6 months in the state of Georgia. The platform is operational end to end, with the cloud infrastructure deployed on a distributed, serverless environment that can serve multiple data streams, an analytics engine and a portal to securely view the data from multiple assets. The data collected through the GAMMA Mobile Phone app showcased the viability of the novel delay tolerant networking architecture, and the data processing algorithms developed through the course of the project, were successful in extracting important information about the overall network, improving the utility’s visibility and situational awareness in the distribution feeder.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2024

Integrated photonic encoder for low power and high-speed image processing

Abstract Modern lens designs are capable of resolving greater than 10 gigapixels, while advances in camera frame-rate and hyperspectral imaging have made data acquisition rates of Terapixel/second a real possibility. The main bottlenecks preventing such high data-rate systems are power consumption and data storage. In this work, we show that analog photonic encoders could address this challenge, enabling high-speed image compression using orders-of-magnitude lower power than digital electronics. Our approach relies on a silicon-photonics front-end to compress raw image data, foregoing energy-intensive image conditioning and reducing data storage requirements. The compression scheme uses a passive disordered photonic structure to perform kernel-type random projections of the raw image data with minimal power consumption and low latency. A back-end neural network can then reconstruct the original images with structural similarity exceeding 90%. This scheme has the potential to process data streams exceeding Terapixel/second using less than 100 fJ/pixel, providing a path to ultra-high-resolution data and image acquisition systems.

47 OTHER INSTRUMENTATION↗
DOE OSTI2021

Online Voltage Event Detection Using Synchrophasor Data with Structured Sparsity-Inducing Norms

This paper develops an accurate and computationally efficient data-driven framework to detect voltage events from PMU data streams. It develops an innovative Proximal Bilateral Random Projection (PBRP) algorithm to quickly decompose the PMU data matrix into a low-rank matrix, a row-sparse event-pattern matrix and a noise matrix. Here, the row-sparse pattern matrix significantly distinguishes events from normal behavior. These matrices are then fed into a clustering algorithm to separate voltage events from normal operating conditions. Large-scale numerical study results on real-world PMU data show that the proposed algorithm is computationally more efficient and achieves higher F scores than state-of-the-art benchmarks.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2020

"Sintering" Models and In-Situ Experiments: Data Assimilation for Microstructure Prediction in SLS Additive Manufacturing of Nylon Components

Selective laser sintering methods are workhorses for additively manufacturing polymer-based components. The ease of rapid prototyping also means it is easy to produce illicit components. It is necessary to have a data-calibrated in-situ physical model of the build process in order to predict expected and defective microstructure characteristics that inform component provenance. Toward this end, sintering models are calibrated and characteristics such as component defects are explored. This is accomplished by assimilating multiple data streams, imaging analysis, and computational model predictions in an adaptive Bayesian parameter estimation algorithm. From these data sources, along with a phase-field model, bulk porosity distributions are inferred. Model parameters are constrained to physically-relevant search directions by sensitivity analysis, and then matched to predictions using adaptive sampling. Using this feedback loop, data-constrained estimates of sintering model parameters along with uncertainty bounds are obtained.

3D printing, Additive Manufacturing, polymer, sint↗
DOE OSTI2022

Volcano infrasound: progress and future directions

Over the past two decades (2000–2020), volcano infrasound (acoustic waves with frequencies less than 20 Hz propagating in the atmosphere) has evolved from an area of academic research to a useful monitoring tool. As a result, infrasound is routinely used by volcano observatories around the world to detect, locate, and characterize volcanic activity. It is particularly useful in confirming subaerial activity and monitoring remote eruptions, and it has shown promise in forecasting paroxysmal activity at open-vent systems. Fundamental research on volcano infrasound is providing substantial new insights on eruption dynamics and volcanic processes and will continue to do so over the next decade. The increased availability of infrasound sensors will expand observations of varied eruption styles, and the associated increase in data volume will make machine learning workflows more feasible. More sophisticated modeling will be applied to examine infrasound source and propagation effects from local to global distances, leading to improved infrasound-derived estimates of eruption properties. Future work will use infrasound to detect, locate, and characterize moving flows, such as pyroclastic density currents, lahars, rockfalls, lava flows, and avalanches. Infrasound observations will be further integrated with other data streams, such as seismic, ground- and satellite-based thermal and visual imagery, geodetic, lightning, and gas data. The volcano infrasound community should continue efforts to make data and codes accessible and to improve diversity, equity, and inclusion in the field. In summary, the next decade of volcano infrasound research will continue to advance our understanding of complex volcano processes through increased data availability, sensor technologies, enhanced modeling capabilities, and novel data analysis methods that will improve hazard detection and mitigation.

58 GEOSCIENCES↗
DOE OSTI2025

Applications of visualization technology in the structural sciences

The structural sciences are undergoing a transformation driven by advancements in visualization technologies that aid researchers in understanding and communicating experimental data from complex molecular systems. New applications of integrative structural biological and biophysical approaches add a wide variety of complementary information from a broad range of scientific disciplines. These approaches extend structural biophysical methodologies to enable research by the incorporation of a variety of data streams and utilization of tools like molecular graphics, virtual reality, and machine learning. To redefine how structural data—particularly from cryo-electron microscopy and x-ray crystallography—are fed forward for scientific exploration and communication, the advances in tools for data visualization and interpretation have been critical. By bringing molecular systems into an interactive three-dimensional space, these novel technologies enhance research workflows, facilitate structure-based drug design, and create engaging educational experiences. Taken together, these visualization innovations are essential tools for advancing the field by making concepts more accessible and compelling.

Eng, Edward T. [New York Structural Biology Center↗
DOE OSTI2022

Classification and Localization of Fracture-Hit Events in Low-Frequency Distributed Acoustic Sensing Strain Rate with Convolutional Neural Networks

Summary Distributed acoustic sensing (DAS) has been used in the oil and gas industry as an advanced technology for surveillance and diagnostics. Operators use DAS to monitor hydraulic fracturing activities, examine well stimulation efficacy, and estimate complex fracture system geometries. Particularly, low-frequency DAS can detect geomechanical events such as fracture hits because hydraulic fractures propagate and create strain rate variations in the rock. Analysis of DAS data today is mostly done post-job and subject to interpretation methods. However, the continuous and dense data stream generated live by DAS poses the opportunity for more efficient and accurate real-time data-driven analysis. The objective of this study is to develop a machine learning-based workflow that can identify and locate fracture-hit events in simulated strain rate responses correlated with low-frequency DAS data. In this paper, “fracture hit” refers to a hydraulic fracture originating from a stimulated well intersecting an offset well. We start with building a single fracture propagation model to produce strain rate patterns observed at a hypothetical monitoring well. This model is used to generate two sets of strain rate responses with one set containing fracture-hit events. The labeled synthetic data are then used to train a custom convolutional neural network (CNN) model for identifying the presence of fracture-hit events. The same model is trained again for locating the event with the output layer of the model replaced with linear units. We achieved near-perfect predictions for both event classification and localization. These promising results prove the feasibility of using CNN for real-time event detection from fiber-optic sensing data. Additionally, we use edge detection techniques to recognize fracture-hit event patterns in strain rate images. The fracture-hit location can be identified using recognized pixels in the image. The accuracy of edge detection-based location identification is also plausible, but edge detection is dependent on the assumption of pattern shape and image quality, hence it is less robust compared to CNN models. This comparison further supports the need for CNN applications in image-based real-time fiber-optic sensing event detection.

Engineering↗
DOE OSTI2024

Navigating Exascale Operational Data Analytics: From Inundation to Insight

In this paper, we address the challenges in achieving sustainable data-driven efficiency by providing a detailed exploration of the end-to-end operational data analytics (ODA) framework that evolved through two generations of supercomputer systems at the Oak Ridge Leadership Computing Facility (OLCF). This framework addresses large data streams ingested from heavily instrumented HPC environment that accumulates multi-terabytes per day. We outline the multifaceted data life cycle across HPC procurement, operations, and research & development, identifying key obstacles and design decisions that shape effective strategies in building and supporting data pipelines end-to-end. By sharing key insights and lessons learned from our experience, we offer recommendations for the HPC community on enabling sustainable operational data analytics and beyond. Our contributions aim to bridge the gap between potential and real benefits of operational data, guiding future efforts towards integrated and sustainable operational intelligence in high-performance computing environments.

Shin, Woong↗
DOE OSTI2025

A digital twin platform for building performance monitoring and optimization: Performance simulation and case studies

Advancements in sensor technology, data analytics, affordable compute, and communication infrastructure have paved the way for Digital Twin technology in optimizing building operations and controls. This study presents the development of an open and interoperable web-based Digital Twin platform for integrating diverse data streams and facilitating effective user interactions. The platform utilizes modern technologies for the web framework and time-series data management, ensuring scalability and responsiveness. The backend supports seamless integration of diverse data sources and emulators, incorporating data from building sensors and meters, external weather Application Programming Interfaces, and advanced EnergyPlus simulation models of the building and its energy systems including the Distributed Energy Resources that are formulated in Functional Mockup Units. A simulation case study was conducted with FlexLab, a test facility on Lawrence Berkeley National Laboratory campus. The case study includes normal operations, Distributed Energy Resource integration, and power outage scenarios, to illustrate the Digital Twin’s ability to provide critical insights into energy performance and thermal resilience. The results demonstrated the platform’s potential as a decision-support tool for optimizing building energy performance and enhancing resilience against extreme weather events. Future work will focus on deploying the Digital Twin platform to a real building for field validation, extending its capabilities to cover more scenarios such as bidirectional Electric Vehicle interactions, and enhancing user engagement.

EnergyPlus↗
DOE OSTI2025

Pinch Heat Integration Tool (PIT) v0.9

The pinch heat integration tool (PIT) is a multi-module tool designed for identifying and analyzing thermal energy integration at facilities. It generates pinch and grand composite curves to identify optimal streams for heat exchange and industrial heat pumps. Users input heating and cooling stream data via an Excel sheet. The tool comprises four modules: 1. 'Main Pinch' for creating a pinch diagram using user-provided data. 2. 'GCC' for generating a grand composite curve to identify utility optimization opportunities. 3. 'Heat Exchanger' for analyzing stream combinations for heat exchange. 4. 'Heat Pump' for evaluating streams as sources and sinks for heat pumps and calculating the heat pump's coefficient of performance based on the selected refrigerant.

Karki, Unique [Lawrence Berkeley National Laborato↗
DOE OSTI2023

Workflows Community Summit 2022: A Roadmap Revolution

Scientific workflows have become integral tools in broad scientific computing use cases. Science discovery is increasingly dependent on workflows to orchestrate large and complex scientific experiments that range from the execution of a cloud-based data preprocessing pipeline to multi-facility instrument-to-edge-to-HPC computational workflows. Given the changing landscape of scientific computing (often referred to as a computing continuum) and the evolving needs of emerging scientific applications, it is paramount that the development of novel scientific workflows and system functionalities seek to increase the efficiency, resilience, and pervasiveness of existing systems and applications. Specifically, the proliferation of machine learning/artificial intelligence (ML/AI) workflows, need for processing large-scale datasets produced by instruments at the edge, intensification of near real-time data processing, support for long-term experiment campaigns, and emergence of quantum computing as an adjunct to HPC, have significantly changed the functional and operational requirements of workflow systems. Workflow systems now need to, for example, support data streams from the edge-to-cloud-to-HPC, enable the management of many small-sized files, allow data reduction while ensuring high accuracy, orchestrate distributed services (workflows, instruments, data movement, provenance, publication, etc.) across computing and user facilities, among others. Further, to accelerate science, it is also necessary that these systems implement specifications/standards and APIs for seamless (horizontal and vertical) integration between systems and applications, as well as enable the publication of workflows and their associated products according to the FAIR principles.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2022

Mu2e DAQ and slow control systems

The Mu2e experiment at the Fermilab Muon Campus will search for the coherent neutrinoless conversion of a muon into an electron in the feld of an aluminum nucleus with a sensitivity improvement by a factor of 10,000 over existing limits. The Mu2e Trigger and Data Acquisition System (TDAQ) uses otsdaq as the online Data Acquisition System (DAQ) solution. Developed at Fermilab, otsdaq integrates both the artdaq DAQ and the art analysis frameworks for event transfer, fltering, and processing. otsdaq is an online DAQ software suite with a focus on fexibility and scalability and provides a multiuser, web-based, interface accessible through a web browser. The data stream from the detector subsystems is read by a software flter algorithm that selects events which are combined with the data fux coming from a Cosmic Ray Veto System. The Detector Control System (DCS) has been developed using the Experimental Physics and Industrial Control System (EPICS) open source platform for monitoring, controlling, alarming, and archiving. The DCS System has been integrated into otsdaq. A prototype of the TDAQ and the DCS systems has been built at Fermilab’s Feynman Computing Center. In this paper, we report on the progress of the integration of this prototype in the online otsdaq software.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗
DOE OSTI2021

Data acquisition and slow control interface for the Mu2e experiment

The Mu2e experiment at the Fermilab Muon Campus will search for the coherent neutrinoless conversion of a muon into an electron in the field of an aluminum nucleus with a sensitivity improvement by a factor of 10000 over existing limits. The Mu2e Trigger and Data Acquisition System (TDAQ) uses otsdaq as the online Data Acquisition System (DAQ) solution. Developed at Fermilab, otsdaq integrates both the artdaq DAQ and the art analysis frameworks for event transfer, filtering, and processing. otsdaq is an online DAQ software suite with a focus on flexibility and scalability and provides a multi-user, web-based, interface accessible through a web browser. The data stream from the detector subsystems is read by a software filter algorithm that selects events which are combined with the data flux coming from a cosmic ray veto system. The Detector Control System (DCS) has been developed using the Experimental Physics and Industrial Control System (EPICS) open source platform for monitoring, controlling, alarming, and archiving. The DCS system has been integrated into otsdaq. A prototype of the TDAQ and the DCS systems has been built at Fermilab's Feynman Computing Center. In this study, we report on the progress of the integration of this prototype in the online otsdaq software.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗
DOE OSTI2020

Evaluation of a streaming readout solution for Jefferson Lab experiments

Jefferson Lab (JLab) operates the CEBAF accelerator that provides a high energy (up to 12 GeV) and high current (up to 150 µA) polarized electron beam for fixed target experiments. JLab's mission is to provide substantial progress in understanding the Quantum Chromo Dynamics (QCD), the current theory of the spectrum and the dynamics of hadronic matter. Given the broad scientific program a general-purpose detector, CLAS12, has been designed and constructed. Such a detector requires a sophisticated trigger and current experiments use an on-line FPGA-based system that relies upon custom firmware and electronics both of which are difficult reconfigure from one experiment to the next. To overcome these challenges transitioning to a streaming readout has been proposed since it would allow a more flexible, easier to debug, software trigger to be developed. To facilitate this JLab is gaining experience with streaming readout systems and is actively working to design and test a streaming data acquisition architecture. Significant work has already been done to make existing front-end electronics compatible with operation in streaming mode. Software-wise, much work remains to be done but an existing framework, TriDAS, which was developed for underwater neutrino telescopes can, in principle, be used with the existing JLab hardware. A test stand is already in place in the INDRA DAQ development lab at JLab. This paper will present the development of this TriDAS based test system and it?s operation in-beam using parts of the CLAS12 detector to evaluate streaming readout for a large general purpose detector.

Battaglieri, Marco↗