Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Streaming data”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 613 records · Page 34

The Grey Zone Test Range Integrated Urban Simulation Environment

Sandia National Laboratories is part of the government test and evaluation team for the Defense Advanced Research Projects Agency Collection and Monitoring via Planning for Active Situational Scenarios program. The program is designed to better understand competition in the area between peace and conventional conflict when adversary actions are subtle and difficult to detect. For the purposes of test and evaluation, Sandia conducted a range of activities for the program: creation of the Grey Zone Test Range; design of the data stream for a user experiment conducted with U.S. Indo-Pacific Command; design, implementation, and execution of the formal evaluation; and analysis and summary of the evaluation results. This report details Sandia's activities and provides additional information on the Grey Zone Test Range urban simulation environment developed to evaluate the performer technologies.

45 MILITARY TECHNOLOGY, WEAPONRY, AND NATIONAL DEF↗

A Grand Challenge "Uncertainty Project" to Accelerate Advances in Earth System Predictability: AI-Enabled Concepts and Applications

This proposal is emerging from GISS ModelE3 ESM development in the area of cloud physics, so we begin with an example of research needs/gaps from that work. Here, some of our greatest development concerns arise where we lack fundamental process-level understanding, as in ice formation. Namely, it is currently unclear what is the main process that is forming the majority of ice crystals in commonly occurring convection, apparently via secondary ice production at warm temperatures. We are keenly awaiting laboratory data for candidate mechanisms, which is not yet in hand to crucially establish their efficiency. Our progress is also hampered by a lack of uncertainty characterization in currently available measurements of ice crystal number size distributions. Furthermore, the same multiplication process may be responsible for a majority of ice crystals in many extratropical mixed-phase clouds, whose variable representation in CMIP6 ESMs may be a leading cause of differences in cloud phase feedback and ECS. Yet we have been required to deliver an ESM with the cloud physics knowledge at hand. The proposed grand challenge project is AI-enabled via application of machine learning (ML) to climate model and observational data streams (focal area 3), and applications include AI-guided observing system design and model/component/parameterization selection (areas 1 and 2). The project is structurally agnostic as to whether model or observing system components use AI approaches or not, but uncertainties must be estimated and propagatable in both.

58 GEOSCIENCES↗

A new era of observationally-infused E3SM: GANs for unifying imagery archives

This paper presents an idea to develop a “Rosetta Stone” for unifying observations from various satellite or remote sensors into a common format that would vastly advance our ability to exploit existing datasets for improving predictability within Earth System Models (ESMs). While the applications of such a unified archive are broad, we believe it will be a critical step toward ushering in a new generation of ESMs that are richly informed, guided by, and validated by extensive observational data. With the vast quantity of both remotely-sensed and in-situ data streams available and coming online, new approaches are needed that can harmonize and thus fully exploit these expensive datasets. While we present the broader idea, we point to examples of applications that impact the water cycle and its representation in ESMs.

58 GEOSCIENCES↗

Open Radiation Monitoring: Histogram Builder Module Design

The Open Radiation Monitoring Project seeks to develop and demonstrate a modular radiation detection architecture designed specifically for use in arms control treaty verification (ACTV) applications that will facilitate rapid development of trusted systems to meet the needs of potential future treaties. A modular architecture can be used to reduce more complex systems to a series of single purpose building blocks, thereby facilitating equipment inspection and in turn building trust in the equipment by all treaty parties. Furthermore, a modular architecture can be used to control data flow within the measurement system, reducing the risk of "hidden switches" and constraining the amount of sensitive information that could potentially be inadvertently leaked. This report details the first revision of a prototype circuit that will convert analog pulses directly into a histogrammed data set for further processing. The circuit was designed with both spectroscopy and multiplicity analysis in mind but can, in principle, be used to reduce any raw data stream into a histogram. The number of output channels is limited, and the histogram bin ranges are user configurable to allow for non-uniform and discontinuous bins, which makes it possible to restrict the information being passed down stream if desired. Pulse processing relies entirely on analog circuitry and non- programmable logic, which enables operation without the need for a central processor or other programmable control unit. The circuit remains untested under the Open Radiation Monitoring project due to the closure of the sponsoring program. However, further development and testing is scheduled to take place in support of a purpose-built trusted verification system development effort known as COGNIZANT, which demonstrates the potential benefit of developing a suite of modular trusted system components.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗

The Building Adapter: Automatic Mapping of Commercial Buildings for Scalable Building Analytics

This project creates new solutions for the manual metadata mapping problem: the costly process of creating a match between a building’s sensor data streams and the inputs of a building analytics engine. This goal is achieved by creating and improving techniques for metadata inference: automatically constructing new contextual information for sensing and control points based on the sensor point names and the raw time series values. The objective is to enable vendors to apply building analytics to 90% of buildings with no manual mapping, and to 10% of buildings with a 90% reduction in manual mapping. These targets are set for all types of metadata required by current analytics engines, including type, location, equipment type, and other relationships. The outcome of this project is a suite of solutions to the manual mapping problem collectively called the Building Adapter that allows vendors to apply analytics engines to new buildings at a significantly reduced cost.

96 KNOWLEDGE MANAGEMENT AND PRESERVATION↗

Oak Ridge National Laboratory Pilot Demonstration of an Attestation and Anomaly Detection Framework using Distributed Ledger Technology for Power Grid Infrastructure

This report summarizes the design and pilot demonstration of a framework called Grid Guard that was created to provide increased data and device trustworthiness to electric grid devices by leveraging distributed ledger technology (DLT), specifically blockchain. Grid Guard contains a combination of core cryptographic methods such as the secure hash algorithm (SHA), and asymmetric cryptography, private permissioned blockchain, baselining configuration data, consensus algorithm (Raft) and the Hyperledger Fabric (HLF) framework. The system implements a low energy, fast, and robust enhancement to system trustworthiness within and across electric grid systems such as substations, control centers and metering infrastructures. Blockchain is a distributed database structured that provides a practically unalterable (immutable) timeline of stored transactions. By relying on hashing and the Raft consensus algorithm, if an entity tries to illegitimately alter a record at one instance of the database the other ledger nodes are not altered. They work to cross-reference each other and easily locate any incorrectly added data and remove it. The bulk raw data is stored in an off-chain storage (outside of the blockchain ledger) and a hash of this baseline data is stored in the Blockchain ledger via hashing windows of time-series and configuration data, after aggregation and filtering. The bulk off-chain data repository is then considered to be trust-anchored using the hashes stored in the blockchain. To secure the electric grid testbed devices and data, device configuration baselines were compared to those baselines that had been previously stored in the ledger. Statistical baselines for device configurations, network communication patterns, and high-speed sensor data are calculated and then stored off-chain and hashes stored in the ledger. Measurements such as three-phase voltage and current, frequency, breaker status, protection scheme settings, network configuration settings (and other device configuration artifacts) and network traffic features (packet interarrival times) are compared every minute or other selected time windows. During phase 1 of the Grid Guard DLT project different DLT technologies were studies, and an assessment was performed on DLT technology vulnerabilities, uses, and key characteristics. DLT consensus protocols were studies (e.g., RAFT, named after Reliable, Replicated, Redundant, And Fault-Tolerant). Also, cryptography, public, private and permissioned or permissionless systems were assessed. Grid Guard implements a permissioned private DLT. Consensus algorithm selection and choice of DLT implementation depended heavily on the use-case. For this use-case, parameters were selected to measure performance and existing tools for assessment. Benchmarking was performed theoretically and practically. During phase 2 hashed transactions/blocks were inserted into the ledger every second. During phase 2 of the Grid Guard DLT project, a prototype framework was developed and demonstrated for attestation of critical substation devices and data using precision timing systems that use PTP and IRIG-B protocols) on a testbed of operational devices that emulated a distribution substation, control center, and power metering infrastructure using real Operational Technology (OT). The testbed includes OT devices such as protective relays, human machine interfaces (HMI), and power meters. To determine when to collect and compare system and network baselines, an initial examination of an anomaly detection capability to identify malicious manipulation of data streams was conducted. The resulting anomaly detection was demonstrated in a set of experiments and leveraged to trigger device artifact attestation checks. Attestation checks occur against device configuration baselines when compared with the immutable blockchain-stored baselines, which provided a cryptographically supported means by which to store baselines. The electrical substation-grid testbed was created to test the Grid Guard framework. The testbed emulates the operations of a portion of a power grid and SCADA systems as closely as possible. The testbed integrates real protocols, mainly IEC 61850 standard protocols, such as the Sampled Value (SV) and the GOOSE protocols. The testbed also supports DNP3 and other layer 2 and layer 3 protocols such as Telnet, SSH, SFTP/FTP and other proprietary protocols needed to connect to industrial control system equipment. The testbed emulates real power conditions using the OpalRT hardware-in-the-loop (HIL) device which can create fault situations that cannot be easily tested on real systems. The electrical substation-grid testbed was created using real measurement, communication, and protection devices that electrical utilities commonly use.

24 POWER TRANSMISSION AND DISTRIBUTION↗

Tools for Assessing Performance: FY23-Q1 Report [Slides]

LANL was tasked with working with NREL to implement QUIC within the TAP API by the end of Q1, FY2023. If QUICURB cannot be implemented in a way that it can be run outside of the QUIC platform, there will be no further R&D funding for QUIC development. QUIC includes a graphical user interface that imports and preprocesses the various input data streams (3D building databases, ambient wind, vegetation, etc.) and writes them in a format that the QUICURB Fortran executable requires. In order to interface with the TAP API, a Python script was developed that would replace the functionality previously only available within QUICGUI. LANL worked with NREL to test the Python script to ensure that it was operation and fulfilled the requirements.

58 GEOSCIENCES↗

Highly Segmented Silicon Strip Detectors for Radiation Detection (LDRD Final Report)

Large area highly segmented silicon strip detectors were investigated for use in applied radiation detection applications. Two ASIC-based readout electronics solutions were tested for compatibility. During the analysis, it was determined that significant computational resources and highly specialized firmware are required to reconstruct the data into a meaningful data stream for generalized spectroscopic performance. Future investment in the technology and careful choice of readout ASIC is expected to demonstrate relevance in measuring high activity samples.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗

Sentinel Devices LLC (CRADA Final Report)

Industrial equipment is a critical component of virtually all at-scale manufacturing and infrastructure. Broadly speaking, modern equipment is predominantly controlled using hardened digital controllers – computers designed to be able to operate continuously for sometimes extremely long periods of time, with little to no maintenance. Due to the nature of how these digital controllers have evolved, they are solely optimized to execute a single task, and do not have the capabilities or resources to monitor or analyze their internal state beyond simple execution of their program. As a result, for many “common sense” situations where individual data points can be easily determined to be out-of-normal, the controllers are unable to identify these incorrect operational modes unless a human has explicitly programmed in detection of this degradation. This project seeks to develop an AI/ML system which can identify incorrect or anomalous trends in industrial data streams, of exactly the kind that would be produced and seen by these digital controllers, with a minimal amount of computing resources. The benefits produced by developing this system would ultimately be self-monitoring and self-reporting infrastructure, capable of identifying and alerting humans to issues as soon as they happen, potentially long before they have the chance to impact the industrial process.

42 ENGINEERING↗

How Dynamic Time Warping Can Assist Conventional Cross-correlation

Waveform cross-correlation is a sensitive phase-matched filtering technique that can detect seismic events for nuclear explosion monitoring. However, there are outstanding challenges with correlation detectors, most notably a direct dependence on the completeness of the waveform template library. To ameliorate these challenges, we investigate how dynamic time warping (DTW) may make waveform correlation more robust. DTW analyzes the differences between two time series and attempts to “warp” one time series relative to another in a recursive manner. We apply DTW to synthetic earthquake and recorded explosion templates to expand the capability of correlation detectors. We explore what conditions (e.g., source, station distance, frequency bands) and/or DTW algorithms generate stronger correlation scores. We show that DTW performs well on noisy signals and can dramatically improve the cross-correlation coefficient between a template and data-stream waveform. We conclude with recommendations on how to utilize DTW in nuclear monitoring detection.

45 MILITARY TECHNOLOGY, WEAPONRY, AND NATIONAL DEF↗

Diaspora: Resilience-enabling services for science from HPC to edge

Scientific applications of interest to DOE must increasingly engage distributed resources (e.g., instruments, remote computers, data stores, edge devices) and deliver more stringent levels of service (e.g., uninterrupted processing of experiment data streams). In such systems, state is distributed and components can fail in many ways, often silently, making application resilience a major concern. Addressing the resilience needs of such applications requires methods for gaining knowledge of resources and applications and for translating that knowledge into action. We are working on addressing these needs in the context of multi-messenger astronomy, where detecting and responding to unusual transient events in multiple cosmic messengers (gravitational wave, electromagnetic, high- energy particles) from different instruments leads to a federated learning problem.

47 OTHER INSTRUMENTATION↗

Graph-Learning-Assisted State and Event Tracking for Solar-Penetrated Power Grids with Heterogeneous Data Sources

Unlike transmission systems, distribution systems do not typically contain sufficient metering to enable real-time state estimation. The lack of sufficient real-time measurements prohibits accurate and timely monitoring of the state of distribution systems. As a result, control and optimal operation of distribution systems, especially those containing large numbers of renewable generation units are not possible without proper data and information about the current state of the system. The main motivation of this project is to address this shortcoming by developing an approach which provides “predicted” real-time measurements so that they can be used to execute a distribution system state estimator. Thus, the objective of the project is to make the distribution systems fully observable, such that the hosting capacity for solar generation can be accurately estimated, and unnecessary solar curtailments can be avoided. In order to accomplish this goal, the project investigated the use of a grid-model-informed machine learning (ML) tool which integrates heterogeneous data streams obtained from AMI meters, SCADA as well as PMU measurements and created synchronous measurement snapshots for the state estimator (SE); and developed a hybrid robust SE which provides not only accurate state estimates but also real-time feedback for the ML model refinement.

14 SOLAR ENERGY↗

Developing and Managing Data Acquisition Software Using Spack

The Data Acquisition systems of particle physics experiments regularly push the boundaries of high-throughput computing, demanding low-latency collection of data from thousands of devices, collating data into time-sliced events, processing these events and making trigger decisions, and writing the selected data streams to disk. To accomplish these tasks, the DAQ Engineering and Operations department at Fermilab leverages multiple software libraries and builds reusable DAQ frameworks on top. These libraries must be delivered in well-defined bundles and are thoroughly tested for compatibility and functionality before being deployed to live detectors. We have several techniques used to ensure that a consistent set of dependencies can be delivered and re-created at need. We must also support active development of DAQ software components, ideally in an environment as close as possible to that of the detectors. This development often occurs across multiple packages which have to be built in concert and features tested in a consistent and reproducible manner. I will present our scheme for accomplishing these goals using Spack environments, bundle packages, and Github Actions-based CI.

Flumerfelt, Eric [Fermilab]↗

Tracking carbon dioxide in the Southern Ocean (Final Technical Report)

We investigated the discrepancy in observed and modeled Southern Ocean carbon dioxide (CO 2 ) absorption through the lens of the Energy Exascale Earth System Model (E3SM), leveraging recent and upcoming E3SM development advances to simulate Lagrangian Biogeochemical-Argo floats. E3SM output, together with observational data streams and output from the evolving CMIP6 archive, are being used to study the spatiotemporal bias in modeled Southern Ocean CO 2 flux.

54 ENVIRONMENTAL SCIENCES↗

The Sensor Dilemma in Intelligent Transportation Systems

Intelligent Transportation Systems (ITS) are at the forefront in advancing the way we interact and perceive with the transportation network. This revolution is fueled by the significant advancement in sensor perception technologies such as Radar, LiDAR and Video Imaging which are the most popular modalities for ITS. Real-time perception data from these sensors allows intelligent infrastructure side decision making to improve the energy, efficiency and safety at traffic intersections. As traffic departments across the United States are transitioning from traditional loop detectors / emulators and embracing newer technologies, they are often left with a dilemma in choosing a sensor technology for infrastructure-based perception which is reliable, inexpensive, easy to setup and has robust performance in varying weather conditions. However, choosing a sensor which checks all boxes is not straightforward as every sensor type has unique benefits and drawbacks. Radar is excellent at detecting long range vehicles and weather resistance but lacks high resolution. LiDAR is expensive and weather-sensitive, while cameras provide rich visual data at a low cost but are constrained by lighting and visibility. This study examines Radar, LiDAR and camera sensors capabilities to ascertain whether any of these qualifies as the "best" sensor for ITS perception. Through this evaluation, we hope to draw attention to the necessity of National Renewable Energy Laboratory's (NREL) Infrastructure Perception and Control (IPC) framework which presents a multi-sensor track data fusion engine to assimilate multiple data streams in order to provide robust and reliable perception. While no single sensor can meet all the demands of ITS, a hybrid approach combining multiple sensor modalities like Radar, LiDAR and cameras, offers the most robust solution for enhancing the safety and efficiency in intelligent transportation systems.

33 ADVANCED PROPULSION SYSTEMS↗

High-frequency Data Integration for Landscape Model Calibration of Carbon Fluxes Across Diverse Tidal Marshes

Terrestrial Aquatic Interfaces (TAIs), and tidal wetlands in particular, store large amounts of carbon yet are not well represented in Earth System Models (ESMs). Predictions of carbon cycling and greenhouse gas (GHG) emissions in tidal wetlands are highly uncertain. Eddy covariance (EC) towers provide ecosystem-scale GHG flux data at a temporal resolution (every 30min) that is helpful for parameterizing and improving mechanistic realism in ESMs. We propose to use a network of eddy covariance towers and standardized ancillary data streams, along with mesocosm experiments and statistical analyses, across diverse tidal wetlands of North America to develop and improve biogeochemical modeling at the TAI. Our overarching objective is to improve understanding and process-based modeling of gross primary productivity (GPP) and CH4 emission responses, both non-linear and asynchronous, to stressors including plant inundation, disturbance, salinity and nitrogen loading.

54 ENVIRONMENTAL SCIENCES↗

Cybersecurity Center for Offshore Wind Energy (Final Project Report)

This project establishes a Cybersecurity Center for Offshore Wind Energy with the objective of designing and operating a cyber-physical testbed for wind energy farms (WEFs) that enables comprehensive cybersecurity research. The testbed incorporates a Supervisory Control and Data Acquisition (SCADA) system connected to turbine models via industrial-grade programmable logic controllers (PLCs) and remote terminal units (RTUs). It supports side-channel data acquisition, implementation and analysis of various cyberattack scenarios, and development of attack detection, mitigation, and best-practice guidance tailored to wind energy systems. During the project, the team expanded the number and fidelity of mathematical turbine models (MTMs), integrated these models with SCADA infrastructure, and deployed a scaled physical turbine and associated sensors. High-resolution operational and side-channel data streams were collected and used to refine machine-learning (ML)-based attack detection systems and to extend the WindCRAFT framework to multi-turbine threat scenarios. The project demonstrated a realistic, scalable environment for evaluating cyber threats, validated attack detection approaches using enriched datasets, and identified new multi-turbine and inter-turbine communication attack vectors. The resulting testbed, models, and security mechanisms provide a foundation for ongoing R&D and deployment of cyber-resilient offshore wind energy systems.

17 WIND ENERGY↗

Continual Learning for Production-Level Machine Learning in Particle Accelerators

Particle accelerators operate in complex environments where data distribution can change dynamically, leading to data drifts that significantly challenge Machine Learning (ML) models. These non-stationary conditions often cause ML models to deteriorate in performance, making it difficult to maintain reliable predictions in operation. The primary sources of data drifts are changes in accelerator settings and changes in equipment performance which cannot be measured directly. To bridge this gap between ML development and long-term deployment in operational settings, we identify key areas within particle accelerators where continual learning can help mitigate drift-induced performance degradation. We will provide a practical guide on selecting the appropriate method given resource constraints and desired stability plasticity trade offs. As a concrete example, we will present a real-world use case for anomaly detection to predict errant beams at the Spallation Neutron Source accelerator, where continual learning has been employed to demonstrate stable performance on drifting data streams. We will present practical challenges, lessons learned, and the results from the deployed ML model.

Rajput, Kishansingh [Thomas Jefferson National Acc↗