Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Streaming data”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 865 records · Page 48

Getting more from heterogeneous HIV-1 surveillance data in a high immigration country: estimation of incidence and undiagnosed population size using multiple biomarkers

Most HIV infections originate from individuals who are undiagnosed and unaware of their infection. Estimation of this quantity from surveillance data is hard because there is incomplete knowledge about (i) the time between infection and diagnosis (TI) for the general population, and (ii) the time between immigration and diagnosis for foreign-born persons. We developed a new statistical method for estimating the incidence of HIV-1 and the number of undiagnosed people living with HIV (PLHIV), based on dynamic modelling of heterogeneous HIV-1 surveillance data. The methods consist of a Bayesian non-linear mixed effects model using multiple biomarkers to estimate TI of HIV-1-positive individuals, and a novel incidence estimator which distinguishes between endogenous and exogenous infections by modelling explicitly the probability that a foreign-born person was infected either before or after immigration. The incidence estimator allows for direct calculation of the number of undiagnosed persons. The new methodology is illustrated combining heterogeneous surveillance data from Sweden between 2003 and 2015. A leave-one-out cross-validation study showed that the multiple-biomarker model was more accurate than single biomarkers (mean absolute error 1.01 vs ≥1.95). We estimate that 816 [95% credible interval (CI) 775-865] PLHIV were undiagnosed in 2015, representing a proportion of 10.8% (95% CI 10.3-11.4%) of all PLHIV. Here, the proposed methodology will enhance the utility of standard surveillance data streams and will be useful to monitor progress towards and compliance with the 90–90-90 UNAIDS target.

60 APPLIED LIFE SCIENCES↗

Development of a Novel AOP for Cyp2F2-Mediated Lung Cancer in Mice

Abstract Traditional methods for carcinogenicity testing rely heavily on the rodent bioassay as the standard for identification of tumorigenic risk. As such, identification of species-specific outcomes and/or metabolism are a frequent argument for regulatory exemption. One example is the association of tumor formation in the mouse lung after exposure to Cyp2F2 ligands. The adverse outcome pathway (AOP) framework offers a theoretical platform to address issues of species specificity that is consistent, transparent, and capable of integrating data from new approach methodologies as well as traditional data streams. A central premise of the AOP concept is that pathway progression from the molecular initiating event (MIE) implies a definable “response-response” (R-R) relationship between each key event (KE) that drives the pathway towards a specific adverse outcome (AO). This article describes an AOP for lung cancer in the mouse from an MIE of Cyp2F2-specific reactive metabolite formation, advancing through KE that include protein and/or nucleic acid adducts, diminished Club Cell 10 kDa (CC10) protein expression, hyperplasia of CC10 deficient Club cells, and culminating in the AO of mixed-cell tumor formation in the distal airways. This tumor formation is independent of route of exposure and our AOP construct is based on overlapping mechanistic events for naphthalene, styrene, ethyl benzene, isoniazid, and fluensulfone in the mouse. This AOP is intended to accelerate the explication of an apparent mouse-specific outcome and serve as a starting point for a quantitative analysis of mouse-human differences in susceptibility to the tumorigenic effects of Cyp2F2 ligands.

Hill, III, Thomas↗

Modeling the differential rate for signal interactions in coincidence with noise fluctuations or large rate backgrounds

The characteristic energy of a relic dark matter interaction with a detector scales strongly with the putative dark matter mass. Consequently, experimental search sensitivity at the lightest masses will always come from interactions whose size is similar to noise fluctuations and low energy backgrounds in the detector. In this paper, we correctly calculate the net change in measured differential event rate due to low rate signal interactions that overlap in time with noise and backgrounds, accounting for both periods of time when the signal is coincident with noise/backgrounds and for the decreased amount of time in which only noise/backgrounds occur and we show that the introduction of random simulated signal events into the continuous raw data stream (a form of “salting”) provides a correct and practical implementation to estimate this net linear signal sensitivity. Unfortunately this does not apply to the situation with non negligible pileups as we show through explicit examples. Consequently, this exclusion technique should be complemented by less sensitive techniques that can separately exclude large rate, high pileup signal parameter space. An analysis threshold above the Gaussian-like noise component of the measured differential rate spectrum should also likely produce conservative limits. Though previous light mass dark matter searches did not correctly account for decreased background only live time effects in their signal sensitivity, conservative analysis choices likely protected their published limits from being nonconservative.

Li, Xinran↗

Application of the gradient method to Hartree-Fock-Bogoliubov theory

A computer code is presented for solving the equations of the Hartree-Fock-Bogoliubov (HFB) theory by the gradient method, motivated by the need for efficient and robust codes to calculate the configurations required by extensions of the HFB theory, such as the generator coordinate method. The code is organized with a separation between the parts that are specific to the details of the Hamiltonian and the parts that are generic to the gradient method. This permits total flexibility in choosing the symmetries to be imposed on the HFB solutions. The code solves for both even and odd particle-number ground states, with the choice determined by the input data stream. Application is made to the nuclei in the sd shell using the universal sd-shell interaction B (USDB) shell-model Hamiltonian.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗

Automated pipeline processing X-ray diffraction data from dynamic compression experiments on the Extreme Conditions Beamline of PETRA III

Presented and discussed here is the implementation of a software solution that provides prompt X-ray diffraction data analysis during fast dynamic compression experiments conducted within the dynamic diamond anvil cell technique. It includes efficient data collection, streaming of data and metadata to a high-performance cluster (HPC), fast azimuthal data integration on the cluster, and tools for controlling the data processing steps and visualizing the data using the DIOPTAS software package. This data processing pipeline is invaluable for a great number of studies. The potential of the pipeline is illustrated with two examples of data collected on ammonia–water mixtures and multiphase mineral assemblies under high pressure. The pipeline is designed to be generic in nature and could be readily adapted to provide rapid feedback for many other X-ray diffraction techniques, e.g. large-volume press studies, in situ stress/strain studies, phase transformation studies, chemical reactions studied with high-resolution diffraction etc.

97 MATHEMATICS AND COMPUTING↗

Architecture for Quantum-in-the Loop Real-Time Simulations for Designing Resilient Smart Grids

With the power grid growing more complex every day with the inclusion of new sensors and regulatory approvals that enable end-users and small local developers to participate in the grid, it is becoming challenging for conventional smart grid simulation, emulation, and testing technologies to keep up. In this work, we propose that quantum-encoded real-time simulations can be helpful under the new paradigm and operational circumstances to solve optimization problems for power grids. By leveraging the principles of quantum mechanics, the proposed quantum-in-loop (QIL) framework will enable better and faster optimization solutions based on real-world, real-time data streams, facilitating the real-time planning and operations of electrical grids that rely on millions of distributed sensors and controllers. Furthermore, QIL will allow researchers and engineers to assist utilities in designing, developing, and de-risking algorithms to optimize power grid operation and resilience by considering inputs from millions of grid-connected devices. QIL framework is being developed to have a self-limiting triage mechanism, which will help engineers and practitioners identify fundamental physical limits on quantum processors, revealing what quantum algorithms can and cannot do in utility-specific use cases and must continue to count on classical high-performance computing infrastructure.

digital real-time simulation↗

Automation for Grid Interconnected Laboratory Emulation

As computational capabilities improve, digital twins are becoming vital for evaluating equipment realistically in laboratories. This paper outlines a digital twin architecture for the power grid, employing electromagnetic transient (EMT) simulation alongside real-time simulation of power hardware and hierarchical control systems. EMT simulation occurs on a high-performance computing server for scalability. Additionally, the paper describes a workflow and real-time data streaming software facilitating connectivity among EMT simulation, hierarchical control systems, and power hardware. This software enables automated equipment connectivity in the laboratory for realistic evaluations, aiding in identifying necessary upgrades for both equipment control systems and the power grid.

Marthi, Phani Ratna Vanamali [ORNL] (ORCID:0000000↗

Scalable Performance Awareness for In Situ Scientific Applications

Part of the promise of exascale computing and the next generation of scientific simulation codes is the ability to bring together time and spatial scales that have traditionally been treated separately. This enables creating complex coupled simulations and in situ analysis pipelines, encompassing such things as "whole device" fusion models or the simulation of cities from sewers to rooftops. Unfortunately, the HPC analysis tools that have been built up over the preceding decades are ill suited to the debugging and performance analysis of such computational ensembles. In this paper, we present a new vision for performance measurement and understanding of HPC codes, MonitoringAnalytics (MONA). MONA is designed to be a flexible, high performance monitoring infrastructure that can perform monitoring analysis in place or in transit by embedding analytics and characterization directly into the data stream, without relying upon delivering all monitoring information to a central database for post-processing. It addresses the trade-offs between the prohibitively expensive capture of all performance characteristics and not capturing enough to detect the features of interest. We demonstrate several uses of MONA; capturing and indexing multi-executable performance profiles to enable later processing, extraction of performance primitives to enable the generation of customizable benchmarks and performance skeletons, and extracting communication and application behaviors to enable better control and placement for the current and future runs of the science ensemble. Relevant performance information based on a system for MONA built from ADIOS and SOSflow technologies is provided for DOE science applications and leadership machines.

Wolf, Matthew D.↗

A lightweight network anomaly detection technique

While the network anomaly detection is essential in network operations and management, it becomes further challenging to perform the first line of detection against the exponentially increasing volume of network traffic. In this paper, we develop a technique for the first line of online anomaly detection with two important considerations: (i) availability of traffic attributes during the monitoring time, and (ii) computational scalability for streaming data. The presented learning technique is lightweight and highly scalable with the beauty of approximation based on the grid partitioning of the given dimensional space. With the public traffic traces of KDD Cup 1999 and NSL-KDD, we show that our technique yields 98.5% and 83% of detection accuracy, respectively, only with a couple of readily available traffic attributes that can be obtained without the help of post-processing. Finally, the results are at least comparable with the classical learning methods including decision tree and random forest, with approximately two orders of magnitude faster learning performance.

97 MATHEMATICS AND COMPUTING↗

Representative Day Selection Using Statistical Bootstrapping for Accelerating Annual Distribution Simulations

Capturing technical and economic impacts of solar photovoltaics (PV) and other distributed energy resources (DERs) on electric distribution systems can require high-time resolution (e.g. 1 minute), long-duration (e.g. 1 year) simulations. However, such simulations can be computationally prohibitive, particularly when including complex control schemes in quasi-steady-state time series (QSTS) simulation. Various approaches have been used in the literature to down select representative time segments (e.g. days), but typically these are best suited for lower time resolutions or consider only a single data stream (e.g. PV production) for selection. We present a statistical approach that combines stratified sampling and bootstrapping to select representative days while also providing a simple method to reassemble annual results. We describe the approach in the context of a recent study with a utility partner. This approach enables much faster QSTS analysis by simulating only a subset of days, while maintaining accurate annual estimates.

14 SOLAR ENERGY↗

PMU-data-driven Event Classification in Power Transmission Grids

This paper presents an event classification in transmission grids. The convolutional neural network (CNN)-based classifier is proposed to capture the temporal similarity of time-synchronized data stream from phasor measurement units (PMUs). The proposed CNN is trained using Bayesian optimization to search for the best hyperparameters. The effectiveness of the proposed event classification is validated through the real-world dataset from the U.S. transmission grids. This dataset includes line outage, transformer outage, frequency, and oscillation events. The validation process also includes different PMU outputs, such as voltage magnitude, phase angle, current magnitude, frequency, and rate of change of frequency (ROCOF). The results show that ROCOF gives the best classification performance compared to other PMU outputs. In addition, it is shown that the classifier trained with a larger dataset has higher accuracy. Moreover, the superiority of the proposed method is validated through comparison with other state-of-the-art classification methods.

Niazazari, Iman↗

Free-Space Optical Channel Turbulence Prediction: A Machine Learning Approach

Channel turbulence is a formidable obstacle for freespace optical (FSO) communication. Anticipation of turbulence levels is highly important for mitigating disruptions but has not been demonstrated without dedicated, auxiliary hardware. We show that machine learning (ML) can be applied to raw FSO data streams to rapidly predict channel turbulence levels with no additional sensing hardware. FSO was conducted through a controlled channel in the lab under six distinct turbulence levels, and the efficacy of using ML to classify turbulence levels was examined. ML-based turbulence level classification was found to be > 98% accurate with multiple ML training parameters. Classification effectiveness was found to depend on the timescale of changes between turbulence levels but converges when turbulence stabilizes over about a one minute timescale.

Zobaer Islam, Md [School of Electrical and Compute↗

Simulating the Autonomous Future: A Look at Virtual Vehicle Environments and How to Validate Simulation Using Public Data Sets

The rapid evolution of autonomous vehicles (AVs) has exposed the need for fast-paced development and testing processes of a variety of perception, planning, and control algorithms. To expedite development, the AV industry and researchers leverage virtual vehicle environments to simulate a range of test scenarios that may otherwise be costly or difficult to conduct on a real test track. However, the various virtual environments may have different results depending on the fidelity of various simulation features, such as vehicle dynamics, sensor simulation, and environment recreation. Herein, this tutorial article examines a proposed framework for constructing, parameterizing, and validating a virtual vehicle environment using an existing AV data set. First, an overview of several open source and commercially available simulation tools, including their associated workflows, for scene and scenario creation is presented. Next, various open AV data sets are examined to inform the data set selection for the validation framework. Then, an example workflow of recreating a real-world scene from the selected data set in a simulation tool with various emulated sensors parameterized to match the data set is demonstrated. Finally, an example AV-perception algorithm is subjected to data streams from virtual and real-world environments and suggested metrics for analyzing the results are discussed.

42 ENGINEERING↗

Event Cause Analysis in Distribution Networks using Synchro Waveform Measurements

This paper presents a machine learning method for event cause analysis to enhance situational awareness in distribution networks. The data streams are captured using time-synchronized high sampling rates synchro waveform measurement units (SWMU). The proposed method is formulated based on a machine learning method, the convolutional neural network (CNN). This method is capable of capturing the spatiotemporal feature of the measurements effectively and perform the event cause analysis. Several events are considered in this paper to encompass a range of possible events in real distribution networks, including capacitor bank switching, transformer energization, fault, and high impedance fault (HIF). The dataset for our study is generated using the real time digital simulator (RTDS) to simulate real-world events. The event cause analysis is performed using only one cycle of the voltage waveforms after the event is detected. The simulation results show the effectiveness of the proposed machine learning-based method compared to the state-of-the-art classifiers.

Niazazari, Iman↗

Real-time Event Detection Using Rank Signatures of Real-world PMU Data

Timely detection of power system events is a crucial task, which can facilitate the implementation of remedial actions to improve reliability, resiliency, and security of the system. Meanwhile, the widespread deployment of phasor measurement units (PMUs) makes it possible to develop data-driven event detection techniques. However, relying purely on data without incorporating domain knowledge for the event detection task in power systems poses substantial security and stability risks due to issues associated with data misinterpretation and model accuracy. In this regard, we propose a real-time event detection method using real-world PMU data by incorporating domain knowledge to adequately capture the event signatures. Specifically, we track the change in rank signatures of PMU data to accurately localize the events. To optimize the detection process, we incorporate an offline Bayesian optimization algorithm to tune the parameters by efficiently searching for the best values. The experiments using the real-world PMU dataset from a U.S. interconnection show that the proposed event detection approach can efficiently detect the events from PMU data streams with high accuracy.

Ghasemkhani, Amir↗

Enabling DER visibility using a distributed dissemination network

The electrical grid is currently undergoing a series of rapid transformational changes that have resulted in the introduction of new actors and operational schemes that have fragmented the data and control planes. To help address the issue, this paper describes the implementation of a sensor-oriented, distributed data dissemination network that seeks to eliminate data silos. The implementation is based on the DGSS architecture previously described in [1]. The developed product seeks to facilitate the seamless integration of multi-operator, multi-origin, multi-domain sensor data by using a distributed systems approach. The proposed solution decouples the sensor’s data streams from the application-specific infrastructure and migrates them into a software-defined databus that can be configured to suit the end application’s demands. To further validate DGSS capabilities, a DER oriented use case has been developed.

Sensor Dissemination Networks, Enhanced DER visibi↗

DeepWelding: a Deep Learning Enhanced Approach to GTAW Using Multi-source Sensing Images

Deep learning has great potential to reshape manufacturing industries. In this paper, we present DeepWelding, a novel framework that applies deep learning techniques to improve gas tungsten arc welding (GTAW) process monitoring and penetration detection using multi-source sensing images. The framework is capable of analyzing multiple types of optical sensing images synchronously and consists of three deep learning enhanced consecutive phases: image preprocessing, image selection, and weld penetration classification. Specifically, we adopted generative adversarial networks (pix2pix) for image denoising and classic convolutional neural networks (AlexNet) for image selection. Both pix2pix and AlexNet delivered satisfactory performance. However, five individual neural networks with heterogeneous architectures demonstrated inconsistent generalization capabilities in the classification phase when holding out multi-source images generated with specific experimental settings. Therefore, two ensemble methods combining multiple neural networks were designed to improve the model performance on unseen data collected from different experimental settings. In conclusion, we also found that the quality of model prediction was heavily influenced by the data stream collection environment. We think these findings are beneficial for the broad intelligent welding community.

42 ENGINEERING↗

Real-Time Ethernet Interface for NSTX-U’s Thomson Scattering Diagnostic (2023)

Here, the multipoint Thomson scattering (MPTS) diagnostic system at the National Spherical Torus Experiment Upgrade (NSTX-U) facility is undergoing an upgrade to operate in real-time and interface with the plasma control system (PCS) for NSTX-U. Previous prototyping efforts have shown that spectral analysis and rapid calculations of electron temperature and density are possible on a real-time Linux machine when using up to a 100-Hz laser pulse repetition rate. A remaining challenge was transferring the real-time data to NSTX-U’s PCS, which utilizes the front panel data port (FPDP) protocol. The original proposed method was to convert the real-time data into analog values, but a new solution was developed to keep the output format digital by using an Ethernet controller with a field-programmable gate array (FPGA). This article focuses on a new input module that has been developed to accept incoming user datagram protocol (UDP) packets sent over Ethernet, convert into FPDP format, and integrate into the existing data stream under NSTX-U’s real-time framework.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗