Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Streaming data”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 379 records · Page 21

Occupancy schedules learning process through a data mining framework

Building occupancy is a paramount factor in building energy simulations. Specifically, lighting, plug loads, HVAC equipment utilization, fresh air requirements and internal heat gain or loss greatly depends on the level of occupancy within a building. Developing the appropriate methodologies to describe and reproduce the intricate network responsible for human-building interactions are needed. Extrapolation of patterns from big data streams is a powerful analysis technique which will allow for a better understanding of energy usage in buildings. A three-step data mining framework is applied to discover occupancy patterns in office spaces. First, a data set of 16 offices with 10 minute interval occupancy data, over a two year period is mined through a decision tree model which predicts the occupancy presence. Then a rule induction algorithm is used to learn a pruned set of rules on the results from the decision tree model. Finally, a cluster analysis is employed in order to obtain consistent patterns of occupancy schedules. Furthermore, the identified occupancy rules and schedules are representative as four archetypal working profiles that can be used as input to current building energy modeling programs, such as EnergyPlus or IDA-ICE, to investigate impact of occupant presence on design, operation and energy use in office buildings.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗

THE ATACAMA COSMOLOGY TELESCOPE: DATA CHARACTERIZATION AND MAPMAKING

We present a description of the data reduction and mapmaking pipeline used for the 2008 observing season of the Atacama Cosmology Telescope (ACT). The data presented here at 148 GHz represent 12% of the 90 TB collected by ACT from 2007 to 2010. In 2008 we observed for 136 days, producing a total of 1423 hr of data (11 TB for the 148 GHz band only), with a daily average of 10.5 hr of observation. From these, 1085 hr were devoted to an 850 deg{sup 2} stripe (11.2 hr by 9. Degree-Sign 1) centered on a declination of -52. Degree-Sign 7, while 175 hr were devoted to a 280 deg{sup 2} stripe (4.5 hr by 4. Degree-Sign 8) centered at the celestial equator. The remaining 163 hr correspond to calibration runs. We discuss sources of statistical and systematic noise, calibration, telescope pointing, and data selection. For the 148 GHz band, out of 1260 survey hours and 1024 detectors in the array, 816 hr and 593 effective detectors remain after data selection, yielding a 38% survey efficiency. The total sensitivity in 2008, determined from the noise level between 5 Hz and 20 Hz in the time-ordered data stream (TOD), is 32 {mu}K{radical}s in cosmic microwave background units. Atmospheric brightness fluctuations constitute the main contaminant in the data and dominate the detector noise covariance at low frequencies in the TOD. The maps were made by solving the least-squares problem using the Preconditioned Conjugate Gradient method, incorporating the details of the detector and noise correlations. Simulations, as well as cross-correlations with Wilkinson Microwave Anisotropy Probe sky maps on large angular scales, reveal that our maps are unbiased at multipoles l > 300. This paper accompanies the public release of the 148 GHz southern stripe maps from 2008. The techniques described here will be applied to future maps and data releases.

79 ASTRONOMY AND ASTROPHYSICS↗

Enhancements and Deployment of the TDAQ System for the Mu2e Experiment

The Real Time Processing Systems Division at Fermilab has deployed new features to the Off-The-Shelf Data Acquisition framework (otsdaq) for the Mu2e experiment. The Mu2e experiment will search for the coherent neutrino-less conversion of a muon into an electron in the field of an aluminum nucleus with a sensitivity improvement of 10,000 times over existing limits. Such a charged lepton flavor-violating reaction probes new physics at a scale unavailable at present or planned high-energy colliders. The Mu2e Trigger and Data Acquisition (TDAQ) system uses otsdaq as its online Data Acquisition System (DAQ) framework. otsdaq integrates the artdaq and art frameworks for event transfer, filtering, and processing. otsdaq is a web-based DAQ software suite focusing on flexibility and scalability and provides a multi-user interface accessible through a web browser. artdaq handles the entire data stream, which is read over the peripheral component interconnect express (PCIe) bus to a software filter algorithm that selects events combined with the data flux coming from a cosmic-ray veto (CRV) system. Detector front-ends are configured through the PCIe bus by customized otsdaq plugins. The otsdaq slow controls infrastructure has been further developed using the experimental physics and industrial control system (EPICS) open-source platform for monitoring, controlling, alarming, and archiving. The detector control system (DCS) for Mu2e has been integrated into otsdaq. The production TDAQ and DCS system has been deployed at the experimental hall and is being debugged and optimized for experiment operations. We report on the feature enhancements and deployment of otsdaq for Mu2e.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗

Recommendations for Updating Liquid Discharged Inventory and Transport Modeling Parameters for Cumulative Impacts Evaluation of Hexavalent Chromium in the 200 West Area

The purpose of this environmental calculation file (ECF) is to document information regarding hexavalent chromium (Cr(VI)) inventory discharged in 200 West Area at the Hanford Site and provide data to support predictive transport through the vadose zone and saturated zone for modeling efforts. This document provides a focused evaluation of historical waste stream data and studies to develop estimates of Cr(VI) inventory, discharge fractions, and transport parameters for the 200 West Area waste sites and tank farms during discharge events and for long-term contaminant releases.

12 MANAGEMENT OF RADIOACTIVE AND NON-RADIOACTIVE W↗

Materials Characterization, Prediction and Control Project: Summary Report on Data Analytics Framework

This report summarizes the activities performed under the data analytics Vertex in the Materials Characterization, Prediction and Control Project funded under laboratory directed research and development at Pacific Northwest National Laboratory. The data analytics Vertex developed models for associating global or local process parameters, microstructural features, and performance properties of friction-stir-processed 316L stainless steel plates. Statistical, machine learning, and deep learning models, as well as generative artificial intelligence approaches, were used to develop the associations between the process-structure-property data streams. These associations formed the basis for predicting global properties of parts manufactured under different process envelopes, providing a basis for predicting performance using data driven as well as physics-informed and physics-constrained approaches. Additionally, the associations were used to predict local process parameters and microstructural features of the product, predictive relationships that have the potential to form the basis of a control framework that could eventually modulate a friction-stir process to maintain product quality.

316L stainless steel↗

Bridging Cloud and Edge Computing at NREL Using CONNECT: Cloud Optimized Networking for Next-Gen Edge Computing Technologies [Slides]

CONNECT is an innovative on-premise hardware and software solution that integrates edge and cloud computing infrastructure at NREL. Built on the AWS Greengrass middleware and leveraging the MQTT protocol, CONNECT enables real-time data streaming from IoT devices and gateways to both cloud and local services, empowering researchers to rapidly capture, analyze, and act upon edge-generated data while leveraging cloud capabilities. The platform addresses research infrastructure challenges by providing a pre-approved platform which is already configured with the correct networking and cybersecurity baselines thus eliminating procurement delays and enabling on-demand availability. CONNECT's hybrid architecture efficiently manages burstable workloads, allowing research teams to dynamically scale computational capacity, handle peak data loads, and reduce operational bottlenecks. Advanced capabilities include built-in GPU support for executing machine learning models which enables low-latency inference at the edge from models trained in the cloud. This architecture supports real-time analytics and filtering, providing a mechanism to allow only transmitting and processing high-value data. Cloud-based configuration management permits engineers to manage on-premise systems remotely, optimizing operational efficiency. By bridging edge and cloud computing, CONNECT provides NREL researchers with a flexible, scalable platform that accelerates scientific discovery while maintaining robust security and performance standards.

97 MATHEMATICS AND COMPUTING↗

Contextual Active Online Model Selection with Expert Advice

How can we collect the most useful labels to learn a model selection policy, when presented with arbitrary heterogeneous data streams? In this paper, we formulate this task as a contextual active model selection problem, where at each round the learner receives an unlabeled data point along with a context. The goal is to output the best model for any given context without obtaining an excessive amount of labels. In particular, we focus on the task of selecting pre-trained classifiers, and propose a contextual active model selection algorithm (CAMS), which relies on a novel uncertainty sampling query criterion defined on a given policy class for adaptive model selection. In comparison to prior art, our algorithm does not assume a globally optimal model. We provide rigorous theoretical analysis for the regret and query complexity under both adversarial and stochastic settings. Our experiments on several benchmark classification datasets demonstrate the algorithm’s effectiveness in terms of both regret and query complexity. Notably, to achieve the same accuracy, CAMS incurs less than 10% of the label cost when compared to the best online model selection baselines on CIFAR10.

Liu, Xuefeng↗

A deep learning approach to anomaly detection in geological carbon sequestration sites using pressure measurements

Carbon capture and storage (CCS) has been extensively investigated as a potential engineering measure to reduce anthropogenic carbon emission to the atmosphere. Real-time monitoring of the safety and integrity of carbon storage reservoirs is a critical aspect of any commercial-scale CCS deployment. Pressure-based sensing is cost effective, suitable for real-time monitoring, and scalable to large monitoring networks. However, questions remain on how to best harness intelligent information from the high-frequency pressure monitoring sensors to support real-time decisions. Here this work presents a deep-learning-based framework for analyzing and detecting anomalies in pressure data streams by using a convolutional long short-term memory (ConvLSTM) neural network model, which allows for the fusion of both static and dynamic reservoir data. In ConvLSTM, the convolutional neural network (CNN) is used for spatial pattern mining and the LSTM is used for temporal pattern recognition. The performance of the ConvLSTM model for real-time anomaly detection is demonstrated using a set of pressure monitoring data collected from Cranfield, Mississippi, an active enhanced-oil-recovery field. The anomaly detection model is trained using bottom-hole pressure data acquired from the base experiment (without leak event) and then tested on pressure data collected during a series of controlled CO 2 release experiments (with artificially created leak events). Results show that the ConvLSTM neural network model successfully detected anomalies in the pressure time series obtained from the controlled release experiments. Inclusion of static information into the model further improves the robustness of ConvLSTM.

13 HYDRO ENERGY↗

The Phase-2 Upgrade of the CMS Data Acquisition

The High Luminosity LHC (HL-LHC) will start operating in 2027 after the third Long Shutdown (LS3), and is designed to provide an ultimate instantaneous luminosity of 7:5 × 10$^{34}$ cm$^{-2}$ s$^{-1}$, at the price of extreme pileup of up to 200 interactions per crossing. The number of overlapping interactions in HL-LHC collisions, their density, and the resulting intense radiation environment, warrant an almost complete upgrade of the CMS detector. The upgraded CMS detector will be read out by approximately fifty thousand highspeed front-end optical links at an unprecedented data rate of up to 80 Tb/s, for an average expected total event size of approximately 8 - 10 MB. Following the present established design, the CMS trigger and data acquisition system will continue to feature two trigger levels, with only one synchronous hardware-based Level-1 Trigger (L1), consisting of custom electronic boards and operating on dedicated data streams, and a second level, the High Level Trigger (HLT), using software algorithms running asynchronously on standard processors and making use of the full detector data to select events for offline storage and analysis. The upgraded CMS data acquisition system will collect data fragments for Level-1 accepted events from the detector back-end modules at a rate up to 750 kHz, aggregate fragments corresponding to individual Level- 1 accepts into events, and distribute them to the HLT processors where they will be filtered further. Events accepted by the HLT will be stored permanently at a rate of up to 7.5 kHz. This paper describes the baseline design of the DAQ and HLT systems for the Phase-2 of CMS.

Badaro, Gilbert↗

Towards a study protocol: A data-driven workflow to identify error sources in direct ink write mechatronics

Abstract Using Direct Ink Write (DIW) technology in a rapid and large-scale production requires reliable quality control for printed parts. Data streams generated during printing, such as print mechatronics, are massive and diverse which impedes extracting insights. In our study protocol approach, we developed a data-driven workflow to understand the behavior of sensor-measured X- and Y- axes positional errors with process parameters, such as print velocity and velocity control. We uncovered patterns showing that instantaneous changes in the velocity, when the build platform accelerates and decelerates, largely influence the positional errors, especially in the X- axis due to the hardware architecture. Since DIW systems share similar mechatronic inputs and outputs, our study protocol approach is broadly applicable and scalable across multiple systems. Graphical abstract

36 MATERIALS SCIENCE↗

A deterministic, gigabit serial timing, synchronization and data link for the RHIC LLRF

A critical capability of the new RHIC low level rf (LLRF) system is the ability to synchronize signals across multiple locations. The 'Update Link' provides this functionality. The 'Update Link' is a deterministic serial data link based on the Xilinx RocketIO protocol that is broadcast over fiber optic cable at 1 gigabit per second (Gbps). The link provides timing events and data packets as well as time stamp information for synchronizing diagnostic data from multiple sources. The new RHIC LLRF was designed to be a flexible, modular system. The system is constructed of numerous independent RF Controller chassis. To provide synchronization among all of these chassis, the Update Link system was designed. The Update Link system provides a low latency, deterministic data path to broadcast information to all receivers in the system. The Update Link system is based on a central hub, the Update Link Master (ULM), which generates the data stream that is distributed via fiber optic links. Downstream chassis have non-deterministic connections back to the ULM that allow any chassis to provide data that is broadcast globally.

43 PARTICLE ACCELERATORS↗

Apparatus and amendment of wind turbine blade impact detection and analysis

A multisensory system provides both temporal and spatial coverage capacities for auto-detection of bird collision events. The system includes an apparatus having a first circuitry to capture and store a series of images or video of a blade of a wind turbine; and a memory to store the images from the first circuitry. The apparatus also has one or more sensors to continuously sense vibration of the blade or for acoustic recordings; and a second circuitry to analyze the sensor data stream and/or the series of images or video to identify a cause of the vibration and to trigger the camera(s). A communication interface transmits data from the second circuitry to another device, wherein the second circuitry applies artificial intelligence or machine learning to control sensitivity of the one or more sensors.

Johnston, Matthew↗

Real-time Anomaly Detection for Liquid Argon Time Projection Chambers

We present a real-time anomaly detection framework for liquid argon time projection chambers (LArTPCs), targeting applications in particle physics experiments such as the Short Baseline Near Detector (SBND) or the future Deep Underground Neutrino Experiment (DUNE). These experiments employ detectors that generate and stream high-resolution but sparse images of neutrino and other particle interactions. Our approach utilizes anomaly detection with autoencoders, compressed through knowledge distillation (KD), to enable the detection of anomalous signals in the data through efficient inference on resource-constrained hardware. The framework is targeted for deployment on computing platforms equipped with field-programmable gate arrays (FPGAs), GPUs, or CPUs, allowing low-latency selection of relevant activity directly from the raw detector data stream. We demonstrate that our approach is suitable for the detection and localization of anomalously "high-multiplicity" activity, and outline promising applications for LArTPC online data filtering and triggering.

FOS: Physical sciences↗

Machine Learning Enabled Sensor Fusion for In-Situ Defect Detection in Laser Powder Bed Fusion

Laser Powder Bed Fusion (L-PBF) Additive Manufacturing (AM) is among the metal 3D printing technologies most broadly adopted by the manufacturing industry. The current industry qualification paradigm for critical-application L-PBF parts relies heavily on expensive non-destructive inspection techniques such as X-Ray Computed Tomography (XCT), which significantly limits the use-cases of L-PBF. In situ monitoring of the process promises a less expensive alternative to ex situ testing, but existing sensor technologies and data analysis techniques struggle to detect sub-surface flaws (e.g., porosity and cracking) on production-scale L-PBF printers. RTX Technologies Research Center (RTRC) has licensed ORNL’s Peregrine software package – a printer- and camera-agnostic data analytics tool designed specifically for detecting process anomalies using in situ data collected during powder bed printing. The goal of this project was to feed temporally rich, multi-modal sensor data, including visible light, integrated near infrared (NIR), and spatially mapped co-axial melt pool thermal emission data into Peregrine to enable detection of subsurface flaws. XCT data was used as ground truth training data to allow Peregrine’s deep learning algorithms to recognize anomalies in these complex data streams in both test artifacts and industrially relevant geometries. Completion of this program has seen the successful implementation of multi-modal, multi-layer sensor data footprints for training of machine learning models in Peregrine. Flaws detected in XCT data have been successfully detected directly from this in situ data footprint, and initial analyses of the in situ probability-of-detection has been conducted, showing performance levels commensurate with traditional non-destructive evaluation (NDE) methods. The in situ monitoring methodology was then applied to an industrially relevant component that was using post-build NDE, highlighting the utility of the proposed method for hard-to-inspect AM components. As a direct result of this program, two journal manuscripts [1], [2] have been published in Additive Manufacturing, with additional manuscripts planned following program completion.

36 MATERIALS SCIENCE↗

Measurement of charged-current $\nu_{\mu}$ and $\nu_e$ interactions with wire-cell in MicroBooNE towards a search for low-energy $\nu_e$ excess

This technote summarizes the existing work in searching for $\nu_e$ low-energy excess (eLEE) in MicroBooNE Booster Neutrino Beam (BNB) data stream based on the Wire-Cell event reconstruction paradigm. The charged-current $\nu_{\mu}$ and $\nu_e$ events are selected from the 5.3e19 POT data from the BNB beam and 2.06e20 POT data from Neutrinos at the Main Injector (NuMI) beam. The charged-current $\nu_e$ selection results from the BNB data that are sensitive to the eLEE search are not included. Various comparisons between data and Monte Carlo predictions are performed to validate the overall model and demonstrate the power of the analysis techniques.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗

ESnet/JLab FPGA Accelerated Transport

To increase the science rate for high data rates/volumes, Thomas Jefferson National Accelerator Facility (JLab) has partnered with Energy Sciences Network (ESnet) to define an edge to data center traffic shaping / steering transport capability featuring data event aware network shaping and forwarding. The keystone of this ESnet+JLab FPGA Accelerated Transport (EJFAT) is the joint development of an AI/ML directed dynamic compute work Load Balancer (LB) of UDP streamed data. The LB is a suite consisting of a Field Programmable Gate Array (FPGA) executing the dynamically configurable, low fixed latency LB data plane featuring real-time packet redirection and high throughput, and a control plane running on the FPGA host computer that monitors network and compute farm telemetry in order to make dynamic AI/ML guided decisions for destination compute host redirection/load balancing and destination resource provisioning. The LB provides for three-tier horizontal scaling across LB suites, core compute hosts, and CPUs within a host. The LB effectively provides seamless integration of edge/core computing to support direct experimental data processing for immediate use by JLab science programs and others such as the EIC as well as data centers of the future requiring high throughput and low latency for both hot and cooled data for both running experiment data acquisition systems and data center use cases.

97 MATHEMATICS AND COMPUTING↗

An Event Detection Framework for Virtual Observation System: Anomaly Identification for an ACME Land Simulation

Based on previous work on in-situ data transfer infrastructure and compiler-based software analysis, we have designed a virtual observation system for real time computer simulations. This paper presents an event detection framework for a virtual observation system. By using signal processing and detection approaches to the memory-based data streams, this framework can be reconfigured to capture high-frequency events and low-frequency events. These approaches used in the framework can dramatically reduce the data transfer needed for in-situ data analysis (between distributed computing nodes or between the CPU/GPU nodes). In the paper, we also use a terrestrial ecosystem system simulation within the Earth System Model to demonstrate the practical values of this effort.

Yao, Cindy↗

Cluster-to-cluster data transfer with data compression over wide-area networks

The recent emergence of ultra high-speed networks up to 100 Gb/s has posed numerous challenges and has led to many investigations on efficient protocols to saturate 100 Gb/s links. However, end-to-end data transfers involve many components, not only protocols, affecting overall transfer performance. These components include disk I/O subsystem, additional computation associated with data streams, and network adapters. For example, achievable bandwidth by TCP may not be implementable if disk I/O or CPU becomes a bottleneck in end-to-end data transfer. In this paper, we first model all the system components involved in end-to-end data transfer as a graph. We then formulate the problem whose goal is to achieve maximum data transfer throughput using parallel data flows. We also propose a variable data flow GridFFP XIO stack to improve data transfer with data compression. Our contributions lie in how to optimize data transfers considering all the system components involved rather than in accurately modeling all the system components involved. Our proposed formulations and solutions are evaluated through experiments on the ESnet 100G testbed and a wide-area cluster-to-cluster testbed. The experimental results on the ESnet 100G testbed show that our approach is several times faster than Globus Online-8 x faster for datasets with many 10 MB files and 3-4 x faster for other datasets of larger size files. The experimental results on the cluster-to-cluster testbed show that our variable data flow approach is up to 4 x faster than a normal cluster data transfer.

Disk-to-Disk Transfer↗