SEARCH · Engineering Papers

Results for “DATA PROCESSING”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 307 records · Page 17

All sources NASA NTRS DOE OSTI

DOE OSTI2023

Integration of Experimental Hydroprocessing and FCC Data with Process (Aspen Plus) and Refinery Optimization (Aspen PIMS) Models

NREL's Economic, Sustainability, and Market Analysis (ESMA) team develops process models in Aspen Plus to support techno-economic analysis (TEA) and life-cycle assessment (LCA) of conversion pathways from renewable and circular resources to fuels and chemicals. These tools are being applied to assess probable operational constraints or bottlenecks associated with refinery co-processing and repurposing opportunities. The team also utilizes optimizable refinery models to quantify opportunities for biofuel and bioproduct pathways through integration with existing refining infrastructure. The goals of analysis are to model the transition of refineries to renewable and circular feedstocks and quantify the costs production and CO2 abatement through utilization of existing refineries. At the core of the ESMA team's work is the process data derived from conversion experiments on hydroprocessing and fluid catalytic cracking performed at NREL. The objective of this presentation is to highlight new analysis approaches and capabilities in pathway analysis and refinery optimization modeling with specific emphasis on integration of experimental data.

BIOMASS FUELS,ENERGY PLANNING, POLICY, AND ECONOMY↗

DOE OSTI2022

3DBFSVBF (3D BatFinder Smart Video BioFilter and Multi-class BatFinder Smart Video BioFilter) [SWR-22-88]

Bats are notoriously difficult to study, therefore, identifying specific behavioral trends and the precise environmental conditions at the time of collision requires a monitoring solution that can reliably collect relevant data. To date, thermal infrared video surveillance has been extensively applied to study bats and has proven to be a powerful yet cumbersome tool. Current analytical approaches are time consuming because data processing data has not been fully automated. In the past, steps have been taken to record avian and bat activity in conjunction with complicated image processing techniques that separate species from other moving objects within the field of view (i.e. clouds and portions of the wind turbine). Once the videos are collected, the post-processing does not allow real time monitoring and identification, leading to a delay in both studying the behavior of these species and determining the effectiveness of any impact reduction strategy being studied. Moreover, object identification capability is lacking, thus limiting the usefulness of video data. To resolve these issues, we are using open source 3D computer vision and machine learning techniques allowing for automatic detection of objects in real-time with the ability to correlate these objects with environmental variables and recording the flight paths of each object. The machine learning has been trained on 3D data and allows for automated real-time data collection, identification and tracking, thereby eliminating the need for long and tedious post-analysis processing of the videos. This machine learning model is an added feature to the previous BatFinder Smart Video BioFilter and increases the accuracy of that systems classification by increasing the accuracy of identifying bats (90% accuracy) and insects (69% accuracy) to a 97% accuracy. There are two object classifier machine learning models, Binary and multi-classification. Binary object classifier labeled BatFinder_Smart_Video_BioFilter.h5 distinguishes between biological objects and non-biological objects. The main goal of this object classifier is to ignore the turbine blades while detecting biological object flying withing the rotor swept area of the turbine. Non-biological objects have a probability of 0 and biological objects have a probability of 1. Multi-classifier labeled Multiclass_BatFinder_Smart_Video_BioFilter.h5 distinguishes between bats, birds, insects and non-biological.

Yarbrough, John↗

DOE OSTI2022

BFSVBF (BatFinder Smart Video BioFilter) [SWR-22-87] and Multi-class BatFinder Smart Video BioFilter Keras

Bats are notoriously difficult to study, therefore, identifying specific behavioral trends and the precise environmental conditions at the time of collision requires a monitoring solution that can reliably collect relevant data. To date, thermal infrared video surveillance has been extensively applied to study bats and has proven to be a powerful yet cumbersome tool. Current analytical approaches are time consuming because data processing data has not been fully automated. In the past, steps have been taken to record avian and bat activity in conjunction with complicated image processing techniques that separate species from other moving objects within the field of view (i.e. clouds and portions of the wind turbine). Once the videos are collected, the post-processing does not allow real time monitoring and identification, leading to a delay in both studying the behavior of these species and determining the effectiveness of any impact reduction strategy being studied. Moreover, object identification capability is lacking, thus limiting the usefulness of video data. To resolve these issues, we are using open source computer vision and machine learning techniques allowing for automatic detection of objects in real-time with the ability to correlate these objects with environmental variables and recording the flight paths of each object. The code has gone through five rounds of development with images used to train the models. This advancement allows for automated real-time data collection, identification, and tracking, thereby eliminating the need for long and tedious post-analysis processing of the videos. We will discuss the two open source and publicly available machine learning models developed within this scope of this work: 1) a binary model with a 97.5% accuracy in identifying the difference between an object and an empty scene, including wind turbine and clouds; and 2) a multiple classification model with the capability of identifying the type of object detected: bats (90% accuracy), birds (83% accuracy), insects (69% accuracy) and non-biological (99% accuracy).

Yarbrough, John↗

DOE OSTI2026

Digital Twin Technology for Safety, Security, and Training in Spent Nuclear Fuel Handling

The increasing complexity of spent nuclear fuel handling requires significant resources to ensure safety, security, and personnel training. As nuclear facilities have continued to advance in scale and technology, the integration of digital tools has become indispensable. Among these tools, digital twins, which are virtual models of physical systems, are emerging as invaluable tools for enhancing safety protocols, security measures, and training in the nuclear sector. These models were conceptualized in the Industry 4.0 revolution. Digital twins can process data from physical systems in real time (by using sensors), include multiple code packages to enable simulations of different physics applications, and even implement artificial intelligence or machine learning techniques for advanced data processing. Despite the advantages that digital twins provide, challenges still exist regarding their widespread implementation. For instance, data used by a digital twin must be accurate to ensure that the digital twin is accurately tuned. Furthermore, if insecure digital twins are targeted by hackers, then they can pose serious risks to the security and safety of nuclear facilities.

Digital twins↗

DOE OSTI2025

Hahn-Echo Assisted Deconvolution (HEAD)

This repository contains the Bruker pulse program for acquire 2D HEAD data in addition to the C++ program used for data processing. Data must be acquired with identical digital resolution in both dimensions. The resulting 2D spectrum is converted to a ASCII file using the Topspin 'totxt' command. This file can be handled by the HEAD_processing program.

Perras, Frederic [Ames National Laboratory; Ames N↗

DOE OSTI2025

LCLS Big Data Handling – How I Learned to Stop Worrying and Love the Data Deluge

Advanced data and computing systems are vital to Linac Coherent Light Source (LCLS) operations, data interpretation and overall scientific productivity. The transition to MHz-era operation marks a fundamental change in scale that requires new infrastructure and architectures to link LCLS to the required scale of computing needed for scientific interpretation. The LCLS-II Data System meets big data challenges by implementing configurable data reduction that can adapt to multiple science areas, real-time analysis frameworks to provide visualization and fast feedback, and the ability to transfer data to local and remote computational facilities for near real time analysis at the appropriate scale. Feature extracted information generated in the data analysis pipeline - at the edge, local compute, or remote High-Performance Computing (HPC) resources - can be used to steer experiments and inform user decisions during beam time. Artificial Intelligence and Machine Learning (AI/ML) techniques present new opportunities to rapidly analyse large datasets and direct experiments, but create new challenges in scaling, adaptability, complexity, and trustworthiness. We describe how the LCLS-II Data System architecture addresses its data-driven challenges in the areas of data acquisition, data processing, data management, and workflow orchestration to decrease the overall time-to-science and provide a vision for future developments.

artificial intelligence↗

DOE OSTI2026

Searches for CE ν NS and physics beyond the standard model using Skipper-CCDs at CONNIE

The Coherent Neutrino-Nucleus Interaction Experiment (CONNIE) aims to detect the coherent scattering ( CE ν NS ) of reactor antineutrinos off silicon nuclei using thick fully depleted high-resistivity silicon CCDs. Two Skipper-CCD sensors with subelectron readout noise capability were installed at the experiment next to the Angra-2 reactor in 2021, making CONNIE the first experiment to employ Skipper-CCDs for reactor neutrino detection. We report on the performance of the Skipper-CCDs, the new data processing, data quality, and event selection for CE ν NS interactions, which enable CONNIE to reach a record low detection threshold of 15 eV. The data were collected over 300 days in 2021–2022 and correspond to exposures of 14.9 g-days with the reactor-on and 3.5 g-days with the reactor-off. The difference between the reactor-on and off event rates shows no excess and yields upper limits for the neutrino interaction rates, comparable with previous CONNIE limits from standard CCDs and higher exposures. Searches for new neutrino interactions beyond the Standard Model improve the previous CONNIE limit on a simplified model with light vector mediators. A first dark matter (DM) search by diurnal modulation by CONNIE obtains the best limits on the DM-electron scattering cross section by a surface-level experiment. These promising results, obtained using a very small-mass sensor, illustrate the potential of Skipper-CCDs to probe rare neutrino interactions and motivate the plans to increase the detector mass in the near future.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗

DOE OSTI2024

Unsupervised multimodal fusion of in-process sensor data for advanced manufacturing process monitoring

Effective monitoring of manufacturing processes is crucial for maintaining product quality and operational efficiency. Modern manufacturing environments often generate vast amounts of complementary multimodal data, including visual imagery from various perspectives and resolutions, hyperspectral data, and machine health monitoring information such as actuator positions, accelerometer readings, and temperature measurements. However, fusing and interpreting this complex, high-dimensional data presents significant challenges, particularly when labeled datasets are unavailable or impractical to obtain. This paper presents a novel approach to multimodal sensor data fusion in manufacturing processes, inspired by the Contrastive Language-Image Pre-training (CLIP) model. We leverage contrastive learning techniques to correlate different data modalities without the need for labeled data, overcoming limitations of traditional supervised machine learning methods in manufacturing contexts. Our proposed method demonstrates the ability to handle and learn encoders for five distinct modalities: visual imagery, audio signals, laser position (x and y coordinates), and laser power measurements. By compressing these high-dimensional datasets into low-dimensional representational spaces, our approach facilitates downstream tasks such as process control, anomaly detection, and quality assurance. The unsupervised nature of our method makes it broadly applicable across various manufacturing domains, where large volumes of unlabeled sensor data are common. We evaluate the effectiveness of our approach through a series of experiments, demonstrating its potential to enhance process monitoring capabilities in advanced manufacturing systems. This research contributes to the field of smart manufacturing by providing a flexible, scalable framework for multimodal data fusion that can adapt to diverse manufacturing environments and sensor configurations. The proposed method paves the way for more robust, data-driven decision-making in complex manufacturing processes.

Contrastive Learning↗

DOE OSTI2021

FEbeam: Cavity and electron emission data conversion, processing, and analysis. A freeware toolkit for rf injectors

FEbeam is an all-in-one field emission data processing interface with the capability to analyze the field emission cathode performance in an rf injector by extracting the field enhancement factor, local field, and effective emission area from the Fowler–Nordheim equations. It also has the capability of processing beam imaging micrographs using its sister software, FEpic. Furthermore, the current version of FEbeam was designed for the Argonne Cathode Test-stand of the Argonne Wakefield Accelerator facility switch yard. With slight modifications, FEbeam could work for many rf field emission injectors. This software is open-source and can be found at GitHub.

43 PARTICLE ACCELERATORS↗

DOE OSTI2022

Efficient Data Query for Gaussian Process Compressed Data through Value Range Estimation [Slides]

When the resolution of the data increases, data reduction methods are applied to simulation output, including Gaussian process, neural representation and compression algorithms. Lots of data analysis/visualization techniques requires data query, but data query from reduced representation is still challenging. This report will provide examples and provide possible answers to why data query from reduced representation is still challenging.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

DOE OSTI2021

TNSL Overview

Thermal neutron scattering law (TNSL) data describe low-energy neutrons scattering off of bound materials, and can have a significant impact on modeling any system with slow neutrons, including nuclear reactors. Previous work to introduce TNSL data to neutron transport codes at LLNL focused on COG and TART [1], with the limitation that these codes require highly specialized data processing and formatting. We have recently increased efforts to process TNSL data with the central LLNL nuclear data processing code FUDGE, to be stored in the generalized nuclear database structure (GNDS) for use in any general transport code with the ability to read GNDS data. The first step in this effort is to verify that the TNSL processing with FUDGE yields results comparable to results obtained using the LANL nuclear data processing code NJOY. The next step is to verify the transport of thermal neutrons in Mercury (a Monte Carlo code) and Ardra (a deterministic code) against one another, as well as against the LANL Monte Carlo neutron transport code MCNP. This verification step has not been completed, due to a number of discrepancies between results obtained using differently processed data. There is ongoing effort to understand differences between FUDGE and NJOY. Finally, we map out our current capability to validate TNSL data against benchmark systems.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗

DOE OSTI2022

200-IA-1 Operable Unit Human Health Risk and Kd Screen

The purpose of this environmental calculation file (ECF) is to provide the following: Document the data processing and data reduction steps taken to prepare the 200-IA-1 Operable Unit (OU) data set that will be used to calculate the sample-specific screening level human health risk evaluation; Document the data processing and data reduction steps taken to prepare the 200-IA-1 OU data set that will be used to identify analytes that could potentially impact groundwater in the future beneath the 200-IA-1 OU representative waste sites; Document the assumptions, equations, and methodologies used to calculate the screening levels for human health cancer risks and noncancer hazards for each representative waste site assigned to the 200-IA-1 OU. Individual measured soil concentrations from 0 to 4.6 m (15 ft) below ground surface (bgs) (shallow vadose zone) are used to calculate the total excess lifetime cancer risk (ELCR) and hazard index (HI) for the outdoor worker scenario to determine if there is a basis for remedial action. Individual measured soil concentrations from the ground surface to the groundwater table are used for the distribution coefficient (Kd) screen to identify analytes that could potentially impact groundwater in the future beneath the 200-IA-1 OU representative waste sites. This ECF supports DOE/RL-2020-51, 200-IA-1 OU Focused Feasibility Study, under the Comprehensive Environmental Response, Compensation, and Liability Act of 1980 (CERCLA). A risk characterization based upon the evaluation of the health risk estimates developed in this ECF will be presented in the focused feasibility study report.

63 RADIATION, THERMAL, AND OTHER ENVIRON. POLLUTAN↗

DOE OSTI2024

CLAS12 remote data-stream processing using ERSAP framework

Implementing a physics data processing application is relatively straightforward with the use of current containerization technologies and container image runtime services, which are prevalent in most high-performance computing (HPC) environments. However, the process is complicated by the challenges associated with data provisioning and migration, impacting the ease of workflow migration and deployment. Transitioning from traditional file-based batch processing to data-stream processing workflows is suggested as a method to streamline these workflows. This transition not only simplifies file provisioning and migration but also significantly reduces the necessity for extensive disk space. Data-stream processing is particularly effective for real-time processing during data acquisition, thereby enhancing data quality assurance. This paper introduces the integration of the JLAB CLAS12 event reconstruction application within the ERSAP data-stream processing framework that facilitates the execution of streaming event reconstruction at a remote data center and enables the return streaming of reconstructed events to JLAB while circumventing the need for temporary data storage throughout the process.

Gyurjyan, Vardan↗

DOE OSTI2024

Automated pipeline processing X-ray diffraction data from dynamic compression experiments on the Extreme Conditions Beamline of PETRA III

Presented and discussed here is the implementation of a software solution that provides prompt X-ray diffraction data analysis during fast dynamic compression experiments conducted within the dynamic diamond anvil cell technique. It includes efficient data collection, streaming of data and metadata to a high-performance cluster (HPC), fast azimuthal data integration on the cluster, and tools for controlling the data processing steps and visualizing the data using the DIOPTAS software package. This data processing pipeline is invaluable for a great number of studies. The potential of the pipeline is illustrated with two examples of data collected on ammonia–water mixtures and multiphase mineral assemblies under high pressure. The pipeline is designed to be generic in nature and could be readily adapted to provide rapid feedback for many other X-ray diffraction techniques, e.g. large-volume press studies, in situ stress/strain studies, phase transformation studies, chemical reactions studied with high-resolution diffraction etc.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2022

Intrinsically stretchable neuromorphic devices for on-body processing of health data with artificial intelligence

For leveraging wearable technologies to advance precision medicine, personalized and learning-based analysis of continuously acquired health data is indispensable, for which neuromorphic computing could provide the most efficient implementation of artificial intelligence (AI) data processing. For realizing on-body neuromorphic computing, skin-like stretchability is required, but yet to be combined with the suite of desired neuromorphic metrics, including linear, symmetric weight update, and sufficient state retention, for achieving high computing efficiency. Here, we report an intrinsically stretchable neuromorphic device based on an electrochemical transistor, which provides a large number (>800) of states, linear/symmetric weight update, excellent switching endurance (>100 million), good state retention (>10 4 s), together with high stretchability of 100% strain. Further integration into a prototype array successfully realized the implementation of vector-matrix multiplication even at 100% strain. Finally, we demonstrate the feasibility of implementing AI-based classification of health signals (as exemplified by electrocardiograms) with a high accuracy that is minimally influenced by the stretched state of the neuromorphic hardware. Finally, this work breaks the ground for combining AI data analysis into skin-like wearable electronics for achieving human-integrated/mimetic intelligent systems.

60 APPLIED LIFE SCIENCES↗

DOE OSTI2022

Evolution of HEP Processing Frameworks

HEP data-processing software must support the disparate physics needs of many experiments. For both collider and neutrino environments, HEP experiments typically use data-processing frameworks to manage the computational complexities of their large-scale data processing needs. Data-processing frameworks are being faced with new challenges this decade. The computing landscape has changed from the past three decades of homogeneous single-core x86 batch jobs running on grid sites. Frameworks must now work on a heterogeneous mixture of different platforms: multi-core machines, different CPU architectures, and computational accelerators; and different computing sites: grid, cloud, and high-performance computing. We describe these challenges in more detail and how frameworks may confront them. Given their historic success, frameworks will continue to be critical software systems that enable HEP experiments to meet their computing needs. Frameworks have weathered computing revolutions in the past; they will do so again with support from the HEP community

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗

DOE OSTI2022

Internet of Things Data Characterization Process: Pattern of Life Behavioral Data Study

The HoneyBee™ TARDIS LDRD team completed a data scoping study that identified the initial processes and procedures to baseline the normal and expected behaviors during operability and interoperability of Internet of Things (IoT) device networks. This research is the initial step in developing a process (or methodology) to inform a much broader information framework incorporating machine learning to determine device pattern-of-life which enables the detection of abnormal IoT behaviors on an individual device, as well as in the context of a larger network.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2023

NSO Processing Scripts (NSO wind loading data set processing scripts) [SWR-23-86]

NREL conducted comprehensive field measurements of the atmospheric turbulent wind conditions and the resulting structural wind loads on parabolic troughs at the Nevada Solar One plant from October 2021 to June, 2023. The measurement set-up included meteorological masts and structural load sensors on four trough rows. Additionally, NREL commissioned a lidar scanning the horizontal plane over the trough field. The high-resolution data set, characterizing the complex flow field and resulting structural loads, is available at https://data.openei.org/submissions/5938. The processing routines published here were used to create the data from the instruments' raw data.

Egerer, Ulrike↗