Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “data processing automation”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 361 records · Page 20
All sourcesNASA NTRSDOE OSTI
DOE OSTI2024

Overview of the distributed image processing infrastructure to produce the Legacy Survey of Space and Time

The Vera C. Rubin Observatory is preparing to execute the most ambitious astronomical survey ever attempted, the Legacy Survey of Space and Time (LSST). Currently the final phase of construction is under way in the Chilean Andes, with the Observatory’s ten-year science mission scheduled to begin in 2025. Rubin’s 8.4-meter telescope will nightly scan the southern hemisphere collecting imagery in the wavelength range 320–1050 nm covering the entire observable sky every 4 nights using a 3.2 gigapixel camera, the largest imaging device ever built for astronomy. Automated detection and classification of celestial objects will be performed by sophisticated algorithms on high-resolution images to progressively produce an astronomical catalog eventually composed of 20 billion galaxies and 17 billion stars and their associated physical properties. In this article we present an overview of the system currently being constructed to perform data distribution as well as the annual campaigns which reprocess the entire image dataset collected since the beginning of the survey. These processing campaigns will utilize computing and storage resources provided by three Rubin data facilities (one in the US and two in Europe). Each year a Data Release will be produced and disseminated to science collaborations for use in studies comprising four main science pillars: probing dark matter and dark energy, taking inventory of solar system objects, exploring the transient optical sky and mapping the Milky Way. Also presented is the method by which we leverage some of the common tools and best practices used for management of large-scale distributed data processing projects in the high energy physics and astronomy communities. We also demonstrate how these tools and practices are utilized within the Rubin project in order to overcome the specific challenges faced by the Observatory.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2022

Power System Waveform Classification Using Time-Frequency and CNN

Many modern reclosers and circuit breakers have microprocessor relays that record waveforms of system events. In some cases, utilities may record a half-a-dozen event captures for every event. This is thousands of events per year. The industry needs faster, more automated, more conclusive, and easy-to-use systems that can process massive amounts of event recordings without extensive input/support from power system engineers. To address the need for a commercially viable solution that can classify waveform data, energies were directed to develop a universal neural network (NN) structure (deep learning algorithm) that works for a wide variety of system event types. The structure that showed the most promise was one that included the use of spectrograms. The technique has shown positive results in audio engineering, particularly with respect to speech recognition. A waveform signature could be treated as a spoken word like audio waveforms for specific things such as “YES” or “UP”. No two people produce the exact same waveform when speaking each of these words, but audio processing algorithms based on spectrograms and convolutional neural networks (CNN) can still distinguish the word regardless of the speaker. No two circuits produce the exact same waveform for a given event, but the NN can be trained to classify the event type regardless of the circuit or location on the circuit. A Power System Neural Network (PSNN) has been developed to use a CNN to classify events within waveform data for power systems. The waveform is converted to an array of values by way of spectrograms and interpreted as an image. This image is passed into the CNN. The test results on independent simulated test and validation datasets show greater than 99% accuracy. While the results thus far are based on simulated data, the performance of the PSNN is very promising and should work for a wide variety of power system conditions of interest. Ultimately, much of the custom code and tools used today and much of the manual effort expended today may be automated using this PSNN.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2024

Towards Next-Generation Urban Decision Support Systems through AI-Powered Construction of Scientific Ontology Using Large Language Models—A Case in Optimizing Intermodal Freight Transportation

The incorporation of Artificial Intelligence (AI) models into various optimization systems is on the rise. However, addressing complex urban and environmental management challenges often demands deep expertise in domain science and informatics. This expertise is essential for deriving data and simulation-driven insights that support informed decision-making. In this context, we investigate the potential of leveraging the pre-trained Large Language Models (LLMs) to create knowledge representations for supporting operations research. By adopting ChatGPT-4 API as the reasoning core, we outline an applied workflow that encompasses natural language processing, Methontology-based prompt tuning, and Generative Pre-trained Transformer (GPT), to automate the construction of scenario-based ontologies using existing research articles and technical manuals of urban datasets and simulations. From these ontologies, knowledge graphs can be derived using widely adopted formats and protocols, guiding various tasks towards data-informed decision support. The performance of our methodology is evaluated through a comparative analysis that contrasts our AI-generated ontology with the widely recognized pizza ontology, commonly used in tutorials for popular ontology software. We conclude with a real-world case study on optimizing the complex system of multi-modal freight transportation. Our approach advances urban decision support systems by enhancing data and metadata modeling, improving data integration and simulation coupling, and guiding the development of decision support strategies and essential software components.

96 KNOWLEDGE MANAGEMENT AND PRESERVATION↗
DOE OSTI2020

A Data Processing Workflow for Fixed-wing Drone Based Radiation Mapping in the Chornobyl Exclusion Zone (CEZ) - 20119

April 2020 marks the 34. anniversary of the high-profile radiological release from the Chornobyl Nuclear Power Plant (ChNPP). The release of radioactive material from reactor number four began on the 26 April 1986 and continued over a period of about 10 days, releasing approximately 1700 PBq of radioactive material (including 85 PBq of 137-Cs) into the environment. To this day, the accident remains the most significant release of radioactive material since civil nuclear power generation began. In the years since the accident, automated and remote radiation monitoring technologies have advanced significantly in their capabilities. One such example of this is the use of unmanned aerial vehicles (UAVs) in radiation mapping investigations. In April 2019, a team of scientists from the University of Bristol showcased a novel radiation mapping system within the CEZ, specifically aiming to map radiation over a large portion of the area immediately surrounding the ChNPP. Over six days of data collection, the system flew a total distance of 583.8 km, covering an area of 14.6 sq.km with an exceptional spatial resolution (sub 20 m/pixel). The work presented herein outlines and explains the data processing procedure to convert the raw data into 137-Cs activity (kBq/sq.m) and cesium-equivalent dose-rate (CED) at 1 m above ground level (μSv/hr). A demonstration of the validity of the method is demonstrated through the successful reduction of the raw data into a single linear relationship between the measured {sup 137}Cs net peak intensity and the {sup 137}Cs activity. (authors)

12 MANAGEMENT OF RADIOACTIVE AND NON-RADIOACTIVE W↗
DOE OSTI2023

Visual Brick model authoring tool for building metadata standardization

In this study, the Brick ontology is a unified semantic metadata standard for building assets and their relationships, serving as a key enabler for effective interoperability and automation of building systems and analytics. However, creating a Brick model, in other words, standard semantic metadata based on the Brick ontology for a building dataset, can be a complex task. This paper presents two case studies of the creation of Brick models for real-world residential and commercial building datasets, highlighting the challenges during the Brick model creation process. Additionally, the paper introduces VizBrick, an interactive authoring tool for creating semantic building metadata. VizBrick facilitates the creation of Brick models by providing an intuitive visual interface and interactive capabilities, such as keyword search, automatic mapping suggestions, and recommendations. The use of VizBrick is shown to significantly reduce the time and effort required during the Brick model creation process.

42 ENGINEERING↗
DOE OSTI2024

GlideinBenchmark: collecting resource information to optimize provisioning

Choosing the right resource can speedup jobs completion, better utilize the available hardware and visibly reduce costs, especially when renting computers on the cloud. This was demonstrated in earlier studies on HEPCloud. But the benchmarking of the resources proved to be a laborious and time-consuming process. This paper presents GlideinBenchmark, a new Web application leveraging the pilot infrastructure of GlideinWMS to benchmark resources, and shows how to use the data collected and published by GlideinBenchmark to automate the optimal selection of resources. An experiment can select the benchmark or the set of benchmarks that most closely evaluate the performance of its workflows. With GlideinBenchmark and the help of the GldieinWMS Factory it controls the benchmark execution. Finally, a scheduler like HEPCloud’s Decision Engine can use the results to optimize resource provisioning.

Mambelli, Marco↗
DOE OSTI2025

ARMing the Edge: Demonstration of Edge Computing Field Campaign Report

Edge computing enables “next-to-instrument” control and intelligent data volume reduction and the potential for autonomous, adaptive measurement strategies such as for automated control of scan strategies for Doppler lidar (DL). Instruments with narrow bandwidth connections (e.g., ship and remote sites) can do scene determination and save phenomenon-appropriate data. For example, Doppler spectrum can be saved when clouds are detected by the instrument or automatic moment detection can take place in camera images and only preserve spectrum when non-monomodal spectra are detected. Automated control at the edge involves changing the sampling (temporal or scanning strategy) of an instrument to suit the phenomena both present and being studied (Jackson et al. 2020). Both data processing and instrument control introduces the possibility of a software-defined instrument.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2023

Label Assist: Personalized Travel Models for Longitudinal Data Collection

Understanding travel behavior is crucial to transportation decarbonization. OpenPATH is an open-source mobility platform which collects and analyzes human travel behavior at the individual level. The mobile application passively senses trips and prompts users to label them. However, users find the labeling process burdensome; less than half the trips are typically labeled, making much of the data unusable in aggregate analyses of mobility patterns. Prior work has addressed the response fatigue challenge through automated mode inference using sensor data, but sensors cannot capture all aspects of travel behavior. We explore an alternative approach in which we leverage prior user input to predict travel choices in novel trips. We first explore trip clustering methods and develop a novel two-step pipeline using DBSCAN and SVMs to extract realistic geospatial clusters. We then propose two strategies to predict trip labels: (i) clustering trips and extrapolating labels for similar trips, and (ii) random forest classification. The random forest approach is able to achieve - $70-80% accuracy (purpose: 72%, mode: 79%, replaced mode: 81%). These novel approaches to trip classification allow us to increase the rate of user labeling by suggesting predicted labels to be verified by the user. Unlabeled trips can also contribute to aggregate analyses, using label predictions and their associated confidences as a substitute. While there exist other travel survey apps with the ability to infer travel choices, to our knowledge, this is the first paper to describe such a supervised system and rigorously evaluate it.

ADVANCED PROPULSION SYSTEMS,ENERGY PLANNING, POLIC↗
DOE OSTI2022

Automated detection of mild cognitive impairment and dementia from voice recordings: A natural language processing approach

Automated computational assessment of neuropsychological tests would enable widespread, cost-effective screening for dementia. A novel natural language processing approach is developed and validated to identify different stages of dementia based on automated transcription of digital voice recordings of subjects’ neuropsychological tests conducted by the Framingham Heart Study (n = 1084). Transcribed sentences from the test were encoded into quantitative data and several models were trained and tested using these data and the participants’ demographic characteristics. Average area under the curve (AUC) on the held-out test data reached 92.6%, 88.0%, and 74.4% for differentiating Normal cognition from Dementia, Normal or Mild Cognitive Impairment (MCI) from Dementia, and Normal from MCI, respectively. The proposed approach offers a fully automated identification of MCI and dementia based on a recorded neuropsychological test, providing an opportunity to develop a remote screening tool that could be adapted easily to any language.

neuropsychological↗
DOE OSTI2020

Automated Identification of Molecular Crystals’ Packing Motifs

Packing motifs—patterns in how molecules orient relative to one another in a crystal structure—are an important concept in many subdisciplines of materials science because of correlations observed between specific packing motifs and properties of interest. That said, packing motif data sets have remained small and noisy due to intensive manual labeling processes and insufficient labeling schemes. The most prominent labeling algorithms calculate relative interplanar angles of nearest neighbor molecules to determine the packing motif of a molecular crystal, but this simple approach can fail when neighbors are naively sampled isotropically around the crystal structure. To remedy this issue, here we propose an optimization algorithm, which rotates the molecular crystal structure to find representative molecules that inform the packing motif. We package this algorithm into an automated framework—Autopack—which both optimally rotates the crystal structure and labels the packing motif based on the appropriate neighboring molecules. In this work, we detail the Autopack framework and its performance, which shows improvements compared to previous state-of-the-art labeling methods, providing the first quantitative point of comparison for packing motif labeling algorithms. Furthermore, using Autopack (available at https://ipo.llnl.gov/technologies/software/autopack), we perform the first large-scale study of potential relationships between chemicals’ compositions and packing motifs, which shows that these relationships are more complex than previously hypothesized from studies that used only tens of polycyclic aromatic hydrocarbon molecules. Autopack’s capabilities help pose next steps for crystal engineering research focusing not only on a molecule’s adoption of a specific packing motif but also on new structure–property relationships.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2024

Condition-Based Maintenance of a Circulating Water System of a Canadian Nuclear Power Plant using Machine Learning and Statistical Tools

Canada Deuterium Uranium pressurized-heavy-water reactors (PHWR) are a type of nuclear power plant that generate clean and reliable energy. The scope of this work is to automate data analysis methodologies to inform a condition-based maintenance strategy of a circulating water system (CWS) of a PHWR. The multiunit CWS provides a continuous supply of water to cool steam condensers, even during transient scenarios, thereby improving the thermal efficiency. This work aims to develop a machine learning (ML) based approach to detect anomalies in heterogeneous data of a CWS in a PHWR to help inform a predictive maintenance strategy. The heterogeneous data include textual and numeric time series data for a PHWR. Natural-language-processing (NLP)-based models are used to analyze textual data contained in work orders and operator logs and an event-timeseries correlation detection method is applied to assist anomalies diagnoses for CWS. An ML model Robust Linear Model (RLM) is also used to remove the seasonal variations in the system variable distributions based on distributions of environmental variables. A machine learning model, Density-Based Spatial Clustering of Applications with Noise (DBSCAN), trained on both original data and data without any seasonal variations will then be used to detect if an anomaly exists. Thus, by moving to an automated methodology to detect, classify, and forecast anomalies, the maintenance strategy would be based on component condition instead of a time-based schedule.

97 - MATHEMATICS AND COMPUTING↗
DOE OSTI2025

Fully Homomorphic Encryption

This code implements a Fully Homomorphic Encryption (FHE) system, enabling secure computation on encrypted data without requiring decryption. It supports encryption, decryption, and homomorphic operations like matrix multiplication and addition. This code is adaptable for integrating FHE into linear-time invariant (LTI) systems, including digital control and filtering. With proper configuration from subject matter expertise, encrypted system parameters and signals can be manipulated to perform tasks like state updates, output calculations, and convolution in the encrypted domain. By preserving the structure of LTI systems while ensuring privacy, the framework facilitates secure applications in areas such as autonomous systems, signal processing, and industrial automation. The code initializes the encryption system using parameters provided in the env dictionary. These parameters include the ciphertext modulus, key dimension, plaintext fixed-point scaling factor, and noise bound. During initialization, a secret key is generated, which is essential for encrypting and decrypting data securely. The modular design allows users to tailor these parameters to specific use cases or security requirements. The code implements multiple cryptographic schemes. The learning with errors (LWE) encryption method encodes cleartext message to their plaintext fixed-point representation then encrypted into ciphertext space with additive noise. This noise ensures the security of the scheme, relying on the computational hardness of the LWE problem. The code also includes the Gentry-Sahai-Waters (GSW) scheme based off the LWE problem. Homomorphic matrix multiplication is performed between the LWE and GSW to encrypted data. This is achieved using a decomposition function on the LWE ciphertext during the multiplication operation. For higher-dimensional data, the code includes a method to encrypt entire matrices (GSWMat) using GSW encryption. These encrypted matrices can then be used for homomorphic matrix multiplications (MatMult). The decryption function uses the secret key to recover the original plaintext, removing the added noise and scaling that was originally applied during encryption.

Lois, Roberts [Idaho National Laboratory (INL), Id↗
DOE OSTI2022

Computation of conventional and alternative jet fuel sensitivity to lean blowout

Large Eddy Simulations (LES) are performed to compute the sensitivity of a conventional (A-2) and an alternate bio-jet (C-1) fuel to Lean Blowout (LBO). A realistic aviation gas turbine engine combustor configuration is considered. Reliable experimental LBO data and OH* chemiluminescence data for the conventional and alternate jet fuel in the combustor configuration have recently become available. The present work utilizes a highly automated, on-the-fly meshing strategy, along with adaptive mesh refinement, to demonstrate the feasibility of capturing the realistic combustion processes. A Lagrangian framework, with initial conditions specified using measurements of spray statistics, is used to model the fuel spray. Newly developed compact reaction mechanisms based on fuel surrogates are validated for the A-2 and the C-1 fuels. The compact reaction mechanisms are implemented using a detailed finite rate chemistry solver. Spray statistics computed by the present LES simulations compare well with available measurements at stable flame conditions near the lean blowout limit. The computed shape of the stable flame as represented by line integrated OH concentrations compares well with the experimental OH* chemiluminescence data. Lean blowout is reached by gradually decreasing the fuel flow rate in the computations, similar to that in the experiments. The results of the LES simulations effectively capture the fuel composition effects and estimate the sensitivity of the LBO limits to the fuel type. The computed trends in LBO limits agree within engineering accuracy with the experimental results for conventional and alternative aviation fuels. The methodology for predicting the fuel composition effects on the lean blowout limits in a fully resolved realistic, complex combustor is established for the first time.

Adaptive mesh refinement↗
DOE OSTI2021

Optode Images, and Dissolved Oxygen Data from Sand and Sediment Flow-through Column Experiments associated with: "2-D Imaging of Dissolved Oxygen Concentration in Flow-Through Sediment Columns"

This data package is associated with the publication "2-D Imaging of Dissolved Oxygen Concentration in Flow-Through Sediment Columns" submitted to Frontiers in Water (Garayburu-Caruso et al., 2022). This work presents a novel application of planar optode technology to sediment columns that allows the study of well-constrained flow fields without the need to assume completely homogeneous flow. We introduce a flow-through column with the interior coated in an oxygen sensing film and a series of sampling ports as a unique tool capable of capturing changes in dissolved oxygen (DO) at high frequency (10 s) and spatial (< 1 mm) resolution across the length of a column. In addition, we provide custom scripts that allows image processing steps to be automated and consistently reproduced over a variety of experimental conditions. This dataset is comprised of four folders (1) 01_Riverbed_Sediment_Experiment, (2) 02_Sand_Experiment, (3) 01_Riverbed_Sediment_Raw_Images, and (4) 02_Sand_Experiment_Raw_Images. 01_Riverbed_Sediment_Experiment contains (1) a subfolder with csv files that contain statistical parameters from the linear regressions applied to each optode image, (2) a subfolder with csv files from DO data measured by in-line sensors, (3) a subfolder with csv files containing column and injection specific input parameters for the image processing script, and (4) a subfolder with scripts used to process the images and create associated figures. 02_Sand_Experiment contains (1) a subfolder with csv files containing column and injection-specific input parameters for the image processing script, and (2) a subfolder with scripts used to process the images and create associated figures. 01_Riverbed_Sediment_Raw_Images (spited in 3 parts due to size) contains a subfolder with raw red, and green images from the optode system during the MilliQ Injection, Vanillin Injection or Vanillin + Sampling Injection respectively. 02_Sand_Experiment_Raw_Images contains a subfolder with raw red, and green images from the optode system. Outside of the main folders there is a csv containing file-level metadata and a csv data dictionary defining column headers for all csv files contained in the data package.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2022

Applications of Autonomous Data Collection and Active Learning

Advances in sensors and robotics have dramatically improved the diversity of experimental approaches available to the materials community. Autonomous data collection platforms, either custom-made or commercially available, provide researchers with novel tools with which to probe materials behavior and perform advanced materials characterization. The application of novel control algorithms and active learning approaches can create much more robust experimental data, or can be used to improve the performance of existing characterization tools. Five papers within this special topic focus on experimental and computational methodologies for use in automatic data collection routines for materials characterization. From novel platforms for materials discovery to new statistical frameworks for assessing the autonomous experimentation process, these five papers highlight the diverse range of applications of automation for advancing materials science.

36 MATERIALS SCIENCE↗
DOE OSTI2020

Multimodal imaging and machine learning to enhance microscope images of shale

A machine learning based image processing workflow is presented to enhance shale source rock microscopic images obtained using diverse imaging platforms. Images were acquired from a 30 μm diameter cylindrical Vaca Muerta shale sample using both nondestructive Transmission X-Ray Microscopy (TXM, alternately referred to as nano computed tomography) and destructive Focused Ion Beam-Scanning Electron Microscopy (FIB-SEM). Output cross-sectional images from each modality were aligned using a combination of manual and automated registration techniques to create a registered image dataset. We then apply this dataset for two image processing tasks: prediction of image cross sections with SEM-like resolution from nondestructive TXM data and repair of charged region artifacts (localized accumulation of electrons) within SEM images. The image processing algorithms for both tasks use deep learning models, specifically image-to-image Convolutional Neural Networks (CNNs) and conditional Generative Adversarial Networks (cGANs). In the image enhancement tasks, we are able to achieve significant qualitative and quantitative improvement in TXM images. Here, the best model reaches an average Peak Signal to Noise Ratio (PSNR) of 15.8 dB. Conditioning on TXM data is also shown to reduce artifacts from SEM charging, achieving an average PSNR of 25.8 dB. Furthermore, our results suggest that properly trained and validated networks are capable of significant enhancement of images obtained using nondestructive techniques, thereby improving interpretation of two- and three-dimensional images while preserving samples for future use.

58 GEOSCIENCES↗
DOE OSTI2026

DIVA/DeviceEditor v6.1.2

DIVA is an end-to-end DNA design and construction management platform that streamlines how researchers design, build, and receive sequence-verified DNA constructs. Through a web-based BioCAD interface (DeviceEditor), researchers independently design DNA constructs and submit them to a centralized queue with a single action. Designs progress transparently through standardized states which allow researchers to track status and access finished constructs via a central DNA repository. Submitted designs are reviewed by dedicated staff for feasibility and optimization, reducing costly failures and improving downstream execution. Automated DNA assembly software optimizes construction strategies by reusing existing parts where possible and sourcing synthetic DNA only when needed. Standardized, sequence-agnostic assembly methods enable many independent constructs to be built in parallel using lab automation, dramatically increasing throughput. High-throughput next-generation sequencing is used to verify construct accuracy, with flexible platforms selected based on task requirements. Throughout the process, detailed success and failure data are captured and analyzed, enabling continuous improvement of assembly protocols. Compared to traditional, manual DNA construction workflows, DIVA offers higher scalability, transparency, reproducibility, and data-driven optimization.

Plahar, Hector [Lawrence Berkeley National Laborat↗
DOE OSTI2023

Evaluation of BISON metallic fuel performance modeling against experimental measurements within FIPD and IMIS databases

Simulations were conducted using the BISON fuel performance code on an automated process to read initial and operating conditions from two databases—the Fuels Irradiation and Physics Database (FIPD) and Integral Fast Reactor Materials Information System (IMIS) database. These databases contain metallic fuel data from the Experimental Breeder Reactor-II (EBR-II) and the Fast Flux Test Facility (FFTF). The work demonstrates use of an integrated framework to access EBR-II fuel pin data for evaluating fuel performance models contained within BISON to predict fuel performance of next-generation metallic fuel systems. Between IMIS and FIPD, there is enough information to conduct 1,977 unique EBR-II metallic fuel pin histories from 29 different experiments, and 338 pins from FFTF MFF-3 and MFF-5 with varying levels of details between the two databases. Each of these fuel performance histories includes a high-resolution power history, flux history, coolant channel flow rates, and coolant channel temperatures, and new model developments in BISON since the initial demonstration of this integrated framework. Fission gas release (FGR), cumulative damage fraction, fuel axial swelling, FCCI wastage thickness, cladding profilometry, and burnup were all simulated in BISON and compared to post-irradiation examination (PIE) results to evaluate BISON fuel performance modeling. Implementation of new fuel performance models into a generic BISON input file coupled with IMIS and FIPD yielded results with a better representation of physics than the initial evaluation of the integrated framework. Cladding profilometry, FGR, and fuel axial swelling were found to be in good agreement with PIE measurements for most of the pins simulated. The chosen mechanical contact solver was found to significantly impact the axial fuel swelling and cladding strain predictions when used in conjunction with the U-Pu-Zr hot-pressing model since it bound the fuel to prevent further swelling and increased hydrostatic stresses. This work suggests that fuel performance modeling in BISON under steady-state conditions represents the PIE data well and should be reassessed when new PIE data become available in IMIS and FIPD databases and when improved physical models to better capture fuel performance are added to BISON.

Paaren, Kyle M.↗