Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “data processing automation”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 487 records · Page 27
All sourcesNASA NTRSDOE OSTI
DOE OSTI2024

Automated analysis of unlabeled PV data with Solar Data Tools software: Overview and feature updates

Distributed rooftop PV systems: ubiquitous, yet commonly have unlabeled data Difficult or impossible to form a performance index We developed Solar Data Tools (SDT), an open-source Python library for analyzing PV power (and irradiance) time-series data SDT enables analysis of unlabeled PV data—no model, no meteorological data, no performance index required Takes a statistical signal processing approach Data processing steps are largely pre-defined and automatic regardless of system type—from utility tracking systems to multi-pitch rooftop systems

Meyers-Im, Bennet E↗
DOE OSTI2020

Quantification of silicon carbide grain structure in TRISO fuel by BSE image analysis

The grain structure of the silicon carbide (SiC) layer in tristructural-isotropic (TRISO) fuels has commonly been qualified in fuel specifications based on a qualitative comparison to visual standards to ensure undesirable microstructure are not produced. This approach is inherently dependent on subjective judgement and provides limited data for analysis or comparison. An alternative method to rapidly quantify (i.e., provide a numeric measure of) the grain structure of the SiC layer in TRISO fuels has been developed. This method relies on the identification of silicon carbide grain boundaries based on brightness variation in backscattered electron (BSE) images through the application of automated image processing algorithms. The speed with which the requisite BSE imaging and analysis may be completed enables the practical use of this method on an industrial scale as a part of qualifying TRISO fuel to meet a given specification on silicon carbide grain structure, which is an important parameter for TRISO fuel performance. The method is benchmarked by comparison to prior Electron-Backscattered Diffraction (EBSD) results for TRISO particles from the Advanced Gas Reactor (AGR) program.

11 NUCLEAR FUEL CYCLE AND FUEL MATERIALS↗
DOE OSTI2024

Machine Learning Automation Pipeline

Machine Learning Automation Pipeline (MLAP) is a package to perform machine learning (ML) analysis in a step by step manner, starting with data extraction until analysis and prediction. The scripts provide the users option to chose an action such as "Extract", "Prep", and "Train" and numerous cases can be launched with just a single command. The inputs for each case are provided using a JSON file. The simulation results of several cases can be assessed using an automated process and analyzed for various metrics pertinent to ML analysis.

Jha, Pankaj↗
DOE OSTI2026

lumicap v0.1

Automated HDR luminance imaging system designed for daylighting research and building science. It controls a fisheye-lens camera to capture time-lapse bracket sequences, merges them into calibrated HDR images, and runs a full post-processing pipeline — all unattended. Features: - Scheduled LDR bracket capture via gphoto2 - HDR merging with vignetting, ND filter, and fisheye projection corrections - Illuminance and luminance meter integration (Konica Minolta T-10A, LS-100/150) - Daylight glare probability (DGP) and solar position computation - Automated false-color rendering, JPEG thumbnails, and daily time-lapse video - CSV data logging per timestep Uses: - Long-term monitoring of daylight conditions in buildings - Glare analysis for occupant comfort research - Solar irradiance and sky luminance studies Advantages: - End-to-end automation — capture, calibration, analysis, and archiving run without manual intervention - Built on the proven Radiance toolchain, ensuring photometrically accurate HDR output - Hardware-agnostic meter support via serial auto-detection - Lightweight — no GUI overhead, deployable on a headless Raspberry Pi or similar embedded system

Wang, Taoning [Lawrence Berkeley National Laborato↗
DOE OSTI2024

A miniaturized feedstocks-to-fuels pipeline for screening the efficiency of deconstruction and microbial conversion of lignocellulosic biomass

Sustainably grown biomass is a promising alternative to produce fuels and chemicals and reduce the dependency on fossil energy sources. However, the efficient conversion of lignocellulosic biomass into biofuels and bioproducts often requires extensive testing of components and reaction conditions used in the pretreatment, saccharification, and bioconversion steps. This restriction can result in a significant and unwieldy number of combinations of biomass types, solvents, microbial strains, and operational parameters that need to be characterized, turning these efforts into a daunting and time-consuming task. Here we developed a high-throughput feedstocks-to-fuels screening platform to address these challenges. The result is a miniaturized semi-automated platform that leverages the capabilities of a solid handling robot, a liquid handling robot, analytical instruments, and a centralized data repository, adapted to operate as an ionic-liquid-based biomass conversion pipeline. The pipeline was tested by using sorghum as feedstock, the biocompatible ionic liquid cholinium phosphate as pretreatment solvent, a “one-pot” process configuration that does not require ionic liquid removal after pretreatment, and an engineered strain of the yeast Rhodosporidium toruloides that produces the jet-fuel precursor bisabolene as a conversion microbe. By the simultaneous processing of 48 samples, we show that this configuration and reaction conditions result in sugar yields (~70%) and bisabolene titers (~1500 mg/L) that are comparable to the efficiencies observed at larger scales but require only a fraction of the time. We expect that this Feedstocks-to-Fuels pipeline will become an effective tool to screen thousands of bioenergy crop and feedstock samples and assist process optimization efforts and the development of predictive deconstruction approaches.

09 BIOMASS FUELS↗
DOE OSTI2026

Best practices in software development for robust and reproducible geoscientific models based on insights from the Global Carbon Budget's dynamic vegetation models

Computational models play an increasingly vital role in scientific research by enabling the numerical simulation of complex processes. Such models are also fundamental in geosciences. For instance, they offer critical insights into the impacts of global change on the Earth system today and in the future. Beyond their value as research tools, models are also software products and should therefore adhere to certain established software engineering standards. However, scientists are rarely trained as software developers, which can lead to potential deficiencies in software quality like unreadable, inefficient, or erroneous code. The complexity of models, coupled with their integration into broader workflows, also often makes it challenging to reproduce results, evaluate processes, and build upon them. In this paper, we review the state and current practices of the development processes of the state-of-the-art land surface models used by the Global Carbon Budget. We combine the experience of modelers from the respective research groups with the expertise of software engineers from tech companies to outline key principles and tools for improving software quality in research. We explore four main areas: (1) model testing and validation, (2) scientific, technical, and user documentation, (3) version control, continuous integration, and code review, and (4) the portability and reproducibility of workflows. Our review reveals that while modeling communities are incorporating many best practices, significant room for improvement remains in areas such as automated testing, automated documentation, and reproducibility. Therefore, we here identify and promote essential software engineering practices, including numerous examples of practices from within the community that can serve as guidelines for other models and could help streamline processes across the entire community. We conclude with an open-source example implementation of these principles, demonstrating portable and reproducible data flows, a continuous integration setup, and web-based visualizations. This example may serve as a practical resource for model developers, users, and all scientists engaged in scientific programming.

Gregor, Konstantin [Technical Univ. of Munich (Ger↗
DOE OSTI2023

Exploring Microsimulation Process for Energy Impact Evaluation of Connected and Automated Vehicles

In this paper, the authors present a microsimulation-based methodological approach for evaluating the energy impact of connected and automated vehicles (CAVs). They use an open-source micro-simulator, SUMO, and provide a way to set up a simulation environment that emulates real-world traffic dynamics. They also employ the Intelligent Driver Model to represent human drivers and calibrate its driving behavior using real-world traffic data and driving statistics. The authors conduct extensive simulation studies considering different penetration rates of CAVs, different car-following models, and varying car-following model parameters. Using the state-of-the-art Future Automotive System Technology Simulator (FASTSim), they estimate the fuel economy of each vehicle and analyze the energy impact of the given CAV implementation. Finally, the authors analyze the possible factors affecting the simulation results, and also discuss limitations and future work.

ADVANCED PROPULSION SYSTEMS↗
DOE OSTI2024

Comparing Sensor Fusion and Multimodal Chemometric Models for Monitoring U(VI) in Complex Environments Representative of Irradiated Nuclear Fuel

Optical sensors and chemometric models were leveraged for the quantification of uranium(VI) (0–100 μg mL –1 ), europium (0–150 μg mL –1 ), samarium (0–250 μg mL –1 ), praseodymium (0–350 μg mL –1 ), neodymium (0–1000 μg mL –1 ), and HNO 3 (2–4 M) with varying corrosion product (iron, nickel, and chromium) levels using laser fluorescence, Raman scattering, and ultraviolet–visible–near-infrared absorption spectra. In this paper, an efficient approach to developing and evaluating tens of thousands of partial least-squares regression (PLSR) models, built from fused optical spectra or multimodal acquisitions, is discussed. Each PLSR model was optimized with unique preprocessing combinations, and features were selected using genetic algorithm filters. The 7-factor D-optimal design training set contained just 55 samples to minimize the number of samples. The performance of PLSR models was evaluated by using an automated latent variable selection script. PLS1 regression models tailored to each species outperformed a global PLS2 model. PLS1 models built using fused spectra data and a multimodal (i.e., analyzed separately) approach yielded similar information, resulting in percent root-mean-square error of prediction values of 0.9–5.7% for the seven factors. Further, the optical techniques and data processing strategies established in this study allow for the direct analysis of numerous species without measuring luminescence lifetimes or relying on a standard addition approach, making it optimal for near-real-time, in situ measurements. Nuclear reactor modeling helped bound training set conditions and identified elemental ratios of lanthanide fission products to characterize the burnup of irradiated nuclear fuel. Leveraging fluorescence, spectrophotometry, experimental design, and chemometrics can enable the remote quantification and characterization of complex systems with numerous species, monitor system performance, help identify the source of materials, and enable rapid high-throughput experiments in a variety of industrial processes and fundamental studies.

11 NUCLEAR FUEL CYCLE AND FUEL MATERIALS↗
DOE OSTI2025

Active oversight and quality control in standard Bayesian optimization for autonomous experiments

The fusion of experimental automation and machine learning has catalyzed a new era in materials research, prominently featuring Gaussian Process (GP) Bayesian Optimization (BO) driven autonomous experiments. Here we introduce a Dual-GP approach that enhances traditional GPBO by adding a secondary surrogate model to dynamically constrain the experimental space based on real-time assessments of the raw experimental data. This Dual-GP approach enhances the optimization efficiency of traditional GPBO by isolating more promising space for BO sampling and more valuable experimental data for primary GP training. We also incorporate a flexible, human-in-the-loop intervention method in the Dual-GP workflow to adjust for unanticipated results. We demonstrate the effectiveness of the Dual-GP model with synthetic model data and implement this approach in autonomous pulsed laser deposition experimental data. This Dual-GP approach has broad applicability in diverse GPBO-driven experimental settings, providing a more adaptable and precise framework for refining autonomous experimentation for more efficient optimization.

36 MATERIALS SCIENCE↗
DOE OSTI2024

Real-time tracking of structural evolution in 2D MXenes using theory-enhanced machine learning

In situ Electron Energy Loss Spectroscopy (EELS) combined with Transmission Electron Microscopy (TEM) has traditionally been pivotal for understanding how material processing choices affect local structure and composition. However, the ability to monitor and respond to ultrafast transient changes, now achievable with EELS and TEM, necessitates innovative analytical frameworks. Here, we introduce a machine learning (ML) framework tailored for the real-time assessment and characterization of in operando EELS Spectrum Images (EELS-SI). We focus on 2D MXenes as the sample material system, specifically targeting the understanding and control of their atomic-scale structural transformations that critically influence their electronic and optical properties. This approach requires fewer labeled training data points than typical deep learning classification methods. By integrating computationally generated structures of MXenes and experimental datasets into a unified latent space using Variational Autoencoders (VAE) in a unique training method, our framework accurately predicts structural evolutions at latencies pertinent to closed-loop processing within the TEM. This study presents a critical advancement in enabling automated, on-the-fly synthesis and characterization, significantly enhancing capabilities for materials discovery and the precision engineering of functional materials at the atomic scale.

47 OTHER INSTRUMENTATION↗
DOE OSTI2025

Expediting field-effect transistor chemical sensor design with neuromorphic spiking graph neural networks

Improving the sensitive and selective detection of analytes in a variety of applications requires accelerating the rational design of field-effect transistor (FET) chemical sensors. Achieving high-performance detection relies on identifying optimal probe materials that can effectively interact with target analytes, a process traditionally driven by chemical intuition and time-consuming trial-and-error methods. To address the difficulties in probe screening for FET sensor development, this work presents a methodology that combines neuromorphic machine learning (ML) architectures, specifically a hybrid spiking graph neural network (SGNN), with an enriched dataset of physicochemical properties through semi-automated data extraction using large language models. Achieving a classification accuracy of 0.89 in predicting sensor sensitivity categories, the SGNN model outperformed traditional ML techniques by leveraging its ability to capture both global physicochemical properties and sparse topological features through a hybrid modeling framework. Next-generation sensor design was informed by the actionable insights into the connections between material properties and sensing performance offered by the SGNN framework. Through virtual screening for the detection of per- and polyfluoroalkyl substances (PFAS) as a use case, the effectiveness of the SGNN model was further validated. Density functional theory simulations confirmed graphene as a promising active material for PFAS detection as suggested by the SGNN framework. By bridging gaps in predictive modeling and data availability, this integrated approach provides a strong foundation for accelerating advancements in FET sensor design and innovation.

Ferreira, Rodrigo Pires [Univ. of Chicago, IL (Uni↗
DOE OSTI2021

CASTELO: clustered atom subtypes aided lead optimization—a combined machine learning and molecular modeling method

Background: Drug discovery is a multi-stage process that comprises two costly major steps: pre-clinical research and clinical trials. Among its stages, lead optimization easily consumes more than half of the pre-clinical budget. We propose a combined machine learning and molecular modeling approach that partially automates lead optimization workflow in silico, providing suggestions for modification hot spots. Results: The initial data collection is achieved with physics-based molecular dynamics simulation. Contact matrices are calculated as the preliminary features extracted from the simulations. To take advantage of the temporal information from the simulations, we enhanced contact matrices data with temporal dynamism representation, which are then modeled with unsupervised convolutional variational autoencoder (CVAE). Finally, conventional and CVAE-based clustering methods are compared with metrics to rank the submolecular structures and propose potential candidates for lead optimization. Conclusion: With no need for extensive structure-activity data, our method provides new hints for drug modification hotspots which can be used to improve drug potency and reduce the lead optimization time. It can potentially become a valuable tool for medicinal chemists.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2024

Application of a Prize Mechanism to Address Data Utilization Challenges at Utilities

The electric industry sector is facing an “explosion” of data from a variety of sources. Electric sector stakeholders need to define how to capitalize on large datasets, both those they create and those from other sources (like data on weather, buildings, electric vehicles, etc.), to improve reliability and resilience and meet the changing system dynamics from renewable integration. For the electricity sector to fully utilize these vast new datasets, it must undergo a transformation in how it manages data quality, storage, and processing. The U.S. Department of Energy (DOE) Office of Electricity (OE) is committed to accelerating research, development, and demonstration of new technologies and tools within the electricity sector to advance reliability, resilience, and affordable operation of the power system. Through the prize mechanism, OE identified two widespread data-related challenges for utilities—load modeling and data analysis automation—and offered an opportunity for utilities and teams of software engineers to identify additional challenges faced by utilities. After completing one round of the American-Made Digitizing Utilities Prize, OE, the National Renewable Energy Laboratory (NREL) as the prize administrator, and Pacific Northwest National Laboratory (PNNL) as the domain experts have compiled the results and lessons learned to feed into the second round of the prize.

29 ENERGY PLANNING, POLICY, AND ECONOMY↗
DOE OSTI2022

Semi-automated Design of Artificial Intelligence Earth Science Models

Prediction and observation of water cycles involve not only patterns isolated in space and time, but rather modeling complex spatio-temporal relationships across multiple sources of data and domains. For instance, Evapotranspiration (ET) and Leaf Area Indexes (LAI) are two critical components in DOE’s Energy Exascale Earth System Model (E3SM). Accurate assessments of ET and LAI are critical for understanding hydrological processes, deforestation, crop yield, and irrigation impacts. However, current ET estimates for global simulations are available at very coarse spatial resolution. They are usually derived from satellite data based on broad plant functional types (PFT), which fail to capture the fine-scale variations due to change in vegetation type across the globe. Within this context and in light of the data-model integration challenges highlighted in the EESSD Strategic Plan, the new era of AI model development for geosciences calls for data-driven methods that provide domain scientists with estimations of parameters such as PFT and LAI in an efficient, interpretable, and easy-to-operate manner.

Ambrozio Dias, Philipe↗
DOE OSTI2026

Exploring the potential of using L-Band InSAR for the mapping of flooded vegetation in tropical wetlands

Wetlands play a critical role in global water and carbon cycles, yet monitoring their water extent remains difficult, particularly beneath dense vegetation. SAR-based techniques such as backscatter thresholding are limited by complex scattering mechanisms, while fully polarimetric SAR (PolSAR) data capable of detecting doublebounce scattering remain scarce. To address these challenges, this study evaluates the potential of Interferometric SAR (InSAR) for mapping water surfaces beneath vegetation, termed flooded vegetation, using the Atrato floodplain in Colombia as a case study. We develop an automated workflow combining InSAR fringe detection with local phase homogeneity analysis and random sampling of processing parameters to generate probabilistic flooded vegetation maps. Applied to ALOS PALSAR-1 L-band image pairs from 2007–2011, the workflow captures seasonal fluctuations in flooded extent ranging from 500 to 1,500 km2. Compared to other L-band SAR inundation products, the InSAR-based maps identify broader flooded areas, with ~70% agreement in pairwise comparisons. Around 84% of detections align with existing wetland inventories and seasonal changes correspond with regional hydrological indicators, including terrestrial water storage anomalies and water gauge measurements. PolSAR analysis shows that InSAR complements backscatter-based methods by detecting inundation in areas with weak double-bounce signals. These findings suggest that combining InSAR with backscatter-based methods can improve detection of flooded vegetation, which is especially relevant for the upcoming NISAR mission that will offer frequent global L-band observations.

Coastal inundation↗
DOE OSTI2026

EDXplorer: A Utility for APA Analysis

The automated particle analysis (APA) method of scanning electron microscopy (SEM) energy dispersive X-ray spectroscopy (EDS/EDX) is a useful tool for analyzing the elemental and morphological data of particulate samples. Often, such datasets have many thousands of particles, and it can be difficult to sift through the data to find meaningful trends. EDXplorer is a software utility for processing APA data. They enable the user to easily load data and determine the important components and aspects of the datasets using a powerful and versatile library of data plotting functions, mainly centered around scatter plots and histograms. These programs are intended to fill a void in the data processing of data from certain instruments, where often the user must rely on their own code or other software that is not user-friendly. EDXplorer is intended as a general plotting utility for browsing through data and discovering data trends.

Moseley, Duncan [ORNL] (ORCID:0000000343518347)↗
DOE OSTI2023

BinaRena: a dedicated interactive platform for human-guided exploration and binning of metagenomes

Background: Exploring metagenomic contigs and “binning” them into metagenome-assembled genomes (MAGs) are essential for the delineation of functional and evolutionary guilds within microbial communities. Despite the advances in automated binning algorithms, their capabilities in recovering MAGs with accuracy and biological relevance are so far limited. Researchers often find that human involvement is necessary to achieve representative binning results. This manual process however is expertise demanding and labor intensive, and it deserves to be supported by software infrastructure. Results: We present BinaRena, a comprehensive and versatile graphic interface dedicated to aiding human operators to explore metagenome assemblies via customizable visualization and to associate contigs with bins. Contigs are rendered as an interactive scatter plot based on various data types, including sequence metrics, coverage profiles, taxonomic assignments, and functional annotations. Various contig-level operations are permitted, such as selection, masking, highlighting, focusing, and searching. Binning plans can be conveniently edited, inspected, and compared visually or using metrics including silhouette coefficient and adjusted Rand index. Completeness and contamination of user-selected contigs can be calculated in real time. In demonstration of BinaRena’s usability, we show that it facilitated biological pattern discovery, hypothesis generation, and bin refinement in a complex tropical peatland metagenome. It enabled isolation of pathogenic genomes within closely related populations from the gut microbiota of diarrheal human subjects. It significantly improved overall binning quality after curating results of automated binners using a simulated marine dataset. Conclusions: BinaRena is an installation-free, dependency-free, client-end web application that operates directly in any modern web browser, facilitating ease of deployment and accessibility for researchers of all skill levels. The program is hosted at https://github.com/qiyunlab/binarena, together with documentation, tutorials, example data, and a live demo. It effectively supports human researchers in intuitive interpretation and fine tuning of metagenomic data.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2025

Informing Plant Asset Reliability and Availability Through AI-Driven Analysis of Operator Logs

The availability and reliability of nuclear power plant (NPP) structures, systems, and components (SSCs) are critical parameters for NPP safety. Tracking these parameters is necessary but costly and labor-intensive, requiring the collection and evaluation of SSC event data such as shutdowns, startups, and failures. To show how these events are needed for the parameters an example is given: one measure of reliability is based on the number of equipment failure events and the number of run hours (i.e., the time from a startup event to a shutdown event). Here, this work investigates using artificial intelligence (AI) to mine NPP operator log entry texts for SSC event data. Four AI approaches were explored for identifying these events, including natural language processing (NLP) methods, generative AI, generative AI combined with NLP, and topic modeling. A key challenge addressed with all four approaches is the brevity of operator log entries. Among these four a neural network–based NLP method was shown to be the most promising for this application, achieving F1 scores of 86.0% for shutdowns, 92.2% for startups, and 80.4% for failures on a subject-matter-expert-curated dataset from NPP operator logs, compared to a baseline of 66.6% for a random classifier. This shows that NLP methods can perform better than generative AI. Additionally, the NLP methods combined with generative AI were shown to perform better than generative AI alone. Generative AI was most successful at providing the background information for the NLP methods to use. This work demonstrates the potential to use AI to automate parameter collection from NPP operator log entries and other records.

97 - MATHEMATICS AND COMPUTING↗