SEARCH · Engineering Papers

Results for “Data processing and standardization”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 325 records · Page 18

All sources NASA NTRS DOE OSTI

DOE OSTI2022

A New Integrated Analysis Suite for Fast-Ion Study in KSTAR

Here, an integrated workflow for fast-ion analysis was developed by adapting the One Modeling Framework for Integrated Task (OMFIT) workflow manager to support a standard and unified analysis platform for KSTAR users. The newly established analysis suite offers a graphical user interface–based workflow to enable users to readily access and handle experimental data archived in various data formats and servers. Further, users can analyze the data by importing modules designed for conducting certain tasks, such as profile fitting, equilibrium reconstruction, and postprocessing of tokamak data. The procedures for preparing the inputs for fast-ion simulations are streamlined by a common workflow manager, which enables the parallel processing of various tasks to efficiently analyze large fast-ion datasets. The OMFIT platform comprises a flexible Python-based application that enables users to freely manipulate the Python scripts for applications that are unavailable in the standard workflow. The framework also offers mapping tools to translate the output data into the Integrated Modeling and Analysis Suite format to maintain application compatibility for future ITER burning plasma experiments.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗

DOE OSTI2020

Generating Uncertainty Distributions for Seismic Signal Onset Times

Signal arrival-time estimation plays a critical role in a variety of downstream seismic analyses, including location estimation and source characterization. Any arrival-time errors propagate through subsequent data-processing results. In this article, we detail a general framework for refining estimated seismic signal arrival times along with full estimation of their associated uncertainty. Using the standard short-term average/long-term average threshold algorithm to identify a search window, we demonstrate how to refine the pick estimate through two different approaches. In both cases, new waveform realizations are generated through bootstrap algorithms to produce full a posteriori estimates of uncertainty of onset arrival time of the seismic signal. The onset arrival uncertainty estimates provide additional data-derived information from the signal and have the potential to influence seismic analysis along several fronts.

58 GEOSCIENCES↗

DOE OSTI2022

FAIRification, Quality Assessment, and Missingness Pattern Discovery for Spatiotemporal Photovoltaic Data

The ongoing growth of the photovoltaic market has pushed the demand for power forecasting and performance evaluation for a huge population of PV power plants. Through access to a large number of time series data sets from different power plants, we have found common issues that impede the modeling process. Namely, the time series data are hard to transfer between groups due to differences in variable nomenclature, and the quality of the data sets can vary. We address the issue of variable nomenclature by FAIRifying spatiotemporal PV time series data. Through the creation of a solar power plant ontology, we propose standards for the naming and structure of metadata used to describe the data from these power plants. Using the structure from this ontology, we have developed both R and Python packages for the automation of the FAIRification process. We have also developed an R package that automates the analysis of the quality of a data set through the designation of letter grades. With access to large time series data sets across many power plants, we can utilize spatiotemporal coherence between the sites in order to improve the quality of our data. To solve the issue of data missingness, we propose the use of Spatiotemporal-GNN autoencoders to detect and impute missing values from a data set by utilizing data from power plants nearby.

14 SOLAR ENERGY↗

DOE OSTI2020

Improved sampling technique to collect natural gas from hydrate-bearing pressure cores

High quality gas compositional data are an important factor in interpreting the genetic source of natural gas hosted in hydrate-bearing sediments and other subsurface systems. In order to accurately characterize the composition of gas samples degassed from hydrate-bearing pressure cores, one must use a reproducible sampling technique that minimizes artifacts of the sampling process. Herein, we review sediment core degassing techniques and compare data obtained from a commonly used degassing approach, which we term the standard quantitative degassing (SQD) technique, to our newly developed modified quantitative degassing (MQD) method designed to minimize atmospheric contamination and gas-water interactions. The SQD method allows sample gas to interact with water in a bubbling chamber, which we hypothesize could alter the gas composition following mixing with water or dissolved gases in the bubbling chamber. Whereas, the MQD method allows for the collection of sample gas prior to the bubbling chamber. To compare the SQD and MQD methods, we performed a side-by-side comparison of noble (He, Ne, Ar, Kr, and Xe), major (H 2 , N 2 , O 2 , and CO 2 ), and hydrocarbon (CH 4 , C 2 H 6 , C 3 H 8 , i-C 4 H 10 , C 4 H 10 , i-C 5 H 12 , C 5 H 12 ) gas concentrations and select isotopic compositions obtained using both sample collection techniques. Gas samples were collected from hydrate-bearing pressure cores recovered and maintained under hydrate stable conditions from the northern Gulf of Mexico during the UT-GOM 2 -1 Expedition. The MQD method displayed significantly lower concentrations of atmospheric gases, higher proportions of hydrocarbon gases, lower ratios of C 1 /C 2 + , and heavier stable carbon and hydrogen isotopes of methane than the SQD method. These results demonstrate that the MQD method reduced air contamination and minimized alteration of the hydrocarbon gases. Finally, we conclude this method may be important for future work that seeks to determine the composition of natural gas from pressure cores using quantitative degassing experiments, especially those seeking to measure major (e.g., N 2 ) and noble gases.

58 GEOSCIENCES↗

DOE OSTI2025

Machine Learning for Automated Weld Quality Monitoring and Control

Resistance Spot Welding (RSW) is a critical process in the automotive industry, valued for its cost-effectiveness, short cycle time, and robustness. However, achieving consistent high-quality joints remains challenging due to the complex interplay of various factors, like materials, processes, and manufacturing uncertainties, etc. Under the collaborative project between Oak Ridge National Laboratory (ORNL) and General Motors (GM), we have developed a robust and expansible machine learning (ML) framework aimed at enhancing quality control in RSW. By harnessing the power of machine learning, we have developed the ability to ensure every aspect of the welding process, from the initial process design stage to the final weld joint quality. The framework operates by analyzing a variety of data streams, including in-line process signals, process parameters, materials, and postprocessed weld joint data. Through this analysis, the models have been trained to detect deviations from optimal quality standards, leveraging their ability to identify signature data patterns and anomalies within in-line signals and construct complex correlations between these signals and weld quality parameters. Meanwhile, the machine learning framework is designed to adapt to a variety of materials, including high strength steels and aluminum alloys, etc. Its flexible architecture facilitates the incorporation of diverse data sources and features, enabling precise modeling and prediction across a broad range of material properties and weld quality variables. The expansible ML frameworks represent a promising transformation in weld quality monitoring and control, empowering industry to achieve high levels of efficiency, consistency, and reliability in manufacturing.

99 GENERAL AND MISCELLANEOUS↗

DOE OSTI2025

Machine Learning for Automated Weld Quality Monitoring and Control

42 ENGINEERING↗

DOE OSTI2021

Latent map Gaussian processes for mixed variable metamodeling

Gaussian processes (GPs) are ubiquitously used in sciences and engineering as metamodels. Standard GPs, however, can only handle numerical or quantitative variables. Here we introduce latent map Gaussian processes (LMGPs) that inherit the attractive properties of GPs and are also applicable to mixed data which have both quantitative and qualitative inputs. The core idea behind LMGPs is to learn a continuous, low-dimensional latent space or manifold which encodes all qualitative inputs. To learn this manifold, we first assign a unique prior vector representation to each combination of qualitative inputs. We then use a low-rank linear map to project these priors on a manifold that characterizes the posterior representations. As the posteriors are quantitative, they can be directly used in any standard correlation function such as the Gaussian or Matern. Hence, the optimal map and the corresponding manifold, along with other hyperparameters of the correlation function, can be systematically learned via maximum likelihood estimation. Through a wide range of analytic and real-world examples, we demonstrate the advantages of LMGPs over state-of-the-art methods in terms of accuracy and versatility. In particular, we show that LMGPs can handle variable-length inputs, have an explainable neural network interpretation, and provide insights into how qualitative inputs affect the response or interact with each other. We also employ LMGPs in Bayesian optimization and illustrate that they can discover optimal compound compositions more efficiently than conventional methods that convert compositions to qualitative variables via manual featurization.

42 ENGINEERING↗

DOE OSTI2025

Toward Drilling the Perfect Geothermal Well: An International Research Coordination Network for Geothermal Drilling Optimization Supported by Deep Machine Learning and Cloud Based Data Aggregation

The EDGE project, supported by the U.S. Department of Energy Geothermal Technologies Office under award DE-EE0008793, established a data-driven framework for improving the efficiency, cost-effectiveness, and reliability of geothermal well drilling. The project focused on developing scalable data infrastructure, advanced machine learning and probabilistic models, and integrated analytics tools to support continuous drilling optimization. A central objective was to reduce geothermal drilling costs by up to seventy percent while minimizing the risk of well failure through predictive diagnostics and adaptive planning. Over the project period, a comprehensive data repository was designed and deployed, incorporating records from over one hundred geothermal wells across varied geological settings. This repository supported both structured and unstructured data and adhered to FAIR data principles, enabling provenance tracking, quality control, and standardized metadata. The project introduced automated ingestion pipelines and a cloud-hosted platform that facilitated access to raw, processed, and derived datasets. This infrastructure served as the foundation for model development and analysis. Machine learning workflows were developed to predict key drilling metrics including rate of penetration, non-productive time, and total drilling costs. Self-organizing maps and dimensionality reduction methods were used to uncover operational patterns and outliers, while supervised learning algorithms such as random forests and deep neural networks were applied to forecast performance outcomes. The models were validated on heterogeneous datasets from both U.S. and Icelandic fields, demonstrating variable but significant predictive accuracy. The results indicated that finer temporal resolution, inclusion of lithological data, and consistency in operational annotations could substantially improve model performance. The project also implemented process mining techniques to reconstruct state-transition models from drilling event logs. These models enabled the identification of deviations from optimal workflows and provided insights into recurring failure modes. Analysis of non-productive time highlighted the impact of equipment failures, geological challenges, and human factors, offering opportunities for targeted mitigation strategies. The EDGE Dashboard was developed as a web-based expert system integrating data visualization, model outputs, and user-driven queries. It provided an accessible interface for operators to explore historical data, evaluate predicted outcomes, and compare drilling scenarios. Initial feedback from project partners suggested that the dashboard could serve as a foundation for more advanced advisory and optimization tools. Overall, the EDGE project demonstrated the feasibility and value of applying modern data science techniques to geothermal drilling. It delivered a set of interoperable tools and models that can support more efficient, lower-risk well development. The findings point toward a viable path for transitioning from advisory analytics to semi-autonomous drilling systems, contingent on continued collaboration, expanded datasets, and field validation. The project results have immediate relevance for drilling operations, data management practices, and future geothermal R&D efforts aimed at achieving reliable, cost-competitive geothermal energy at scale.

15 GEOTHERMAL ENERGY↗

DOE OSTI2021

Using GANs with adaptive training data to search for new molecules

The process of drug discovery involves a search over the space of all possible chemical compounds. Generative Adversarial Networks (GANs) provide a valuable tool towards exploring chemical space and optimizing known compounds for a desired functionality. Standard approaches to training GANs, however, can result in mode collapse, in which the generator primarily produces samples closely related to a small subset of the training data. In contrast, the search for novel compounds necessitates exploration beyond the original data. In this work, we present an approach to training GANs that promotes incremental exploration and limits the impacts of mode collapse using concepts from Genetic Algorithms. In our approach, valid samples from the generator are used to replace samples from the training data. We consider both random and guided selection along with recombination during replacement. By tracking the number of novel compounds produced during training, we show that updates to the training data drastically outperform the traditional approach, increasing potential applications for GANs in drug discovery.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

DOE OSTI2024

Towards in-situ certification of additively manufactured parts: the vital roles of physics-based and data-driven models

Certifying additively manufactured (AM) parts in-situ at the completion of a build is an enticing prospect, as it can help reduce the high costs associated with post-build testing and evaluation. However, achieving this goal presents significant challenges that may keep it aspirational for the foreseeable future. Nonetheless, incremental progress can pave the way forward. A critical aspect of in-situ certification involves continuous quality checking due to the random nature of the AM process and the difficulties in detecting defects or anomalies once layers are built over. While real-time in-situ monitoring strategies assisted by machine learning (ML) play a pivotal role in auditing part quality, they must ideally be supported by real-time (or near real-time) adaptive process control enabled by ML-assisted decision-making. By analyzing in-situ monitoring data in real-time (or near real- time) to dynamically adjust manufacturing parameters, such intervention can ensure AM parts are built to meet stringent certification standards. This can be achieved virtually by using high-fidelity performance models for the physical testing and evaluation tasks. In this short editorial, we discuss the key contributions made by data-driven and physics-based models in providing intelligence to the monitoring and process control tasks underpinning in-situ certification and in the simulation of the build’s performance under test and service conditions. While our focus lies in metal AM, the concepts discussed here are also relevant to other AM processes.

: In-situ monitoring↗

DOE OSTI2023

A genomic data archive from the Network for Pancreatic Organ donors with Diabetes

The Network for Pancreatic Organ donors with Diabetes (nPOD) is the largest biorepository of human pancreata and associated immune organs from donors with type 1 diabetes (T1D), maturity-onset diabetes of the young (MODY), cystic fibrosis-related diabetes (CFRD), type 2 diabetes (T2D), gestational diabetes, islet autoantibody positivity (AAb+), and without diabetes. nPOD recovers, processes, analyzes, and distributes high-quality biospecimens, collected using optimized standard operating procedures, and associated de-identified data/metadata to researchers around the world. Herein describes the release of high-parameter genotyping data from this collection. 372 donors were genotyped using a custom precision medicine single nucleotide polymorphism (SNP) microarray. Data were technically validated using published algorithms to evaluate donor relatedness, ancestry, imputed HLA, and T1D genetic risk score. Additionally, 207 donors were assessed for rare known and novel coding region variants via whole exome sequencing (WES). These data are publicly-available to enable genotype-specific sample requests and the study of novel genotype:phenotype associations, aiding in the mission of nPOD to enhance understanding of diabetes pathogenesis to promote the development of novel therapies.

59 BASIC BIOLOGICAL SCIENCES↗

DOE OSTI2022

Techno-economic analysis of the use of atomic layer deposited transition metal oxides in silicon heterojunction solar cells

The industry for producing silicon solar cells and modules has grown remarkably over the past decades, with more than a 100-fold reduction in price over the past 45?years. The main solar cell fabrication technology has shifted over that time and is currently dominated by the passivated emitter and rear cell (PERC). Other technologies are expected to increase in market share, including tunnel-oxide passivated contact (TOPCon) and heterojunction technology (HJT). In this paper, we examine the cost potential for using atomic layer deposition (ALD) to form transition metal oxide (TMO) layers (MoO x , TiO x and aluminium-doped zinc oxide [AZO]) to use as lower cost alternatives of the p-doped, n-doped and indium tin oxide (ITO) layers, respectively, the layers normally used in HJT solar cells. Using a bottom-up cost and uncertainty model with equipment cost data and process experience in the lab, we find that the production cost of these variations will likely be lower per wafer than standard HJT, with the main cost drivers being the cost of the ALD precursors at high-volume production. We then considered what efficiency is required for these sequences to be cost effective in $/W and discuss whether these targets are technically feasible. This work motivates further work in developing these ALD TMO processes to increase their efficiency towards their theoretical limits to take advantage of the processing cost advantage.

14 SOLAR ENERGY↗

DOE OSTI2023

Characterizing the performance of node-aware strategies for irregular point-to-point communication on heterogeneous architectures

Supercomputer architectures are trending toward higher computational throughput due to the inclusion of heterogeneous compute nodes. These multi-GPU nodes increase on-node computational efficiency, while also increasing the amount of data to be communicated and the number of potential data flow paths. In this work, we characterize the performance of irregular point-to-point communication with MPI on heterogeneous compute environments through performance modeling, demonstrating the limitations of standard communication strategies for both device-aware and staging-through-host communication techniques. Presented models suggest staging communicated data through host processes then using node-aware communication strategies for high inter-node message counts. Notably, the models also predict that node-aware communication utilizing all available CPU cores to communicate inter-node data leads to the most performant strategy when communicating with a high number of nodes. Furthermore, model validation is provided via a case study of irregular point-to-point communication patterns in distributed sparse matrix–vector products. Importantly, we include a discussion on the implications model predictions have on communication strategy design for emerging supercomputer architectures.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2024

Comparing Sensor Fusion and Multimodal Chemometric Models for Monitoring U(VI) in Complex Environments Representative of Irradiated Nuclear Fuel

Optical sensors and chemometric models were leveraged for the quantification of uranium(VI) (0–100 μg mL –1 ), europium (0–150 μg mL –1 ), samarium (0–250 μg mL –1 ), praseodymium (0–350 μg mL –1 ), neodymium (0–1000 μg mL –1 ), and HNO 3 (2–4 M) with varying corrosion product (iron, nickel, and chromium) levels using laser fluorescence, Raman scattering, and ultraviolet–visible–near-infrared absorption spectra. In this paper, an efficient approach to developing and evaluating tens of thousands of partial least-squares regression (PLSR) models, built from fused optical spectra or multimodal acquisitions, is discussed. Each PLSR model was optimized with unique preprocessing combinations, and features were selected using genetic algorithm filters. The 7-factor D-optimal design training set contained just 55 samples to minimize the number of samples. The performance of PLSR models was evaluated by using an automated latent variable selection script. PLS1 regression models tailored to each species outperformed a global PLS2 model. PLS1 models built using fused spectra data and a multimodal (i.e., analyzed separately) approach yielded similar information, resulting in percent root-mean-square error of prediction values of 0.9–5.7% for the seven factors. Further, the optical techniques and data processing strategies established in this study allow for the direct analysis of numerous species without measuring luminescence lifetimes or relying on a standard addition approach, making it optimal for near-real-time, in situ measurements. Nuclear reactor modeling helped bound training set conditions and identified elemental ratios of lanthanide fission products to characterize the burnup of irradiated nuclear fuel. Leveraging fluorescence, spectrophotometry, experimental design, and chemometrics can enable the remote quantification and characterization of complex systems with numerous species, monitor system performance, help identify the source of materials, and enable rapid high-throughput experiments in a variety of industrial processes and fundamental studies.

11 NUCLEAR FUEL CYCLE AND FUEL MATERIALS↗

DOE OSTI2023

Towards a Comparative Assessment of Data-Driven Process Models in Health Information Technology

Process mining for conformance analysis focuses on comparing a reference process model against a data-driven process model that is generated via log files from information technology systems. While this approach is helpful when there is an existing process model in an organization, it leaves the question of what to do in the absence of a complete reference process model unanswered. In this paper, we present a comparative assessment approach that combines process mining, process mapping for dimensionality reduction, and statistical analysis. Our goal is to find similarities and dissimilarities in data-driven process models among U.S. Veterans Health Administration (VHA) facilities to assess process conformance among different healthcare facilities, which can help assess the standardization of care. We illustrate our approach by applying it to two clinical radiology order process models generated by two similar facilities. Our results demonstrate statistical similarities in the standardization of care among those two facilities.

Klasky, Hilda↗

DOE OSTI2024

AI Data Readiness Inspector (AIDRIN) for Quantitative Assessment of Data Readiness for AI

None

Hiniduma, Kaveen↗

DOE OSTI2024

PV Performance Modeling and Stakeholder Engagement (Final Technical Report)

This core capability project’s objective is to increase the value of photovoltaic (PV) performance models by improving their functionality, demonstrating, and quantifying their validity, and offering a wide range of stakeholder engagement opportunities. In FY22-24, we developed new and improved modeling algorithms and functions to represent PV performance more accurately in a variety of environments and conditions. The “Model parameter toolkit” was developed and includes functions to translate between different module temperature models, incidence angle modifier models, and single-diode models. A new modeling capability named “PV Atlas” was also developed leveraging Sandia’s High Performance Computing resources. This capability allows us to investigate several questions and provide climate-specific best practices and geographic data files; all these are hosted on an interactive website on Sandia’s GitHub and can be used for training, system optimization, or to provide best practices for uncertainty reduction. For model validation, we published high-quality PV performance, and weather data; these data are well documented, filtered, and processed for quality and include examples on how to run PV simulations. We also developed well documented, standardized methods for validating PV models and ran independent model validation and 2 blind modeling intercomparisons engaging with 49 organizations from 17 countries. We co-led and contributed to a growing, well documented and maintained suite of open-source functions for PV modeling (i.e., the pvlib-python) and we outreached to the PV modeling stakeholders via the PVPMC workshops and web resources. In addition, this project supported US representation and leadership for the International Energy Agency (IEA) PVPS Task 13; specifically, members of our team led and supported 3 subtasks on: 1) Best practices for the optimization of bifacial photovoltaic tracking, 2) Extreme weather events and their multiple impact on PV power plants: Risks, failure mechanisms and mitigation strategies, and 3) Best practice guidelines for the use of economic and technical Key Performance Indicators (KPIs). This project resulted in the publications of 14 peer reviewed journal papers, 37 conference presentations, 6 SAND reports, 5 public datasets and 6 new webpages on the PVPMC website. It supported the release of 13 pvlib-python versions where 28 enhancements were from this PV Performance Modeling project. We co-organized 5 PVPMC workshops in FY22-24 with the participation of 214 unique institutions and around 700 participants. The PVPMC website was redesigned, and its reliability was improved; it receives over 50,000 visitors/year from 202 unique countries.

14 SOLAR ENERGY↗

DOE OSTI2026

Coupling of high-resolution mass spectrometer and photosynthesis system for comprehensive leaf volatile metabolite profiling

Background Leaf-level biogenic volatile organic compounds (BVOCs) emissions represent a major source of organic gases in the atmosphere, influencing both climate and air quality. These emissions are strongly driven by environmental perturbations, which affect individual plant- to ecosystem-level processes. Uncovering all the BVOCs and understanding how their emissions respond to altered environmental conditions provide critical insights into vegetation-driven changes in atmospheric chemistry. We developed a tandem instrumentation setup that integrates a proton transfer reaction time-of-flight mass spectrometer (PTR-ToF-MS) with parts-per-trillion detection limits and a photosynthetic infrared gas exchange system for the untargeted survey of all the BVOCs. This novel system enables simultaneous, real-time monitoring of BVOC emissions and photosynthetic parameters at the leaf level, offering new opportunities to disentangle the physiological and environmental drivers of VOC release. Furthermore, we established the VOC Analysis and Processing Optimization Resource (VAPOR), an open-access software tool designed for rapid data post-processing and the analysis of the variability of hundreds of BVOCs. We assessed the performance of the tandem system under varying background conditions, using standard gas mixtures and a range of environmental factors. Results Blank emissions were substantially lower for major BVOCs (e.g., isoprene) compared to those observed in plant emissions. Despite this, the observation of background-level VOCs highlights the importance of routinely acquiring and accounting for blank measurements in analyses using the coupled instrumentation. Introduction of known VOC concentrations to the system demonstrated a linear response across different compounds with varying molecular compositions, indicating minimal gas loss regardless of chemical moieties within the coupled instrumentation. We applied the optimized system to investigate the physiological mechanisms driving BVOC emissions across different genotypes of poplar and pennycress. The high mass resolution capabilities of the PTR-ToF-MS, coupled with comprehensive VAPOR-driven data analysis, enabled the identification of several important BVOCs, including methanol and methanethiol; these BVOCs displayed substantial variation across pennycress genotypes and showed concentrations ~ 100–350% higher than the blank. Moreover, isoprene emissions varied significantly among poplar genotypes grown in different potting media. Conclusions Tandem instrumentation offers a powerful tool for profiling volatile molecular markers and elucidating their genetic and environmental underpinnings. This approach enhances our ability to predict BVOC emissions in response to genotype by environmental interactions and contributes to a deeper understanding of vegetation responses to environmental changes.

Biogenic volatile organic compounds↗