Results for “unsupervised learning” — Page 19

DOE OSTI2025

Unsupervised Clustering and Supervised Regression Learning to Select High Temperature Oxidation-Resistant Materials

High temperature oxidation and corrosion degradation mechanisms dictate the lifetime of materials critical to energy production. The combination of modeling and experimental approaches such as machine learning (ML) and data analytics, with sufficient experimental data, can accelerate the development of new materials while limiting its cost. In the present work, ML will be applied to two high temperature oxidation data libraries (Oak Ridge National Laboratory and National Air and Space Administration) that comprised of about 5000 mass change sample datasheets for a variety of materials and temperatures in dry air and air + 10 % H2O. A python code was developed to prepare the data for machine learning by collecting and formatting oxidation rate constants, alloy compositions and environment of exposure into a single data frame. Scikit-learn library and Statistics and Machine Learning Toolbox within MathWorks were then used to perform unsupervised clustering and supervised regression learning. The impact of dataset distribution on the performance of the developed ML models was evaluated. Potential strategies to improve the predictions and enhance extrapolative capability of the previously trained model were investigated.

Romedenne, Marie [ORNL] (ORCID:0000000317936561)↗

DOE OSTI2025

From clutter to clarity: Emergent neural operators via questionnaire metrics

Real-world datasets in chemical engineering and bioengineering processes—such as those from catalytic reactors, multiphase flows, polymerization reactors, bioreactors, and clinical trials—can often be unlabeled or disorganized, rendering the training of existing supervised learning models ineffective at learning the underlying dynamics. To salvage these datasets for decision-making, we first seek to obtain clarity from the cluttered data. Here, we present a framework for developing “structural” generative models, discovering emergent equations, and constructing efficient emulators from scrambled datasets by integrating unsupervised organizational learning techniques (Questionnaires) with advanced deep learning architectures (Deep Hidden Physics Models and Deep Operator Networks). Our approach is demonstrated on two illustrative model systems: (a) a 1D advection–diffusion partial differential equation representing a winding underground pipe and (b) an ensemble of Stuart–Landau oscillators, an agent-based system of coupled ordinary differential equations. In both cases, we successfully reconstruct meaningful spatial, temporal, and parameter embeddings from scrambled data, enabling good predictions of system dynamics. As a result, we highlight the framework’s potential for broader applications, enabling data-driven system identification in fields with inherently disorganized or hidden parameter spaces.

42 ENGINEERING↗

DOE OSTI2019

Large Scale Unsupervised Domain Adaptation of Segmentation Networks with Adversarial Learning

Most current state-of-the-art methods for semantic segmentation on remote sensing imagery require large labeled data, which is scarcely available. Due to the distribution shifting phenomenon inherent in remote sensing imagery, the reuse of pre-trained models on new areas of interest rarely yield satisfactory results. In this paper, we approach this problem from an adversarial learning perspective toward unsupervised domain adaptation. The core concept is to infuse fully convolutional neural networks and adversarial networks for semantic segmentation assuming the structures in the scene and objects of interest are similar in two set of images. Models are trained on a source dataset where ground truth is available and adapted to new target dataset iteratively via a adversarial loss on unlabeled samples. We use two real large scale datasets to validate the framework: 1) cross city road extraction and 2) cross country building extraction. The preliminary results show the usefulness of considering adversarial learning for indirect re-use of the pre-trained models. Experimental validation suggests significant benefits over models without adaptation.

Deng, Xueqing↗

DOE OSTI2024

Identifying Climate Patterns Using Clustering Autoencoder Techniques

Abstract The complexity of growing spatiotemporal resolution of climate simulations produces a variety of climate patterns under different projection scenarios. This paper proposes a new data-driven climate classification workflow via an unsupervised deep learning technique that can dimensionally reduce the vast volume of spatiotemporal numerical climate projection data into a compact representation. We aim to identify distinct zones that capture multiple climate variables as well as their future changes under different climate change scenarios. Our approach leverages convolutional autoencoders combined with k -means clustering (standard autoencoder) and online clustering based on the Sinkhorn–Knopp algorithm (clustering autoencoder) across the conterminous United States (CONUS) to capture unique climate patterns in a data-driven fashion from the Geophysical Fluid Dynamics Laboratory Earth System Model with GOLD component (GFDL-ESM2G). The developed approach compresses 70 years of GFDL-ESM2G simulation at 0.125° spatial resolution across the CONUS under multiple warming scenarios to a lower-dimensional space by a factor of 660 000 and then tested on 150 years of GFDL-ESM2G simulation data. The results show that five climate clusters capture physically reasonable and spatially stable climatological patterns matched to known climate classes defined by human experts. Results also show that using a clustering autoencoder can reduce the computational time for clustering by up to 9.2 times when compared to using a standard autoencoder. Our five unique climate patterns resulting from the deep learning–based clustering of the lower-dimensional space thereby enable us to provide insights on hydrometeorology and its spatial heterogeneity across the conterminous United States immediately without downloading large climate datasets. Significance Statement This paper presents a data-driven climate classification approach using unsupervised deep learning to dimensionally reduce climate model outputs and to identify distinct climate regions for their future changes. Our approach compresses climate information for 70 years of Geophysical Fluid Dynamics Laboratory Earth System Model data across the conterminous United States (CONUS) at 0.125° spatial resolution. The results reveal that five climate clusters capture reasonable and stable climatological patterns matched to known climate patterns. The embedded clustering process in deep learning provides ×9.2 times faster execution than the k -means clustering technique. These results give us insight about climate spatial patterns and heterogeneity of hydrological patterns across the conterminous United States without downloading large climate datasets.

Kurihana, Takuya↗

NASA NTRSDate not supplied

Using Machine Learning to Infer Material Properties of Debris Fragments from X-ray Images in the DebriSat Project

The DebriSat project is a collaboration effort with the NASA Orbital Debris Program Office, the U.S. Space Force Space Systems Command Center, The Aerospace Corporation, and the University of Florida. To date, over 200,000 fragments from this ground-based, hypervelocity impact experiment have been collected, and processing is underway to determine their physical characteristics, such as material, shape, color, characteristic length, and average cross-sectional area. The x-ray process is primarily used to identify the location of the fragments and estimated size for extraction, so that these physical characteristics can be assessed. This paper proposes a machine learning-based approach to characterize materials from x-ray images of debris fragments embedded in soft-catch foam used in the DebriSat project. The novel methodology discussed in this paper will highlight the use of x-ray imagery data to characterize these fragments without extraction or a human-in-the-loop. Both supervised and unsupervised machine learning techniques are utilized with this approach to infer the physical parameters of the fragments embedded in the soft-catch foam panels used in the impact experiment based on x-ray images of the foam panels. Additionally, 3D reconstructions of the extracted fragments are created with images taken from two different angles using the structure from motion (SfM) method. The characteristic lengths and shape from the 3D reconstruction, alongside the physical characteristics of the debris, are used in the inference of the material type. To develop and test the approach, a dataset of x-ray images of debris fragments of varying sizes and materials is collected. Supervised learning methods such as convolutional neural networks (CNNs), support vector machines (SVM), decision trees, and random forest classifiers are used due to the high-dimensional feature spaces of the debris and nonlinear decision boundaries for material categorization. Given the limited pre-labeled data of embedded debris materials smaller than 10 mm, unsupervised machine learning techniques such as clustering algorithms and autoencoders are used, in addition to supervised learning methods. The clustering algorithms group similar fragments together based on their physical properties, and autoencoders reduce the dimensionality of the x ray images and extract relevant features. The performance of the proposed approach's is analyzed using a range of statistical methods, including confusion matrices, receiver operating characteristic curves, and precision-recall curves. The results are compared with those obtained using a baseline approach that relies on manual identification and classification of debris fragments. To evaluate the effectiveness of different machine learning methods, statistical tests such as t-tests, ANOVA, and cross-validation are performed, comparing the performance of CNNs, SVMs, clustering algorithms, and autoencoders. Additional analysis needs to be conducted to identify any sources of bias or variability that may affect the results, such as variations in imaging conditions or fragmentation patterns. Other topics explored are limitations, refinements, and the potential use of semi-supervised learning techniques, such as self-training to label unlabeled datasets and co-training using x-ray images taken from two different angles as two different models.

Saik Anam Siam↗

NASA NTRS2023

Using Machine Learning to Infer Material Properties of Debris Fragments from X-ray Images in the DebriSat Project

The DebriSat project is a collaboration effort with the NASA Orbital Debris Program Office, the U.S. Space Force Space Systems Command Center, The Aerospace Corporation, and the University of Florida. To date, over 200,000 fragments from this ground-based, hypervelocity impact experiment have been collected, and processing is underway to determine their physical characteristics, such as material, shape, color, characteristic length, and average cross-sectional area. The x-ray process is primarily used to identify the location of the fragments and estimated size for extraction, so that these physical characteristics can be assessed. This paper proposes a machine learning-based approach to characterize materials from x-ray images of debris fragments embedded in soft-catch foam used in the DebriSat project. The novel methodology discussed in this paper will highlight the use of x-ray imagery data to characterize these fragments without extraction or a human-in-the-loop. Both supervised and unsupervised machine learning techniques are utilized with this approach to infer the physical parameters of the fragments embedded in the soft-catch foam panels used in the impact experiment based on x-ray images of the foam panels. Additionally, 3D reconstructions of the extracted fragments are created with images taken from two different angles using the structure from motion (SfM) method. The characteristic lengths and shape from the 3D reconstruction, alongside the physical characteristics of the debris, are used in the inference of the material type. To develop and test the approach, a dataset of x-ray images of debris fragments of varying sizes and materials is collected. Supervised learning methods such as convolutional neural networks (CNNs), support vector machines (SVM), decision trees, and random forest classifiers are used due to the high-dimensional feature spaces of the debris and nonlinear decision boundaries for material categorization. Given the limited pre-labeled data of embedded debris materials smaller than 10 mm, unsupervised machine learning techniques such as clustering algorithms and autoencoders are used, in addition to supervised learning methods. The clustering algorithms group similar fragments together based on their physical properties, and autoencoders reduce the dimensionality of the x ray images and extract relevant features. The performance of the proposed approach's is analyzed using a range of statistical methods, including confusion matrices, receiver operating characteristic curves, and precision-recall curves. The results are compared with those obtained using a baseline approach that relies on manual identification and classification of debris fragments. To evaluate the effectiveness of different machine learning methods, statistical tests such as t-tests, ANOVA, and cross-validation are performed, comparing the performance of CNNs, SVMs, clustering algorithms, and autoencoders. Additional analysis needs to be conducted to identify any sources of bias or variability that may affect the results, such as variations in imaging conditions or fragmentation patterns. Other topics explored are limitations, refinements, and the potential use of semi-supervised learning techniques, such as self-training to label unlabeled datasets and co-training using x-ray images taken from two different angles as two different models.

Saik Anam Siam↗

DOE OSTI2020

Leading-Order Analysis by Artificial Intelligence [Slides]

The following topics are addressed in this seminar presentation: The author's background; What is leading-order analysis?; What are supervised and unsupervised machine learning and what is artificial intelligence?; The definition of AI; and, Conclusions and outlook.

42 ENGINEERING↗

DOE OSTI2021

High temperature oxidation of corrosion resistant alloys from machine learning

Parabolic rate constants, k p , were collected from published reports and calculated from corrosion product data (sample mass gain or corrosion product thickness) and tabulated for 75 alloys exposed to temperatures between ~800 and 2000 K (~500–1700 °C; 900–3000°F). Data were collected for environments including lab air, ambient and supercritical carbon dioxide, supercritical water, and steam. Materials studied include low- and high-Cr ferritic and austenitic steels, nickel superalloys, and aluminide materials. A combination of Arrhenius analysis, simple linear regression, supervised and unsupervised machine learning methods were used to investigate the relations between composition and oxidation kinetics. The supervised machine learning techniques produced the lowest mean standard errors. The most significant elements controlling oxidation kinetics were Ni, Cr, Al, and Fe, with Mo and Co composition also found to be significant features. The activation energies produced from the machine learning analysis were in the correct distributions for the diffusion constants for the oxide scales expected to dominate in each class.

Materials Science↗

DOE OSTI2021

Fast Grain Mapping with Sub-Nanometer Resolution Using 4D-STEM with Grain Classification by Principal Component Analysis and Non-Negative Matrix Factorization

High-throughput grain mapping with sub-nanometer spatial resolution is demonstrated using scanning nanobeam electron diffraction (also known as 4D scanning transmission electron microscopy, or 4D-STEM) combined with high-speed direct-electron detection. An electron probe size down to 0.5 nm in diameter is used and the sample investigated is a gold–palladium nanoparticle catalyst. Computational analysis of the 4D-STEM data sets is performed using a disk registration algorithm to identify the diffraction peaks followed by feature learning to map the individual grains. Two unsupervised feature learning techniques are compared: principal component analysis (PCA) and non-negative matrix factorization (NNMF). The characteristics of the PCA versus NNMF output are compared and the potential of the 4D-STEM approach for statistical analysis of grain orientations at high spatial resolution is discussed.

47 OTHER INSTRUMENTATION↗

DOE OSTI2023

Automatic Point Cloud Building Envelope Segmentation (AutoCuBES)

The Auto-CuBES algorithm is based on unsupervised machine learning that automatically labels 3D point cloud data and reduces the time spent in manual segmentation. The algorithm can process high-resolution point clouds and generate a wire-frame building envelope model with a small set of calibration parameters. The algorithm inputs a 3D point cloud generated by commonly available surveying equipment and outputs a wire-frame model of the building envelope. Unsupervised machine learning methods were used to identify facades, windows, and doors while minimizing the number of calibration parameters.

Puente, BryanMaldonado↗

DOE OSTI2022

Condition monitoring and anomaly detection in cyber-physical systems

The modern industrial environment is equipping myriads of smart manufacturing machines where the state of each device can be monitored continuously. Such monitoring can help identify possible future failures and develop a cost-effective maintenance plan. However, it is a daunting task to perform early detection with low false positives and negatives from the huge volume of collected data. This requires developing a holistic machine learning framework to address the issues in condition monitoring of high priority components and develop efficient techniques to detect anomalies that can detect and possibly localize the faulty components. This paper presents a comparative analysis of recent machine learning approaches for robust, cost-effective anomaly detection in cyber-physical systems. While detection has been extensively studied, very few researchers have analyzed the localization of the anomalies. We show that supervised learning outperforms unsupervised algorithms. For supervised cases, we achieve near-perfect accuracy of 98% (specifically for tree-based algorithms). In contrast, the best-case accuracy in the unsupervised cases was 63%—the area under the receiver operating characteristic curve (AUC) exhibits similar outcomes as an additional metric.

Marfo, William↗

DOE OSTI2026

OmicsMLMentor: A Web Application for Guided Machine Learning Analysis of Omics Data

Expression-based omics technologies (e.g. proteomics, metabolomics, transcriptomics, etc.) increasingly rely on supervised and unsupervised machine learning (ML) models to find key biomolecules distinguishing conditions, identify natural groupings in biological data, or generate predictions for outcomes of interest. Fitting ML models to omics data presents several challenges, including handling missing data, selecting a normalization method, choosing a valid model, and optimizing hyperparameters, all requiring statistical programming skills to address these challenges. Thus, the open-source web application SLOPE was designed to lower the barrier to ML modeling for omics data. SLOPE supports the fitting of 15 ML models (10 supervised and 5 unsupervised) tailored to omics datasets, such as proteomics, metabolomics, lipidomics, and transcriptomics. SLOPE offers several omics-specific features, including methods for handling missingness (imputation, conversion, removal), normalization tests, ranking of models based on the structure of a user’s data and user input, and optimal hyperparameter selections using cross-validation splits. By streamlining ML workflows for omics analysis, SLOPE address critical gaps in existing online web tools, facilitating a broader adoption of these models for omics research. Here, SLOPE is applied to data from a lignin exposure study to highlight the workflow for fitting both supervised and unsupervised models to data.

lipidomics↗

NASA NTRSDate not supplied

Virtual Sensing with Unsupervised Image-to-Image Translation

Earth observing satellites carrying multi-spectral sensors are widely used to monitor the physical and biological states of the atmosphere, land, and oceans. These satellites have different vantage points above the earth and different spectral imaging bands resulting in inconsistent imagery from one to another. This presents challenges in building downstream applications. What if we could generate synthetic bands for existing satellites from the union of all domains? We tackle the problem of generating synthetic spectral imagery for multispectral sensors as an unsupervised image-to-image translation problem with partial labels and introduce a novel shared spectral reconstruction loss. Simulated experiments performed by dropping one or more spectral bands show that cross-domain reconstruction outperforms measurements obtained from a second vantage point. On a downstream cloud detection task, we show that generating synthetic bands with our model improves segmentation performance beyond our baseline. Our proposed approach enables synchronization of multispectral data and provides a basis for more homogeneous remote sensing datasets.

Geostationary satellites↗

NASA NTRS2021

Spectral Synthesis for Geostationary Satellite-to-Satellite Translation

Earth-observing satellites carrying multispectral sensors are widely used to monitor the physical and biological states of the atmosphere, land, and oceans. These satellites have different vantage points above the Earth and different spectral imaging bands resulting in inconsistent imagery from one to another. This presents challenges in building downstream applications. What if we could generate synthetic bands for existing satellites from the union of all domains? We tackle the problem of generating synthetic spectral imagery for multispectral sensors as an unsupervised image-to-image translation problem modeled with a variational autoencoder (VAE) and generative adversarial network (GAN) architecture. Our approach introduces a novel shared spectral reconstruction loss to constrain the high-dimensional feature space of multispectral images. Simulated experiments performed by dropping one or more spectral bands show that cross-domain reconstruction outperforms measurements obtained from a second vantage point. Our proposed approach enables the synchronization of multispectral data and provides a basis for more homogeneous remote sensing datasets.

Geostationary satellites↗

DOE OSTI2022

Source-agnostic gravitational-wave detection with recurrent autoencoders

Abstract We present an application of anomaly detection techniques based on deep recurrent autoencoders (AEs) to the problem of detecting gravitational wave (GW) signals in laser interferometers. Trained on noise data, this class of algorithms could detect signals using an unsupervised strategy, i.e. without targeting a specific kind of source. We develop a custom architecture to analyze the data from two interferometers. We compare the obtained performance to that obtained with other AE architectures and with a convolutional classifier. The unsupervised nature of the proposed strategy comes with a cost in terms of accuracy, when compared to more traditional supervised techniques. On the other hand, there is a qualitative gain in generalizing the experimental sensitivity beyond the ensemble of pre-computed signal templates. The recurrent AE outperforms other AEs based on different architectures. The class of recurrent AEs presented in this paper could complement the search strategy employed for GW detection and extend the discovery reach of the ongoing detection campaigns.

47 OTHER INSTRUMENTATION↗

DOE OSTI2022

Learning Stochastic Parametric Differentiable Predictive Control Policies

We present a scalable unsupervised learning-based method for obtaining explicit control policies for model predictive control problems for stochastic linear systems with additive uncertainties subject to nonlinear chance constraints. We call the proposed method stochastic parametric differentiable predictive control (SP-DPC), which extends the recently proposed deterministic DPC policy optimization algorithm. We formulate the SP-DPC as a deterministic approximation to the stochastic parametric constrained optimal control problem via independent sampling of the problem's parameters and uncertainties. This formulation allows us to directly compute the policy gradients via automatic differentiation of the problem's value function, evaluated over sampled parameters and uncertainties. In particular, the computed expectation of the problem's value function is backpropagated through the finite-time closed-loop system rollouts parametrized by a known nominal system dynamics model and neural control policy. We also provide theoretical probabilistic guarantees on closed-loop stability and chance constraints satisfaction for systems controlled by learned neural policies. We demonstrate the computational efficiency and scalability of the proposed policy optimization algorithm in three numerical examples, including systems with a large number of states or subject to nonlinear constraints.

Drgona, Jan↗

DOE OSTI2025

Unsupervised Process Anomaly Detection and Identification Using the Leave-One-Variable-Out Approach

Automated anomaly detection and identification can signal equipment issues and pinpoint causes in large-scale industrial systems. For systems with limited failure history, unsupervised machine learning methods can be utilized as they do not require past failures. This study introduces the leave-one-variable-out (LOVO) model, which masks one variable at a time to predict the others, learning underlying process correlations. Detection performance was assessed with synthetic and experimental data, while identification performance used only synthetic data due to its ability to generate labeled anomaly types. For detection using synthetic data, the LOVO model generally outperformed comparative models; while using experimental data, the comparative methods outperformed the LOVO model. However, the comparative methods required selecting a latent size, and these conclusions pertain to using the optimal size. In practice, it would not be feasible to always select the optimal value, and incorrect selections impacted performance. In contrast, the LOVO model does not require a latent space. For identification using synthetic data, the LOVO model was slightly outperformed in interpretability and repeatability but still demonstrated impressive results. These outcomes suggest that the LOVO model is an effective model and may be more easily implemented without the challenging tuning process of selecting a latent size.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗

DOE OSTI2019

Nonnegative tensor decomposition with custom clustering for microphase separation of block copolymers

High–dimensional datasets are becoming ubiquitous in many applications and therefore unsupervised tensor methods to interrogate them are needed. Here, we report a new unsupervised machine learning (ML) approach (NTFk) based on nonnegative tensor factorization integrated with a custom k–means clustering. We demonstrate the ability of NTFk to extracting temporal and spatial features of phase separation of copolymers as they are modeled by self–consistent field theory. Microphase separation of block copolymers has been extensively studied both experimentally and theoretically. However, the interpretation of computer simulations and/or experimental data, representing temporal and spatial changes of molecular species concentration is still a challenging task. Thus, extracting the phase diagram from simulations or experimental data as well as the interpretation of data requires discernment of the model/experimental parameters (such as, temperature, concentrations, the number of molecular species and the interaction between species) impact on the microphase separation process. An attractive and unique aspect of the introduced ML method is that it ensures the nonnegativity of the extracted latent features. Nonnegativity is an essential constraint needed to obtain interpretable and sparse latent features that are parts–based representation of the data. The custom clustering in NTFk serves to estimate the number of latent features in the data.

97 MATHEMATICS AND COMPUTING↗