Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “supervised”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 37 records · Page 2
All sourcesNASA NTRSDOE OSTI
DOE OSTI2025

Constrained GAN-Generated X-Ray CT Data For Self-Supervised And Foundation-Model Segmentation Of Concrete Microstructures

Three-dimensional characterization of materials using X-ray computed tomography (XCT) is challenging due to the complexity of internal structures, noise, and variations in resolution. Traditional computer vision models often struggle to accurately segment these images, particularly in domain-specific applications like materials science. While supervised deep learning approaches have been developed to address the limitations of conventional algorithms, they typically require large amounts of labeled training data and often fail to generalize across different datasets. Self-supervised, few-and zero-shot learning methods have gained prominence in natural image processing and segmentation tasks, but their application to scientific imaging remains limited due to the unique structural complexity, noise, and textural artifacts present in materials science data. In this work, we investigate how domain adaptation, leveraging physics-based and GAN-generated synthetic data, impacts segmentation performance. We introduce a modified Contrastive Unpaired Translation (CUT) model designed to generate realistic labeled data, which can be used for training, pre-training, and fine-tuning segmentation models for real XCT microstructure data. We evaluate the performance of two segmentation approaches: a self-supervised network (SSL-ALPNet) and a foundation model (Segment Anything Model), assessing their improvements when pre-trained and/or fine-tuned on the synthesized data. Our results demonstrate that leveraging synthetic data significantly enhances segmentation performance, particularly in challenging materials science applications.

Ziabari, Amir [ORNL] (ORCID:000000034776457X)↗
DOE OSTI2025

DeepAndes: A Self-Supervised Vision Foundation Model for Multispectral Remote Sensing Imagery of the Andes

By mapping sites at large scales usingremotely sensed data, archaeologists can generate unique insights into long-term demographic trends, interregional social networks, and human adaptations in the past. Remote sensing surveys complement field-based approaches, and their reach can be especially great when combined with deep learning and computer vision techniques. However, conventional supervised deep learning methods face challenges in annotating fine-grained archaeological features at scale. In addition, while recent vision foundation models have shown remarkable success in learning large-scale remote sensing data with minimal annotations, most off-the-shelf solutions are designed for RGB images rather than multispectral satellite imagery, such as the eight-band data used in our study. In this article, we introduce DeepAndes, a transformer-based vision foundation model trained on three million multispectral satellite images, specifically tailored for Andean archaeology. DeepAndes incorporates a customized DINOv2 self-supervised learning algorithm optimized for eight-band multispectral imagery, marking the first foundation model designed explicitly for the Andes region. We evaluate its image understanding performance through imbalanced image classification, image instance retrieval, and pixel-level semantic segmentation tasks. Our experiments show that DeepAndes achieves superior F1 scores, mean average precision, and Dice scores in few-shot learning scenarios, significantly outperforming models trained from scratch or pretrained on smaller datasets. This underscores the effectiveness of large-scale self-supervised pretraining in archaeological remote sensing.

Guo, Junlin [Vanderbilt Univ., Nashville, TN (Unit↗
DOE OSTI2021

System and method for structural characterization of materials by supervised machine learning-based analysis of their spectra

A method of supervised machine learning-based spectrum analysis information, using a neural network trained with spectrum information, to identify a specified feature of a given material, a system for supervised machine learning-based spectrum analysis, and a method of training a neural network to analyze spectrum data. The method of supervised machine learning-base spectrum analysis comprises inputting into the neural network spectrum data obtained from a sample of the given material; and the neural network processing the spectrum data, in accordance with the training of the neural network, and outputting one or more values for the specified feature of the sample of the material. In an embodiment, the training set of data includes x-ray absorption spectroscopy data for the given material. In an embodiment, the training set of data includes electron energy loss spectra (EELS) data.

Frenkel, Anatoly↗
DOE OSTI2022

Supervised Learning for Distribution Secondary Systems Modeling: Improving Solar Interconnection Processes

The current interconnection process and hosting capacity analysis for distributed energy resources (DERs), such as photovoltaics (PV) and battery energy storage systems, are based on analyzing grid network constraints (voltage and thermal) using only medium-voltage distribution network models. This is because most utilities do not have secondary low-voltage system models that connect service transformers and residential customers. This is important because in many cases the main impact of interconnecting DERs could occur on the low-voltage distribution systems. This paper proposes a supervised learning method to approximate local secondary models to improve the interconnection process. The proposed supervised learning method includes a decision tree model that predicts the secondary topology and a logistic regression model that predicts conductor types. The case studies demonstrate the benefits of including secondary low-voltage circuits in the interconnection process. We report the proposed modeling methodology is readily scalable and thus can reduce the cost and effort of PV interconnection for the industry and stakeholders.

14 SOLAR ENERGY↗
DOE OSTI2025

Semi-supervised permutation invariant particle-level anomaly detection

The development of analysis methods to distinguish potential beyond the Standard Model phenomena in a model-agnostic way can significantly enhance the discovery reach in collider experiments. However, the typical machine learning (ML) algorithms employed for this task require fixed length and ordered inputs that break the natural permutation invariance in collision events. To address this, a semi-supervised anomaly detection tool is presented that takes a variable number of particle-level inputs and leverages a signal model to encode this information into a permutation invariant, event-level representation via supervised training with a Particle Flow Network (PFN). Data events are then encoded into this representation and given as input to an autoencoder for unsupervised ANomaly deTEction on particLe flOw latent sPacE (ANTELOPE), classifying anomalous events based on a low-level and permutation invariant input modeling. Performance of the ANTELOPE architecture is evaluated on simulated samples of hadronic processes in a high energy collider experiment, showing good capability to distinguish disparate models of new physics.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗
DOE OSTI2026

Sim-to-real supervised domain adaptation for radioisotope identification

Machine learning has the potential to improve the speed and reliability of radioisotope identification using gamma spectroscopy. However, meticulously labeling an experimental dataset for training is often prohibitively expensive, while training models purely on synthetic data is risky due to the domain gap between simulated and experimental measurements. In this research, we demonstrate that supervised domain adaptation can substantially improve the performance of radioisotope identification models by transferring knowledge between synthetic and experimental data domains. We consider two domain adaptation scenarios: (1) a simulation-to-simulation adaptation, where we perform multi-label proportion estimation using simulated high-purity germanium detectors, and (2) a simulation-to-experimental adaptation, where we perform multi-class, single-label classification using measured spectra from handheld lanthanum bromide (LaBr) and sodium iodide (NaI) detectors. We begin by pretraining a spectral classifier on synthetic data using a custom transformer-based neural network. After subsequent fine-tuning on just 64 labeled experimental spectra, we achieve a test accuracy of 96% in the sim-to-real scenario with a LaBr detector, far surpassing a synthetic-only baseline model (75%) and a model trained from scratch (80%) on the same 64 spectra. Furthermore, we demonstrate that domain-adapted models learn more human-interpretable features than experiment-only baseline models. Overall, our results highlight the potential for supervised domain adaptation techniques to bridge the sim-to-real gap in radioisotope identification, enabling the development of accurate and explainable classifiers even in real-world scenarios where access to experimental data is limited.

Lalor, Peter W.↗
DOE OSTI2026

Dense autoencoders, clustering techniques, and semi-supervised learning for HPGe $γ$-spectra

Classifying high-resolution gamma spectra by their isotopic content is an essential task in nuclear forensics and other applications. Traditional analysis methods are often time-intensive, but machine learning (ML) may help analysts quickly process many spectra. Such methods tend to rely on abundant, well-labeled data for training. Historical gamma data exists in various fields but is not uniformly useful for supervised ML due to inconsistent labeling. Here, to address some of these challenges, we present a method to classify and organize unlabeled data from high-purity germanium detectors using an autoencoding neural network (autoencoder). We trained dense autoencoders to compress gamma data into latent representations that enable efficient data characterization. By clustering the encoded spectra or lower-dimensional mappings of them, we identified and removed portions of over-abundant data categories, resulting in a more balanced dataset and improved autoencoder performance. This encoding and clustering pipeline also enabled the organization of spectra into self-consistent categories. Finally, we found that encoded representations showed potential as inputs for semi-supervised learning of nuclide identification (NID) labels, achieving an average F1 score of 0.85 ± 0.03 when mapping encodings to a set of 65 isotope labels.

Autoencoders↗
DOE OSTI2020

Semi-supervised Machine Learning Enables the Robust Detection of Multireference Character at Low Cost

Multireference (MR) diagnostics are common tools for identifying strongly correlated electronic structure that makes single-reference (SR) methods (e.g., density functional theory or DFT) insufficient for accurate property prediction. However, MR diagnostics typically require computationally demanding correlated wave function theory (WFT) calculations, and diagnostics often disagree or fail to predict MR effects on properties. To overcome these challenges, we introduce a semi-supervised machine learning (ML) approach with virtual adversarial training (VAT) of an MR classifier using 15 WFT and DFT MR diagnostics as inputs. In semi-supervised learning, only the most extreme SR or MR points are labeled, and the remaining point labels are learned. The resulting VAT model outperforms the alternatives, as quantified by the distinct property distributions of SR- and MR-classified molecules. Additionally, to reduce the cost of generating inputs to the VAT model, we leverage the VAT model’s robustness to noisy inputs by replacing WFT MR diagnostics with regression predictions in an MR decision engine workflow that preserves excellent performance. We demonstrate the transferability of our approach to larger molecules and those with distinct chemical composition from the training set. This MR decision engine demonstrates promise as a low-cost, high-accuracy approach to the automatic detection of strong correlation for predictive high-throughput screening.

36 MATERIALS SCIENCE↗
DOE OSTI2022

Inferring topological transitions in pattern-forming processes with self-supervised learning

Abstract The identification of transitions in pattern-forming processes are critical to understand and fabricate microstructurally precise materials in many application domains. While supervised methods can be useful to identify transition regimes, they need labels, which require prior knowledge of order parameters or relevant microstructures describing these transitions. Instead, we develop a self-supervised, neural-network-based approach that does not require predefined labels about microstructure classes to predict process parameters from observed microstructures. We show that assessing the difficulty of solving this inverse problem can be used to uncover microstructural transitions. We demonstrate our approach by automatically discovering microstructural transitions in two distinct pattern-forming processes: the spinodal decomposition of a two-phase mixture and the formation of binary-alloy microstructures during physical vapor deposition of thin films. This approach opens a path forward for discovering unseen or hard-to-discern transitions and ultimately controlling complex pattern-forming processes.

Abram, Marcin↗
DOE OSTI2025

Leveraging unlabeled SEM datasets with self-supervised learning for enhanced particle segmentation

Scanning Electron Microscopes (SEMs) are widely used in experimental science laboratories, often requiring cumbersome and repetitive user analysis. Automating SEM image analysis processes is highly desirable to address this challenge. In particle sample analysis, Machine Learning (ML) has emerged as the most effective approach for particle segmentation. However, the time-intensive process of manually annotating thousands of SEM images limits the applicability of supervised learning approaches. Self-Supervised Learning (SSL) offers a promising alternative by enabling knowledge extraction from raw, unlabeled data. This study presents a framework for evaluating SSL techniques in SEM image analysis, focusing on novel methods leveraging the ConvNeXtV2 architecture for particle detection. A dataset comprising 25,000 SEM images is curated to benchmark these proposed SSL methods. The results demonstrate that ConvNeXtV2 models, with varying parameter counts, consistently outperform other techniques in particle detection across different length scales, achieving up to a 34% reduction in relative error compared to established SSL methods. Furthermore, an ablation study explores the relationship between dataset size and SSL performance, providing actionable insights for practitioners regarding model selection and resource efficiency. This research advances the integration of SSL into autonomous analysis pipelines and supports its application in accelerating materials science discovery.

Rettenberger, Luca↗
DOE OSTI2024

CryoTRANS: predicting high-resolution maps of rare conformations from self-supervised trajectories in cryo-EM

Cryogenic electron microscopy (cryo-EM) has revolutionized structural biology, enabling efficient determination of structures at near-atomic resolutions. However, a common challenge arises from the severe imbalance among various conformations of vitrified particles, leading to low-resolution reconstructions in rare conformations due to a lack of particle images in these quasi-stable states. We introduce CryoTRANS, a method that predicts high-resolution maps of rare conformations by constructing a self-supervised pseudo-trajectory between density maps of varying resolutions. This trajectory is represented by an ordinary differential equation parameterized by a deep neural network, ensuring retention of detailed structures from high-resolution density maps. By leveraging a single high-resolution density map, CryoTRANS significantly improves the reconstruction of rare conformations and has been validated on four real-world datasets: alpha-2-macroglobulin, actin-binding protein complexes, SARS-CoV-2 spike glycoprotein, and the 70S ribosome. CryoTRANS can also predict high-resolution structures in cryogenic electron tomography maps using a high-resolution cryo-EM map.Cryogenic electron microscopy (cryo-EM) has revolutionized structural biology, enabling efficient determination of structures at near-atomic resolutions. However, a common challenge arises from the severe imbalance among various conformations of vitrified particles, leading to low-resolution reconstructions in rare conformations due to a lack of particle images in these quasi-stable states. We introduce CryoTRANS, a method that predicts high-resolution maps of rare conformations by constructing a self-supervised pseudo-trajectory between density maps of varying resolutions. This trajectory is represented by an ordinary differential equation parameterized by a deep neural network, ensuring retention of detailed structures from high-resolution density maps. By leveraging a single high-resolution density map, CryoTRANS significantly improves the reconstruction of rare conformations and has been validated on four real-world datasets: alpha-2-macroglobulin, actin-binding protein complexes, SARS-CoV-2 spike glycoprotein, and the 70S ribosome. CryoTRANS can also predict high-resolution structures in cryogenic electron tomography maps using a high-resolution cryo-EM map.

47 OTHER INSTRUMENTATION↗
DOE OSTI2022

Supervised learning and the finite-temperature string method for computing committor functions and reaction rates

A central object in the computational studies of rare events is the committor function. Though costly to compute, the committor function encodes complete mechanistic information of the processes involving rare events, including reaction rates and transition-state ensembles. Under the framework of transition path theory, Rotskoff et al. [Proceedings of the 2nd Mathematical and Scientific Machine Learning Conference, Proceedings of Machine Learning Research (PLMR, 2022), Vol. 145, pp. 757–780] proposes an algorithm where a feedback loop couples a neural network that models the committor function with importance sampling, mainly umbrella sampling, which collects data needed for adaptive training. Here, in this work, we show additional modifications are needed to improve the accuracy of the algorithm. The first modification adds elements of supervised learning, which allows the neural network to improve its prediction by fitting to sample-mean estimates of committor values obtained from short molecular dynamics trajectories. The second modification replaces the committor-based umbrella sampling with the finite-temperature string (FTS) method, which enables homogeneous sampling in regions where transition pathways are located. We test our modifications on low-dimensional systems with non-convex potential energy where reference solutions can be found via analytical or finite element methods, and show how combining supervised learning and the FTS method yields accurate computation of committor functions and reaction rates. We also provide an error analysis for algorithms that use the FTS method, using which reaction rates can be accurately estimated during training with a small number of samples. The methods are then applied to a molecular system in which no reference solution is known, where accurate computations of committor functions and reaction rates can still be obtained.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2022

Effectiveness of supervised exercise, home-based exercise, or walk advice strategies on walking performance and muscle endurance in patients with intermittent claudication (SUNFIT trial): a randomized clinical trial

Abstract Aims Supervised exercise is a guideline-recommended treatment in intermittent claudication (IC). Hospital-based supervised exercise programmes (SEPs) are underutilized, while home-based structured exercise programmes (HSEPs) have attracted interest. The results from HSEP in IC are inconsistent and may confer no benefit over walk advice (WA) and be less effective than SEP. The aim of the study was to compare the effectiveness of best medical treatment, including Nordic pole WA alone, or WA + SEP or WA + HSEP for patients with IC. Methods and results This three-armed, multicentre randomized clinical trial enrolled patients with IC; all patients received best medical treatment including walking poles and the advice of regular Nordic pole walking (WA). For HSEP and SEP, additional exercise programmes were provided. The primarily investigated hypothesis was a non-inferiority analysis of SEP vs. HSEP regarding the 6-min walk test (6MWT) maximum distance, with a pre-defined non-inferiority margin of 50 m. Supporting outcomes included muscle endurance tests and the walking impairment questionnaire. Outcomes were assessed at baseline, 3, 6, and 12 months by a blinded evaluator. Altogether 166 patients (mean age 72 years; 59% males) were randomized. In HSEP and SEP, 24 and 26% patients, respectively, were fully exercise adherent. All three groups improved pain-free walking distance over time, but there were no significant intergroup differences. The intergroup 6MWT difference between SEP and HSEP from 0 to 12 months was –11.6 m, 95% confidence interval: –36.4 to 13.0 m (i.e. within the pre-specified non-inferiority margin). Conclusion The HSEP was non-inferior to SEP in patients with IC. There were no significant differences observed between the three groups at 1 year. Registration ClinicialTrials.gov: NCT02341716.

Sandberg, Anna (ORCID:0000000258432784)↗
DOE OSTI2022

Practical galaxy morphology tools from deep supervised representation learning

Astronomers have typically set out to solve supervised machine learning problems by creating their own representations from scratch. We show that deep learning models trained to answer every Galaxy Zoo DECaLS question learn meaningful semantic representations of galaxies that are useful for new tasks on which the models were never trained. We exploit these representations to outperform several recent approaches at practical tasks crucial for investigating large galaxy samples. The first task is identifying galaxies of similar morphology to a query galaxy. Given a single galaxy assigned a free text tag by humans (e.g. ‘#diffuse’), we can find galaxies matching that tag for most tags. The second task is identifying the most interesting anomalies to a particular researcher. Our approach is 100 per cent accurate at identifying the most interesting 100 anomalies (as judged by Galaxy Zoo 2 volunteers). The third task is adapting a model to solve a new task using only a small number of newly labelled galaxies. Models fine-tuned from our representation are better able to identify ring galaxies than models fine-tuned from terrestrial images (ImageNet) or trained from scratch. We solve each task with very few new labels; either one (for the similarity search) or several hundred (for anomaly detection or fine-tuning). This challenges the longstanding view that deep supervised methods require new large labelled data sets for practical use in astronomy. To help the community benefit from our pretrained models, we release our fine-tuning code zoobot. Zoobot is accessible to researchers with no prior experience in deep learning.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2023

Data compression and inference in cosmology with self-supervised machine learning

ABSTRACT The influx of massive amounts of data from current and upcoming cosmological surveys necessitates compression schemes that can efficiently summarize the data with minimal loss of information. We introduce a method that leverages the paradigm of self-supervised machine learning in a novel manner to construct representative summaries of massive data sets using simulation-based augmentations. Deploying the method on hydrodynamical cosmological simulations, we show that it can deliver highly informative summaries, which can be used for a variety of downstream tasks, including precise and accurate parameter inference. We demonstrate how this paradigm can be used to construct summary representations that are insensitive to prescribed systematic effects, such as the influence of baryonic physics. Our results indicate that self-supervised machine learning techniques offer a promising new approach for compression of cosmological data as well as its analysis.

Astronomy & Astrophysics↗
DOE OSTI2025

Resimulation-based self-supervised learning for pretraining physics foundation models

Self-supervised learning (SSL) is at the core of training modern large machine learning models, providing a scheme for learning powerful representations that can be used in a variety of downstream tasks. However, SSL strategies must be adapted to the type of training data and downstream tasks required. We propose resimulation-based self-supervised representation learning (RS3L), a novel simulation-based SSL strategy that employs a method of resimulation to drive data augmentation for contrastive learning in the physical sciences, particularly, in fields that rely on stochastic simulators. By intervening in the middle of the simulation process and rerunning simulation components downstream of the intervention, we generate multiple realizations of an event, thus producing a set of augmentations covering all physics-driven variations available in the simulator. Using experiments from high-energy physics, we explore how this strategy may enable the development of a foundation model; we show how RS3L pretraining enables powerful performance in downstream tasks such as discrimination of a variety of objects and uncertainty mitigation. In addition to our results, we make the RS3L dataset publicly available for further studies on how to improve SSL strategies.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2022

Semi-Supervised Disaggregation of Load Profiles at Transmission Buses with Significant Behind-the-Meter Solar Generations

It is of imperative interests for regional transmission organizations (RTOs) to effectively extract daily load profiles at transmission buses, which remains a gap in existing technology paradigm. This digest proposes an explicit yet efficient linear estimator, to disaggregate metered load profiles at buses with significant behind-the-meter (BTM) solar generations in a data driven manner. The proposed estimator is based on utility zonal load profiles and proxy solar irradiance profiles, which in reality is the aggregated waveform at each transmission bus and equivalent to the mix of summed load profiles minus actual BTM solar generation. To overcome technical challenges in the lack of “ground truth” and validate the performance of supervised learning algorithms, we propose semi-supervised mechanisms with parameter tuning, and leverage the unique characteristics of zero-crossing points in BTM solar peaking behaviors.

machine leaning, behind-the-meter, power grid anal↗
DOE OSTI2022

MAD: Self-Supervised Masked Anomaly Detection Task for Multivariate Time Series

In this paper, we introduce Masked Anomaly Detection (MAD), a general self-supervised learning task for multivariate time series anomaly detection. With the increasing availability of sensor data from industrial systems, being able to detecting anomalies from streams of multivariate time series data is of significant importance. Given the scarcity of anomalies in real-world applications, the majority of literature has been focusing on modeling normality. The learned normal representations can empower anomaly detection as the model has learned to capture certain key underlying data regularities. A typical formulation is to learn a predictive model, i.e., use a window of time series data to predict future data values. In this paper, we propose an alternative self-supervised learning task. By randomly masking a portion of the inputs and training a model to estimate them using the remaining ones, MAD is an improvement over the traditional left-to-right next step prediction (NSP) task. Our experimental results demonstrate that MAD can achieve better anomaly detection rates over traditional NSP approaches when using exactly the same neural network (NN) base models, and can be modified to run as fast as NSP models during test time on the same hardware, thus making it an ideal upgrade for many existing NSP-based NN anomaly detection models.

97 MATHEMATICS AND COMPUTING↗