Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “deep transfer learning”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 91 records · Page 5
All sourcesNASA NTRSDOE OSTI
DOE OSTI2023

MACHINE LEARNING-ENABLED PREDICTION OF TRANSIENT INJECTION MAP IN AUTOMOTIVE INJECTORS WITH UNCERTAINTY QUANTIFICATION

Accurate prediction of injection profiles is a critical aspect of linking injector operation with engine performance and emissions. However, highly resolved injector simulations can take one to two weeks of wall-clock time, which is incompatible with engine design cycles with desired turnaround times of less than a day. Hence, it is important to reduce the time-to-solution of the internal flow simulations by several orders of magnitude to make it compatible with engine simulations. This work demonstrates a data-driven approach for tackling the computational overhead of injector simulations, whereby the transient injection profiles are emulated for a side-oriented, single-hole diesel injector using a Bayesian machine-learning framework. First, an interpretable Bayesian learning strategy was employed to understand the effect of design parameters on the total void fraction field. Then, autoencoders are utilized for efficient dimensionality reduction of the flowfields. Gaussian process models are finally used to predict the spatiotemporal void fraction field at the injector exit for unknown operating conditions. The Gaussian process models produce principled uncertainty estimates associated with the emulated flowfields, which provide the engine designer with valuable information of where the data-driven predictions can be trusted in the design space. The Bayesian flowfield predictions are compared with the corresponding predictions from a deep neural network, which has been transfer-learned from static needle simulations from a previous work by the authors. The emulation framework can predict the void fraction field at the exit of the orifice within a few seconds, thus achieving a speed-up factor of up to 38 x 10(6) over the traditional simulation-based approach of generating transient injection maps.

machine learning↗
NASA NTRS2018

GEONEX: Progressive Conditional Generative Adversarial Training Using Transfer learning

Obtaining accurate segmentation on large scale images is an open problem in deep learning. The main problem is the amount of labeled data that exists for large scale images. Traditionally, the common solution to this problem is to crop the large images into smaller images to increase the amount of available data and train a Conditional Generative Adversarial Network (CGAN). CGANs are currently the state of the art in image to image translation and provide better accuracy than the traditional method of training a encoder based conv-net architecture to minimize the loss at each pixel. This method can produce noisy and discontinuous images with inaccurate results. We seek to solve this problem by utilizing the concepts of transfer learning and progressive training to create a CGAN that can segment large scale images with a limited amount of labeled data. In transfer learning we recognize that many learned features are applicable to many classes from multiple domains. This introduces the concept of feature reusability, which is the basis for finetuning. Progressive training got its start in training models on the same images at different resolutions. In this work we instead train a GAN on increasing image scales by transferring the weights from the smaller scales to the larger scales. The learned features at the smaller scales are continually reused and applied to larger scales to create a CGAN that can perform accurate segmentation on large scale images. We apply this method to detect building footprints on very high-resolution overhead imagery (e.g Digital Globe and high resolution airborne platforms).

GEONEX↗
DOE OSTI2023

A transfer learning approach for acoustic emission zonal localization on steel plate-like structure using numerical simulation and unsupervised domain adaptation

The detection and localization of damage in metallic structures using acoustic emission (AE) monitoring and artificial intelligence technology such as deep learning has been widely studied. However, a current challenge of this approach is the difficulty of obtaining sufficient labeled historical AE signals for the training process of deep learning models. This problem can be approached through the implementation of transfer learning. The innovation of this paper lies in the development of a transfer learning approach for AE source localization on a stainless-steel structure when no historical labeled AE signals are available for training. A finite element model is developed to generate numerical AE signals for the training. Unsupervised domain adaptation (UDA) technology is utilized to reduce the distribution difference between the numerical and the realistic AE signals and to derive the localization results of the unlabeled realistic AE signals. Finally, the results suggest that the proposed approach is capable of localizing AE signals with high accuracy in the absence of labeled training data.

42 ENGINEERING↗
DOE OSTI2025

Development and transferability of neural-network models for plasma-surface interactions

Plasma-surface interactions are increasingly critical to modern technologies; yet, accurate molecular dynamics simulations remain limited by the capabilities of interatomic potentials. Deep Potentials (DPs) promise to revolutionize the field by providing a systematic method for producing accurate interatomic potentials. The primary challenge of DP development is selecting a dataset, which efficiently spans the set of atomic environments one expects to encounter in the subsequent molecular dynamics simulations. The computational cost of density functional theory calculations, which are the typical basis for DP development, makes it impossible to directly verify the quality of a given DP. To address this challenge, we explore the development of a deep-learned interatomic potential, “DeepREBO,” trained to reproduce the behavior of the REBO2 empirical potential, enabling direct validation of training methodology and transferability. Using an active learning framework, we begin with a minimal dataset and iteratively expand it to train a Deep Potential-Smooth Edition model that faithfully reproduces REBO2 results for 25 eV hydrogen bombardment of diamond (001), a particularly challenging case. We show that small, carefully curated datasets can outperform large, unguided ones, with effective models requiring fewer than 15 000 snapshots. Subsequent transferability tests demonstrate that while DeepREBO generalizes well to diamond (111) surfaces, performance degrades for amorphous carbon or higher-energy impacts, highlighting the need for use-case-specific training data. We also evaluate methods to improve short-range repulsion. This study outlines best practices for training robust deep potentials and underscores the importance of dataset design for predictive plasma simulations.

Ab-initio molecular dynamics↗
NASA NTRSDate not supplied

Exploring Transfers Between Earth-Moon Halo Orbits via Multi-Objective Reinforcement Learning

Multi-Reward Proximal Policy Optimization, a multi-objective deep reinforcement learning algorithm, is used to examine the design space of low-thrust trajectories for a SmallSat transferring between two libration point orbits in the Earth-Moon system. Using Multi-Reward Proximal Policy Optimization, multiple policies are simultaneously and efficiently trained on three distinct trajectory design scenarios. Each policy is trained to create a unique control scheme based on the trajectory design scenario and assigned reward function: a unique combination of weights scaling competing objectives that guide the spacecraft to the target mission orbit, incentivize faster flight times, and penalize propellant mass usage. Then, the policies are evaluated on the same set of perturbed initial conditions in each scenario to generate the propellant mass usages, flight times, and state discontinuities from a reference trajectory for each control scheme. This solution space of low-thrust trajectories for a SmallSat is used to examine the multi-objective trade space for the trajectory design scenario. By autonomously constructing the solution space, insights into the required propellant mass, flight time, and transfer geometry are rapidly achieved.

Christopher J Sullivan↗
NASA NTRS2021

Exploring Transfers Between Earth-Moon Halo Orbits via Multi-Objective Reinforcement Learning

Multi-Reward Proximal Policy Optimization, a multi-objective deep reinforcement learning algorithm, is used to examine the design space of low-thrust trajectories for a SmallSat transferring between two libration point orbits in the Earth- Moon system. Using Multi-Reward Proximal Policy Optimiza- tion, multiple policies are simultaneously and efficiently trained on three distinct trajectory design scenarios. Each policy is trained to create a unique control scheme based on the trajectory design scenario and assigned reward function: a unique combination of weights scaling competing objectives that guide the spacecraft to the target mission orbit, incentivize faster flight times, and penalize propellant mass usage. Then, the policies are evaluated on the same set of perturbed initial conditions in each scenario to generate the propellant mass usages, flight times, and state discontinuities from a reference trajectory for each control scheme. This solution space of low-thrust trajectories for a SmallSat is used to examine the multi-objective trade space for the trajectory design scenario. By autonomously constructing the solution space, insights into the required propellant mass, flight time, and transfer geometry are rapidly achieved.

Mashiku, Alinda K.↗
DOE OSTI2022

Driver Distraction Behavior Detection using a Vision Transformer Model based on Transfer Learning Strategy

Driver distraction behavior is one of the critical factors in traffic accidents. Thus, advanced driver state detection system has become the focus in the field of intelligent vehicle. However, in practical applications, insufficient samples of driving distraction behaviors bring great challenges to training a personalized behavior distraction detection model for a specific driver. To this end, a novel transformer model based on a transfer learning strategy is proposed in this paper to accurately recognize driver distraction behavior. Inspired by the effect of the transformer network in visual recognition, we firstly present a transformer behavior distraction detection system to identify the behavior categories that cause driver distraction. Then, for the specific driving dataset in practical application scenarios, the transfer learning strategy is introduced into the driver distraction detection model to further train the general transformer network. The effectiveness of the transformer based on the transfer learning strategy is validated compared with other traditional deep learning methods. The results show that the proposed detection method has better generalization ability and higher accuracy.

Fang, Zhenwu↗
DOE OSTI2022

Suppressing simulation bias in multi-modal data using transfer learning

Abstract Many problems in science and engineering require making predictions based on few observations. To build a robust predictive model, these sparse data may need to be augmented with simulated data, especially when the design space is multi-dimensional. Simulations, however, often suffer from an inherent bias. Estimation of this bias may be poorly constrained not only because of data sparsity, but also because traditional predictive models fit only one type of observed outputs, such as scalars or images, instead of all available output data modalities, which might have been acquired and simulated at great cost. To break this limitation and open up the path for multi-modal calibration, we propose to combine a novel, transfer learning technique for suppressing the bias with recent developments in deep learning, which allow building predictive models with multi-modal outputs. First, we train an initial neural network model on simulated data to learn important correlations between different output modalities and between simulation inputs and outputs. Then, the model is partially retrained, or transfer learned, to fit the experiments; a method that has never been implemented in this type of architecture. Using fewer than 10 inertial confinement fusion experiments for training, transfer learning systematically improves the simulation predictions while a simple output calibration, which we design as a baseline, makes the predictions worse. We also offer extensive cross-validation with real and carefully designed synthetic data. The method described in this paper can be applied to a wide range of problems that require transferring knowledge from simulations to the domain of experiments.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗
DOE OSTI2022

Pre-trained network-based transfer learning: A small-sample machine learning approach to nuclear power plant classification problem

Some research topics belonging to classification problems in the nuclear industry, such as fault diagnosis and accident identification, can be solved by feature extraction and subsequent application of statistical machine learning classifiers. Recently, deep neural network-based methods with automatic feature extraction and high accuracy have gained wide attention. They usually require large-scale training data, however, plant fault or accident data are scarce or difficult to obtain. Here this paper proposes a convolutional network (CNN)-based transfer learning method to solve this problem. The network's shallow layer is derived from a pre-trained CNN based on the ImageNet database to automatically extract features, and the deep layer is customized to match the classification problem. Data in non-image formats are converted to image formats and subsequently used to train the network. Case studies of rotating machines fault diagnosis show that the proposed method requires only limited training data to achieve high accuracy.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗
DOE OSTI2024

Quasi-Classical Trajectory Calculation of Rate Constants Using an Ab Initio Trained Machine Learning Model (aML-MD) with Multifidelity Data

Machine learning (ML) provides a great opportunity for the construction of models with improved accuracy in classical molecular dynamics (MD). However, the accuracy of a ML trained model is limited by the quality and quantity of the training data. Generating large sets of accurate ab initio training data can require significant computational resources. Furthermore, inconsistent or incompatible data with different accuracies obtained using different methods may lead to biased or unreliable ML models that do not accurately represent the underlying physics. Recently, transfer learning showed its potential for avoiding these problems as well as for improving the accuracy, efficiency, and generalization of ML models using multifidelity data. In this work, ab initio trained ML-based MD (aML-MD) models are developed through transfer learning using DFT and multireference data from multiple sources with varying accuracy within the Deep Potential MD framework. Further, the accuracy of the force field is demonstrated by calculating rate constants for the H + HO 2 → H 2 + 3 O 2 reaction using quasi-classical trajectories. We show that the aML-MD model with transfer learning can accurately predict the rate constants while reducing the computational cost by more than five times compared to the use of more expensive quantum chemistry training data sets. Hence, the aML-MD model with transfer learning shows great potential in using multifidelity data to reduce the computational cost involved in generating the training set for these potentials.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2022

Detecting damaged buildings using real-time crowdsourced images and transfer learning

After significant earthquakes, we can see images posted on social media platforms by individuals and media agencies owing to the mass usage of smartphones these days. These images can be utilized to provide information about the shaking damage in the earthquake region both to the public and research community, and potentially to guide rescue work. This paper presents an automated way to extract the damaged buildings images after earthquakes from social media platforms such as Twitter and thus identify the particular user posts containing such images. Using transfer learning and ~ 6500 manually labelled images, we trained a deep learning model to recognize images with damaged buildings in the scene. The trained model achieved good performance when tested on newly acquired images of earthquakes at different locations and when ran in near real-time on Twitter feed after the 2020 M7.0 earthquake in Turkey. Furthermore, to better understand how the model makes decisions, we also implemented the Grad-CAM method to visualize the important regions on the images that facilitate the decision.

58 GEOSCIENCES↗
DOE OSTI2025

Applying Deep Learning for Wildfire Identification: Economical and Accessible Solutions Leveraging Small Datasets

Wildfires significantly impact human health, air quality, visibility, weather, and climate change and cause substantial economic losses. While state and county-operated air quality monitors provide critical insights during wildfires, they are not available in all regions. This highlights the need for affordable, accessible tools that allow the general public to assess air quality impacts. In this study, we apply machine learning with deep neural networks to diagnose air quality rapidly from sky images taken at the Pacific Northwest National Laboratory in Richland, WA, USA. Using a convolutional neural network (CNN) framework, we trained a deep learning model to classify air quality indices based on sky images. By leveraging transfer learning, our approach fine-tunes a pre-trained model on a small dataset of sky images, significantly reducing training time while maintaining high accuracy. Our results demonstrate the potential of deep learning to provide rapid air quality diagnostics during wildfire episodes, offering early warnings to the public and enabling timely mitigation strategies, particularly for vulnerable populations. Additionally, we show that lower respiratory infections pose the highest health risk during acute smoke exposures. Reactive oxygen species (ROS) from wildfire particles further exacerbate health risks by triggering inflammation and other adverse effects.

54 ENVIRONMENTAL SCIENCES↗
NASA NTRS2022

Microstructure Segmentation with Deep Learning Encoders Pre-Trained on a Large Microscopy Dataset

This study examined the improvement of microscopy segmentation accuracy by transfer learning from a large dataset of microscopy images called MicroNet. Many neural network encoder architectures, including VGG, Inception, and ResNet, were trained on over 100,000 labelled microscopy images from 54 classes. These pre-trained encoders were then embedded into multiple segmentation architectures including U-Net and DeepLabV3+ to evaluate segmentation performance on newly created benchmark microscopy datasets. Compared to ImageNet pre-training, models pre-trained on MicroNet generalized better to out-of-distribution micrographs taken under different imaging and sample conditions and were more accurate with less training data. When training with only a single Ni-superalloy image, pre-training on MicroNet produced a 72.2 percent reduction in relative segmentation error. These results suggest that transfer learning from large in-domain datasets generate models with learned feature representations that are more useful for downstream tasks and will likely improve any microscopy image analysis technique that can leverage pre-trained encoders.

machine learning↗
DOE OSTI2021

Deeplasmid: deep learning accurately separates plasmids from bacterial chromosomes

Plasmids are mobile genetic elements that play a key role in microbial ecology and evolution by mediating horizontal transfer of important genes, such as antimicrobial resistance genes. Many microbial genomes have been sequenced by short read sequencers and have resulted in a mix of contigs that derive from plasmids or chromosomes. New tools that accurately identify plasmids are needed to elucidate new plasmid-borne genes of high biological importance. We have developed Deeplasmid, a deep learning tool for distinguishing plasmids from bacterial chromosomes based on the DNA sequence and its encoded biological data. It requires as input only assembled sequences generated by any sequencing platform and assembly algorithm and its runtime scales linearly with the number of assembled sequences. Deeplasmid achieves an AUC–ROC of over 89%, and it was more accurate than five other plasmid classification methods. Finally, as a proof of concept, we used Deeplasmid to predict new plasmids in the fish pathogen Yersinia ruckeri ATCC 29473 that has no annotated plasmids. Deeplasmid predicted with high reliability that a long assembled contig is part of a plasmid. Using long read sequencing we indeed validated the existence of a 102 kb long plasmid, demonstrating Deeplasmid's ability to detect novel plasmids.

59 BASIC BIOLOGICAL SCIENCES↗
NASA NTRS2018

Advances in Hyperspectral Image Classification Methods for Vegetation and Agricultural Cropland Studies

Hyperspectral data are becoming more widely available via sensors on airborne and unmanned aerial vehicle (UAV) platforms, as well as proximal platforms. While space-based hyperspectral data continue to be limited in availability, multiple spaceborne Earth-observing missions on traditional platforms are scheduled for launch, and companies are experimenting with small satellites for constellations to observe the Earth, as well as for planetary missions. Land cover mapping via classification is one of the most important applications of hyperspectral remote sensing and will increase in significance as time series of imagery are more readily available. However, while the narrow bands of hyperspectral data provide new opportunities for chemistry-based modeling and mapping, challenges remain. Hyperspectral data are high dimensional, and many bands are highly correlated or irrelevant for a given classification problem. For supervised classification methods, the quantity of training data is typically limited relative to the dimension of the input space. The resulting Hughes phenomenon, often referred to as the curse of dimensionality, increases potential for unstable parameter estimates, overfitting, and poor generalization of classifiers. This is particularly problematic for parametric approaches such as Gaussian maximum likelihood–based classifiers that have been the backbone of pixel-based multispectral classification methods. This issue has motivated investigation of alternatives, including regularization of the class covariance matrices, ensembles of weak classifiers, development of feature selection and extraction methods, adoption of nonparametric classifiers, and exploration of methods to exploit unlabeled samples via semi-supervised and active learning. Data sets are also quite large, motivating computationally efficient algorithms and implementations. This chapter provides an overview of the recent advances in classification methods for mapping vegetation using hyperspectral data. Three data sets that are used in the hyperspectral classification literature (e.g., Botswana Hyperion satellite data and AVIRIS airborne data over both Kennedy Space Center and Indian Pines) are described in Section 3.2 and used to illustrate methods described in the chapter. An additional high-resolution hyperspectral data set acquired by a SpecTIR sensor on an airborne platform over the Indian Pines area is included to exemplify the use of new deep learning approaches, and a multiplatform example of airborne hyperspectral data is provided to demonstrate transfer learning in hyperspectral image classification. Classical approaches for supervised and unsupervised feature selection and extraction are reviewed in Section 3.3. In particular, nonlinearities exhibited in hyperspectral imagery have motivated development of nonlinear feature extraction methods in manifold learning, which are outlined in Section 3.3.1.4. Spatial context is also important in classification of both natural vegetation with complex textural patterns and large agricultural fields with significant local variability within fields. Approaches to exploit spatial features at both the pixel level (e.g., co-occurrence–based texture and extended morphological attribute profiles [EMAPs]) and integration of segmentation approaches (e.g., HSeg) are discussed in this context in Section 3.3.2. Recently, classification methods that leverage nonparametric methods originating in the machine learning community have grown in popularity. An overview of both widely used and newly emerging approaches, including support vector machines (SVMs), Gaussian mixture models, and deep learning based on convolutional neural networks is provided in Section 3.4. Strategies to exploit unlabeled samples, including active learning and metric learning, which combine feature extraction and augmentation of the pool of training samples in an active learning framework, are outlined in Section 3.5. Integration of image segmentation with classification to accommodate spatial coherence typically observed in vegetation is also explored, including as an integrated active learning system. Exploitation of multisensor strategies for augmenting the pool of training samples is investigated via a transfer learning framework in Section 3.5.1.2. Finally, we look to the future, considering opportunities soon to be provided by new paradigms, as hyperspectral sensing is becoming common at multiple scales from ground-based and airborne autonomous vehicles to manned aircraft and space-based platforms.

Pasolli, Edoardo↗
DOE OSTI2022

Data-driven cyber-attack detection for photovoltaic systems: A transfer learning approach

With increasing exposure to software-based sensing and control, power systems are facing higher risks of cyber/physical attacks. Here, to ensure system stability and minimize the potential economic losses, it is imperative to monitor the operating states and detect those attacks at the early stage. In this paper, a transfer learning method is proposed to detect cyber-attacks in photovoltaic (PV) systems with much less training data. First of all, two PV systems with a different number of PV inverters and power ratings are analyzed and their attack models are studied. Next, an attack detection Convolutional Neural Network (CNN) model was trained with rich amount of data from PV #1. Then, transfer learning was proposed to transfer the well-trained features from PV #1 to PV #2. Lastly, the attack detection model on PV #2 was trained based on the transferred CNN model. The experiment results show that the proposed transfer learning method achieves better accuracy and a faster convergence rate with a much less training dataset than conventional deep learning.

14 SOLAR ENERGY↗
DOE OSTI2026

Data-Driven Modeling and Correction of Vehicle Dynamics

We develop a data-driven framework for learning and correcting nonautonomous vehicle dynamics. Physics-based vehicle models are often simplified for tractability and therefore exhibit inherent model-form uncertainty, motivating the need for data-driven correction. Moreover, nonautonomous dynamics are governed by time-dependent control inputs, which pose challenges in learning predictive models directly from temporal snapshot data. To address these, we reformulate the vehicle dynamics via a local parameterization of the time-dependent inputs, yielding a modified system composed ofa sequence of local parametric dynamical systems. Here, we approximate these parametric systems using two complementary approaches. First, we employ the dimension reduction and interpolation in parameter space (DRIPS) methodology to construct efficient linear surrogate models, equipped with lifted observable spaces and manifold-based operator interpolation. This enables data-efficient learning of vehicle models whose dynamics admit accurate linear representations in the lifted spaces. Second, for more strongly nonlinear systems, we employ flow map learning (FML), a deep neural network (DNN) approach that approximates the parametric evolution map without requiring special treatment of nonlinearities. We further extend FML with a transfer-learning-based model correction procedure, enabling the correction of misspecified prior models using only a sparse set of high-fidelity or experimental measurements, without assuming a prescribed form for the correction term. Through a suite of numerical experiments on unicycle, simplified bicycle, and slip-based bicycle models, we demonstrate that DRIPS offers robust and highly data-efficient learning of nonautonomous vehicle dynamics, while FML provides expressive nonlinear modeling and effective correction of model-form errors under severe data scarcity.

data-driven modeling↗
DOE OSTI2025

Transfer learning of neural surrogates on multifidelity groundwater simulations

Multifidelity data used in the paper published in Advances in Water Resources 206 (2025) 105140, https://doi.org/10.1016/j.advwatres.2025.105140 The code used to process the data is openly available on GitHub at https://github.com/Model-Reduction-and-UQ-Group/Transfer_Learning_K_reconstruction Computationally inexpensive surrogates of process-based models, such as deep neural networks, enable ensemble-based computations used in risk assessment, data assimilation, etc. However, generation of large datasets required to train a neural network can be as expensive as the ensemble simulations themselves. We ameliorate this challenge by using data from multifidelity (MF) groundwater simulations and transfer learning (TL) to reduce data generation costs while maintaining model accuracy. As a computational example, we train a deep convolutional neural network (CNN) to reconstruct permeability fields from saturation maps derived from a multiphase flow model. Starting with very low- and low-fidelity data generated on increasingly coarse meshes, we pretrain the CNN, followed by output-layer training and fine-tuning using only a limited number of high-fidelity samples. We demonstrate the surrogate’s robustness when interpreting low-quality inputs—such as interpolated maps or data affected by noise—which has strong implications for the applicability in practical hydrogeological scenarios. This multilevel MF-TL strategy achieves a favorable trade-off between computational efficiency and predictive accuracy, significantly outperforming high-fidelity-only approaches under the same computational budget.

Chiofalo, Alessia [University of Bologna] (ORCID:0↗