Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “transfer learning.”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 55 records · Page 3
All sourcesNASA NTRSDOE OSTI
DOE OSTI2023

Using deep transfer learning and satellite imagery to estimate urban air quality in data-poor regions

Urban air pollution is a critical public health challenge in low-and-middle-income countries (LMICs). At the same time, LMICs tend to be data-poor, lacking adequate infrastructure to monitor air quality (AQ). As LMICs undergo rapid urbanization, the socio-economic burden of poor AQ will be immense. Here we present a globally scalable two-step deep learning (DL) based approach for AQ estimation in LMIC cities that mitigates the need for extensive AQ infrastructure on the ground. We train a DL model that can map satellite imagery to AQ in high-income countries (HICs) with sufficient ground data, and then adapt the model to learn meaningful AQ estimates in LMIC cities using transfer learning. The trained model can explain up to 54% of the variation in the AQ distribution of the target LMIC city without the need for target labels. The approach is demonstrated for Accra in Ghana, Africa, with AQ patterns learned and adapted from two HIC cities, specifically Los Angeles and New York.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2022

Multisource Mobile Transfer Learning Algorithm Based on Dynamic Model Compression

With the development of the Internet of Things, the application of computer vision on mobile phones is becoming more and more extensive and people have higher and higher requirements for the timeliness of the recognition results returned and the processing capabilities of the mobile phone for image recognition. However, the processing capability and storage capability of the user terminal equipment cannot meet the needs of identifying and storing a large number of pictures, and the data transmission process will cause high energy consumption of the terminal equipment. At the same time, multisource deep transfer learning has outstanding performance in computer vision and image classification. However, due to the huge amount of calculation of the deep network model, it is impossible to use the existing excellent network model to realize image recognition and classification on the mobile terminal. In order to solve the abovementioned problems, we propose a multisource mobile transfer learning algorithm based on dynamic model compression, this algorithm considers the realization of multisource transfer learning computing in the case of multiple mobile device computing source domains, and the method also guarantees data privacy and security for each device (origin domain). Meanwhile, extensive experiments show that our method can achieve remarkable results in popular image classification datasets.

Gao, Peng↗
DOE OSTI2021

Predicting fault slip via transfer learning

Abstract Data-driven machine-learning for predicting instantaneous and future fault-slip in laboratory experiments has recently progressed markedly, primarily due to large training data sets. In Earth however, earthquake interevent times range from 10’s-100’s of years and geophysical data typically exist for only a portion of an earthquake cycle. Sparse data presents a serious challenge to training machine learning models for predicting fault slip in Earth. Here we describe a transfer learning approach using numerical simulations to train a convolutional encoder-decoder that predicts fault-slip behavior in laboratory experiments. The model learns a mapping between acoustic emission and fault friction histories from numerical simulations, and generalizes to produce accurate predictions of laboratory fault friction. Notably, the predictions improve by further training the model latent space using only a portion of data from a single laboratory earthquake-cycle. The transfer learning results elucidate the potential of using models trained on numerical simulations and fine-tuned with small geophysical data sets for potential applications to faults in Earth.

58 GEOSCIENCES↗
DOE OSTI2024

A data-driven framework for predicting machining stability: employing simulated data, operational modal analysis, and enhanced transfer learning

Chatter, a self-excited vibration phenomenon, presents a significant challenge in machining operations, particularly in high-speed milling, where it can degrade tool life, reduce material removal efficiency, and compromise workpiece quality. Addressing this challenge requires a reliable predictive model that can accommodate the complex dynamics of various machining scenarios. This study introduces a novel, data-driven approach to predicting machining stability, leveraging over 140,000 simulated datasets and employing advanced techniques such as operational modal analysis (OMA), enhanced transfer learning (TL), and receptance coupling substructure analysis (RCSA). By integrating these methodologies, the framework effectively classifies and predicts chatter across diverse operational modes, achieving robust and accurate outcomes. Our model utilizes a Random Forest (RF) classifier trained with the comprehensive dataset, which demonstrates substantial improvements in both predictive accuracy and robustness. Specifically, the RF model achieved an accuracy rate of 85%, an area under the curve (AUC) of 0.90, and an F1 score of 0.88, underscoring its capability to adapt to varying machining configurations. These results highlight the framework’s potential to enhance operational efficiency and machining quality by providing reliable chatter predictions across a broad range of machining parameters. In conclusion, this research thus offers a significant advancement in predictive maintenance for machining processes, enabling more stable and efficient manufacturing operations.

42 ENGINEERING↗
DOE OSTI2025

Transfer learning nonlinear plasma dynamic transitions in low dimensional embeddings via deep neural networks

Deep learning algorithms provide a new paradigm to study high-dimensional dynamical behaviors, such as those in fusion plasma systems. Development of novel, data-driven model reduction methods, coupled with detection of abnormal modes with plasma physics, opens a unique opportunity to identify plasma instabilities through automated construction of parsimonious models that can be tuned to balance accuracy and cost. Our fusion transfer learning (FTL) model demonstrates success in rapidly reconstructing nonlinear kink mode structures by learning from a limited amount of nonlinear simulation data. The knowledge transfer process leverages a pre-trained neural encoder–decoder network, initially trained on linear simulations, to effectively capture nonlinear dynamics. The low-dimensional embeddings extract the coherent structures of interest, while preserving the inherent dynamics of the complex system. Experimental results highlight FTL’s capacity to capture transitional behaviors and dynamical features in plasma dynamics—a task often challenging for conventional methods. The model developed in this study is generalizable and can be extended broadly through transfer learning to address various magnetohydrodynamics modes.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗
DOE OSTI2022

Power System Event Identification with Transfer Learning Using Large-scale Real-world Synchrophasor Data in the United States

The lack of sufficient labeled events and long training time limit the applicability of deep neural network-based power system event identification using synchrophasor data. In this paper, we propose to leverage transfer learning technique to boost the reliability and reduce the required training time of neural classifier for power system event identification. We use the weights of a neural classifier trained on one transmission system as the initial parameters of another neural classifier for a different transmission system. Numerical tests with real-world synchrophasor data from the Eastern and Western Interconnections of the United States show that the proposed transfer learning approach is very effective in not only improving the training reliability but also reducing the training time.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2025

Leveraging transfer learning and leaf spectroscopy for leaf trait prediction with broad spatial, species, and temporal applicability

Accurate and reliable prediction of leaf traits is crucial for understanding plant adaptations to environmental variation, monitoring terrestrial ecosystems, and enhancing comprehension of functional diversity and ecosystem functioning. Currently, various approaches (e.g., statistical, physical models) have been developed to estimate leaf traits through hyperspectral remote sensing and leaf spectroscopy. However, the absence of high-performing, transferable, and stable models across various domains of space, plant functional types (PFTs) and seasons hinder our ability to quantify and comprehend spatiotemporal variations in leaf traits. This study proposes robust and highly transferable models for better predicting leaf traits with hyperspectral reflectance. Initially, three datasets were assembled, pairing common leaf traits — chlorophyll (Chla+b), carotenoids (Ccar), leaf mass per area (LAM), equivalent water thickness (EWT) — with leaf spectra measurements collected across diverse geographic locations in the U.S. and Europe, PFTs, and seasons. Measurements were acquired using spectroradiometers (e.g., ASD FieldSpec 3/4/Pro and SVC HR-1024i) with integrating spheres, leaf clips, and contact probes. Here, we then developed transfer learning-based hybrid models that incorporated the domain knowledge of radiative transfer models (RTMs) through pretraining processes and were well-constrained by fine-tuning with field measurements. Through comparison with other state-of-the-art statistical models, including partial-least squares regression (PLSR) and Gaussian Process Regression (GPR), as well as pure physical models, we found that the proposed transfer learning models achieved better predictive performance and higher transferability. Specifically, compared to other statistical models and pure RTMs, the transfer learning model exhibited higher coefficient of determination (R 2 ) values with range of 0.01 to 0.79, lower normalized root mean square error (NRMSE) with range of 0.06 % to 33.25 % in model performance. Additionally, the models exhibited improved transferability, with higher R 2 values range from 0.04 to 0.32, lower NRMSE range from 0.08 % to 30.81 %. The findings underscore that transfer learning models through integrating domain knowledge from RTMs and limited observations, can harness the advantages of both RTMs and statistical models and serve as a promising approach for effectively predicting leaf traits.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2023

A novel transfer learning framework for sorghum biomass prediction using UAV-based remote sensing data and genetic markers

Yield for biofuel crops is measured in terms of biomass, so measurements throughout the growing season are crucial in breeding programs, yet traditionally time- and labor-consuming since they involve destructive sampling. Modern remote sensing platforms, such as unmanned aerial vehicles (UAVs), can carry multiple sensors and collect numerous phenotypic traits with efficient, non-invasive field surveys. However, modeling the complex relationships between the observed phenotypic traits and biomass remains a challenging task, as the ground reference data are very limited for each genotype in the breeding experiment. In this study, a Long Short-Term Memory (LSTM) based Recurrent Neural Network (RNN) model is proposed for sorghum biomass prediction. The architecture is designed to exploit the time series remote sensing and weather data, as well as static genotypic information. As a large number of features have been derived from the remote sensing data, feature importance analysis is conducted to identify and remove redundant features. A strategy to extract representative information from high-dimensional genetic markers is proposed. To enhance generalization and minimize the need for ground reference data, transfer learning strategies are proposed for selecting the most informative training samples from the target domain. Consequently, a pre-trained model can be refined with limited training samples. Field experiments were conducted over a sorghum breeding trial planted in multiple years with more than 600 testcross hybrids. The results show that the proposed LSTM-based RNN model can achieve high accuracies for single year prediction. Further, with the proposed transfer learning strategies, a pre-trained model can be refined with limited training samples from the target domain and predict biomass with an accuracy comparable to that from a trained-from-scratch model for both multiple experiments within a given year and across multiple years.

36 MATERIALS SCIENCE↗
DOE OSTI2024

Transfer learning for probabilistic localization of hidden cracks in concrete structures

Abstract The utility of discriminative supervised learning models built using multiple training-data sources is investigated for hidden crack localization in concrete. Feed-forward neural network (FFNN) is chosen as the model architecture, and transfer learning is used to assimilate the information obtained from different sources (computational physics simulations and laboratory experiments). The labeled training data consists of values of a damage index and the known locations of hidden cracks. The classification models need to learn how the presence of damage (hidden cracks) affects the damage index at different sensors for different test conditions. To this end, diagnostic FFNN models are built by sequentially adding and training new hidden layers to assimilate labeled information from computer models (different model geometries, test conditions, crack lengths, crack locations) and laboratory experiments on a plain cement slab. These transfer learning-based models are then used to localize damage in concrete specimens that reflect real-world conditions (i.e., specimens with steel reinforcement and randomly distributed aggregate). The actual damage state in these specimens is determined by extracting cores and performing petrographic studies on the extracted cores. The damage probability estimated by transfer learning-based models is compared with the petrographic damage rating index (DRI) to identify the most suitable approach to train the diagnostic models. The transfer learning-based diagnostic methodology shows promise and could be used in various structural health monitoring applications, where sufficient labeled data are typically not available from a single data source.

Miele, S.↗
DOE OSTI2023

Accelerating Scientific Simulations with Bi-Fidelity Weighted Transfer Learning

High-fidelity modeling is an essential design tool for many engineering applications. However, for complex systems, computational cost can be a limiting factor. Analyzing parameter sensitivity, uncertainty quantification, and design optimization require many model evaluations. Surrogate models are often used to develop the relationship between model parameters and quantities of interest. However, in the case of complex systems, surrogate models require several degrees of freedom and, thus, a large number of data points to determine the correct dependencies. For many applications, this may be prohibitively expensive. The reduction of computational requirements can be achieved by leveraging low-fidelity models. Low-fidelity models represent the system at a coarser resolution with the advantage of computational efficiency. Therefore, a bi-fidelity modeling paradigm, which augments the accuracy of a low-fidelity model in a computationally efficient manner by invoking limited runs of a high-fidelity model, can be leveraged to sufficiently balance the accuracy and computational requirements. In this work, a bi-fidelity weighted transfer learning method using neural networks was applied to a computational fluid dynamics heat transfer modeling problem. The transfer learning advantage was investigated as a function of hyperparameters. Our main finding is that the use of a bi-fidelity modeling paradigm achieves accuracy close to that of a high-fidelity Gaussian process model while significantly reducing computational cost. The bi-fidelity model achieves comparable performance with 90 high-fidelity samples-that is, 60% less than the samples needed to achieve similar accuracy without the use of bi-fidelity modeling,

Borowiec, Katarzyna↗
DOE OSTI2023

Trust-Enhancing Probabilistic Transfer Learning for Sparse and Noisy Data Environments

There is an increasing aspiration to utilize machine learning (ML) for various tasks of relevance to national security. ML models have thus far been mostly applied to tasks and domains that, while impactful, have sufficient volume of data. For predictive tasks of national security relevance, ML models of great capacity (ability to approximate nonlinear trends in input-output maps) are often needed to capture the complex underlying physics. However, scientific problems of relevance to national security are often accompanied by various sources of sparse and/or incomplete data, including experiments and simulations, across different regimes of operation, of varying degrees of fidelity, and include noise with different characteristics and/or intensity. State-of-the-art ML models, despite exhibiting superior performance on the task and domain they were trained on, may suffer detrimental loss in performance in such sparse data environments. This report summarizes the results of the Laboratory Directed Research and Development project entitled Trust-Enhancing Probabilistic Transfer Learning for Sparse and Noisy Data Environments. The objective of the project was to develop a new transfer learning (TL) framework that aims to adaptively blend the data across different sources in tackling one task of interest, resulting in enhanced trustworthiness of ML models for mission- and safety-critical systems. The proposed framework determines when it is worth applying TL and how much knowledge is to be transferred, despite uncontrollable uncertainties. The framework accomplishes this by leveraging concepts and techniques from the fields of Bayesian inverse modeling and uncertainty quantification, relying on strong mathematical foundations of probability and measure theories to devise new uncertainty-aware TL workflows.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2021

Energy and Emission Prediction for Mixed-Vehicle Transit Fleets Using Multi-task and Inductive Transfer Learning

Public transit agencies are focused on making their fixed-line bus systems more energy efficient by introducing electric (EV) and hybrid (HV) vehicles to their fleets. However, because of the high upfront cost of these vehicles, most agencies are tasked with managing a mixed-fleet of internal combustion vehicles (ICEVs), EVs, and HVs. In managing mixed-fleets, agencies require accurate predictions of energy use for optimizing the assignment of vehicles to transit routes, scheduling charging, and ensuring that emission standards are met. The current state-of-the-art is to develop separate neural network models to predict energy consumption for each vehicle class. Although different vehicle classes’ energy consumption depends on a varied set of covariates, we hypothesize that there are broader generalizable patterns that govern energy consumption and emissions. In this paper, we seek to extract these patterns to aid learning to address two problems faced by transit agencies. First, in the case of a transit agency which operates many ICEVs, HVs, and EVs, we use multi-task learning (MTL) to improve accuracy of forecasting energy consumption. Second, in the case where there is a significant variation in vehicles in each category, we use inductive transfer learning (ITL) to improve predictive accuracy for vehicle class models with insufficient data. As this work is to be deployed by our partner agency, we also provide an online pipeline for joining the various sensor streams for fixed-line transit energy prediction. Here, we find that our approach outperforms vehicle-specific baselines in both the MTL and ITL settings.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2023

Intra-hour Solar Irradiance Forecast in Multiple Locations using Deep Transfer Learning

In recent years, solar power system installation imposes several challenges on the operations of local and regional power grids due to the inherent variability of ground-level solar irradiance. This work proposes a novel real-time solar forecast methodology for intra-hour solar irradiance based on deep transfer learning from ground-based sky imager for time horizons ranging from 5-15 min. There are three unique aspects of the proposed methodology: (1) a Deep Learning based algorithm development which is modeled as a classification approach rather than a traditional regression approach; (2) the use of the Transfer Learning technique to show generalization capability, robustness, and portability of baseline model in the newly deployed location where availability of enough data for training is typically scarce, and (3) redefinition of point-based irradiation forecast error estimation technique with a window-based one that is more intuitive and user-friendly. The system is developed using multiple years of irradiance and sky image recording in New Jersey and one-year data from Colorado, USA. The method is validated against ground telemetry from these two locations of diverse geographic and climatic conditions. Results show that the forecasting method proposed in this work is robust and highly accurate (8% MAPE error) for multiple locations deployment.

Deep Learning, Convolution Neural Networks, transf↗
DOE OSTI2023

Predicting Kyasanur forest disease in resource-limited settings using event-based surveillance and transfer learning

In recent years, the reports of Kyasanur forest disease (KFD) breaking endemic barriers by spreading to new regions and crossing state boundaries is alarming. Effective disease surveillance and reporting systems are lacking for this emerging zoonosis, hence hindering control and prevention efforts. We compared time-series models using weather data with and without Event-Based Surveillance (EBS) information, i.e., news media reports and internet search trends, to predict monthly KFD cases in humans. We fitted Extreme Gradient Boosting (XGB) and Long Short-Term Memory models at the national and regional levels. We utilized the rich epidemiological data from endemic regions by applying Transfer Learning (TL) techniques to predict KFD cases in new outbreak regions where disease surveillance information was scarce. Overall, the inclusion of EBS data, in addition to the weather data, substantially increased the prediction performance across all models. The XGB method produced the best predictions at the national and regional levels. The TL techniques outperformed baseline models in predicting KFD in new outbreak regions. Novel sources of data and advanced machine-learning approaches, e.g., EBS and TL, show great potential towards increasing disease prediction capabilities in data-scarce scenarios and/or resource-limited settings, for better-informed decisions in the face of emerging zoonotic threats.

60 APPLIED LIFE SCIENCES↗
DOE OSTI2024

Structure-aware graph neural network based deep transfer learning framework for enhanced predictive analytics on diverse materials datasets

Abstract Modern data mining methods have demonstrated effectiveness in comprehending and predicting materials properties. An essential component in the process of materials discovery is to know which material(s) will possess desirable properties. For many materials properties, performing experiments and density functional theory computations are costly and time-consuming. Hence, it is challenging to build accurate predictive models for such properties using conventional data mining methods due to the small amount of available data. Here we present a framework for materials property prediction tasks using structure information that leverages graph neural network-based architecture along with deep-transfer-learning techniques to drastically improve the model’s predictive ability on diverse materials (3D/2D, inorganic/organic, computational/experimental) data. We evaluated the proposed framework in cross-property and cross-materials class scenarios using 115 datasets to find that transfer learning models outperform the models trained from scratch in 104 cases, i.e., ≈90%, with additional benefits in performance for extrapolation problems. We believe the proposed framework can be widely useful in accelerating materials discovery in materials science.

Chemistry↗
DOE OSTI2026

A transfer learning approach to energy-efficient control of small and medium-sized commercial buildings

Model-free reinforcement learning (RL) provides a data-driven and adaptive approach to optimize building energy use while satisfying occupant comfort. This powerful tool does not need any prior knowledge about the environment and system it is optimizing and can adapt its policy based on the changes in captures. Like any other data-driven tool, it faces high training costs due to the extensive agent-environment interactions required to capture long-term building dynamics and user comfort. Transfer learning, particularly policy distillation, offers a promising way to accelerate training by leveraging pretrained RL agents in different building and system types. Here, this study investigates online student distillation, in which the student model updates its neural network weights using outputs from teacher models. The work introduces a student distillation strategy designed for efficient knowledge transfer, along with a teacher selection method that ensures high-quality guidance. The approach is validated using a highly calibrated whole building energy model for a small/medium commercial building test facility. Results show substantial reductions in training time and data requirements while surpassing the performance of ASHRAE Guideline 36, an advanced rule-based control strategy. The distilled RL model required 45% less data and achieved 20% higher cumulative rewards than a state-of-the-art RL model, with faster convergence and lower energy consumption. These outcomes demonstrate that effective transfer learning enables a scalable and data-efficient energy management solution for commercial buildings.

ASHRAE guideline 36↗
DOE OSTI2022

Automated bubble analysis of high-speed subcooled flow boiling images using U-net transfer learning and global optical flow

Capturing and analyzing the bubble dynamics is crucial to improving the understanding of boiling heat transfer mechanisms and predicting boiling heat transfer coefficient and boiling crisis. High speed video (HSV) imaging has been used for decades towards this end. Still, there is no universal approach to quantitatively analyze bubble dynamics from HSV images. In this study, we propose a data-driven post-processing approach to segment, track, and identify wall-attached vapor bubbles from HSV images of the boiling process in subcooled flow conditions. Firstly, we employ a transfer learning framework with a U-Net-based convolution neural network (CNN) architecture to detect and segment bubbles in HSV images of diverse contrast and surface texture using very little data (e.g., 10 images) for training. Then, we evaluate the trained CNN model with 100 ground-truth images, and the validation results show that the model accuracy and precision in detecting the optical footprint of bubbles are higher than 90%. Finally, we suggest a criterion to identify a condensing bubble based on the divergence of the bubble displacement, which is calculated from sequential segmented bubble images using a global optical flow code. Using this combination of machine learning and optical flow, we can identify nucleation sites and track the growth of bubbles nucleating at each site to quantify nucleation site density, nucleation frequency, and other fundamental boiling parameters. The proposed system is validated using results obtained on a special heater, which enables both infrared (IR) thermometry and HSV imaging on a metallic surface. We compare the fundamental boiling parameters obtained by the two different diagnostics. The results show good agreement. In conclusion, the difference between the measurements of nucleation site density, averaged nucleation frequency, and averaged growth time performed with the two techniques is always within ± 20% and mostly ± 10% of the values measured with IR thermometry.

42 ENGINEERING↗