Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “transfer learning.”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 109 records · Page 6
All sourcesNASA NTRSDOE OSTI
DOE OSTI2025

Quantifying dislocation-type defects in post irradiation examination via transfer learning

The quantitative analysis of dislocation-type defects in irradiated materials is critical to materials characterization in the nuclear energy industry. The conventional approach of an instrument scientist manually identifying any dislocation defects is both time-consuming and subjective, thereby potentially introducing inconsistencies in the quantification. This work approaches dislocation-type defect identification and segmentation using a standard open-source computer vision model, YOLO11, that leverages transfer learning to create a highly effective dislocation defect quantification tool while using only a minimal number of annotated micrographs for training. This model demonstrates the ability to segment both dislocation lines and loops concurrently in micrographs with high pixel noise levels and on two alloys not represented in the training set. Inference of dislocation defects using transmission electron microscopy on three different irradiated alloys relevant to the nuclear energy industry are examined in this work with widely varying pixel noise levels and with completely unrelated composition and dislocation formations for practical post irradiation examination analysis. Code and models are available at https://github.com/idaholab/PANDA.

36 MATERIALS SCIENCE↗
DOE OSTI2020

The PAU Survey: Photometric redshifts using transfer learning from simulations

In this paper, we introduce the DEEPZ deep learning photometric redshift (photo-z) code. As a test case, we apply the code to the PAU survey (PAUS) data in the COSMOS field. DEEPZ reduces the σ68 scatter statistic by 50 percent at iAB = 22.5 compared to existing algorithms. This improvement is achieved through various methods, including transfer learning from simulations where the training set consists of simulations as well as observations, which reduces the need for training data. The redshift probability distribution is estimated with a mixture density network (MDN), which produces accurate redshift distributions. Our code includes an autoencoder to reduce noise and extract features from the galaxy SEDs. It also benefits from combining multiple networks, which lowers the photo-z scatter by 10 percent. Furthermore, training with randomly constructed coadded fluxes adds information about individual exposures, reducing the impact of photometric outliers. In addition to opening up the route for higher redshift precision with narrow bands, these machine learning techniques can also be valuable for broad-band surveys.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2024

Electronic structure prediction of multi-million atom systems through uncertainty quantification enabled transfer learning

The ground state electron density — obtainable using Kohn-Sham Density Functional Theory (KS-DFT) simulations — contains a wealth of material information, making its prediction via machine learning (ML) models attractive. However, the computational expense of KS-DFT scales cubically with system size which tends to stymie training data generation, making it difficult to develop quantifiably accurate ML models that are applicable across many scales and system configurations. Here, we address this fundamental challenge by employing transfer learning to leverage the multi-scale nature of the training data, while comprehensively sampling system configurations using thermalization. Our ML models are less reliant on heuristics, and being based on Bayesian neural networks, enable uncertainty quantification. We show that our models incur significantly lower data generation costs while allowing confident — and when verifiable, accurate — predictions for a wide variety of bulk systems well beyond training, including systems with defects, different alloy compositions, and at multi-million-atom scales. Moreover, such predictions can be carried out using only modest computational resources.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2020

EGS Collab Experiment 1: 3D Seismic Velocity Model and Updated Microseismic Catalog Using Transfer-Learning Aided Double-Difference Tomography

This package contains a 3D Seismic velocity model and an updated microseismic catalog associated with a proceedings paper (Chai et al., 2020) published in the 45th Workshop on Geothermal Reservoir Engineering. The 3D_seismic_velocity_model text file contains x (m), y(m), z(m), P-wave velocity (km/s), P-wave velocity quality indicator (1 for well-constrained; 0 for poorly constrained), S-wave velocity (km/s), and S-wave velocity quality indicator (1 for well-constrained; 0 for poorly constrained). The Updated_MEQ_catalog text file contains event origin time, x(m), y(m), z(m), error in x (m), error in y (m), error in z (m), and RMS misfit (millisecond). The 3D_seismic_P-wave_velocity_model animation file shows slices of the 3D P-wave velocity model. The 3D_seismic_S-wave_velocity_model animation file shows slices of the 3D S-wave velocity model. The Interactive_MEQ_locations API file is an interactive visualization of the updated microseismic event locations. The visualization allows users to view the event locations by dragging, rotating, and zooming in. References: Chai, C., Maceira, M., Santos-Villalobos, H. J., Venkatakrishnan, S. V., Schoenball, M., and EGS Collab Team, 2020, Automatic Seismic Phase Picking Using Deep Learning for the EGS Collab Project, in PROCEEDINGS, 45th Workshop on Geothermal Reservoir Engineering, edited, Stanford University, Stanford, California, 45, 1266-1276.

15 GEOTHERMAL ENERGY↗
DOE OSTI2022

Monitoring of Temperature Measurements for Different Flow Regimes in Water and Galinstan with Long Short-Term Memory Networks and Transfer Learning of Sensors

Temperature sensing is one of the most common measurements of a nuclear reactor monitoring system. The coolant fluid flow in a reactor core depends on the reactor power state. We investigated the monitoring and estimation of the thermocouple time series using machine learning for a range of flow regimes. Measurement data were obtained, in two separate experiments, in a flow loop filled with water and with liquid metal Galinstan. We developed long short-term memory (LSTM) recurrent neural networks (RNNs) for sensor predictions by training on the sensor’s own prior history, and transfer learning LSTM (TL-LSTM) by training on a correlated sensor’s prior history. Sensor cross-correlations were identified by calculating the Pearson correlation coefficient of the time series. The accuracy of LSTM and TL-LSTM predictions of temperature was studied as a function of Reynolds number (Re). The root-mean-square error (RMSE) for the test segment of time series of each sensor was shown to linearly increase with Re for both water and Galinstan fluids. Using linear correlations, we estimated the range of values of Re for which RMSE is smaller than the thermocouple measurement uncertainty. For both water and Galinstan fluids, we showed that both LSTM and TL-LSTM provide reliable estimations of temperature for typical flow regimes in a nuclear reactor. The LSTM runtime was shown to be substantially smaller than the data acquisition rate, which allows for performing estimation and validation of sensor measurements in real time.

Pantopoulou, Stella↗
DOE OSTI2024

Enhanced Carbon Storage Forecasting via Cross-Geology Transfer Learning

Rapid simulation of the spatiotemporal evolution of pressure & saturation for SACROC 1. Neural operator was trained on only 153 simulation runs 2. Trained to account for heterogeneity and variations/uncertainties in engineering, fluids, and geology 3. Pressure forecast has less than 1% error 4. Saturation forecast has less than 2% error 5. Traditional simulator takes 1 hour for a single scenario, while neural operator takes less than 1 minute. Rapid simulation of the spatiotemporal evolution of pressure & saturation for IBDP 1. Transfer Learning was implemented on the SACROC-based Neural Operator that was trained on only 20 simulation runs for IBDP Site 2. SACROC and IBDP Sites have several significant differences in geology and engineering parameters. 3. Pressure forecast has less than 5 psi error 4. Saturation forecast has less than 7% error 5. Traditional simulator takes 1 hour for a single scenario, while neural operator takes less than 1 minute and only 20 simulations for training/validation.

Misra, Siddharth↗
DOE OSTI2022

Preliminary Transfer Learning Results on Israel Data

In this preliminary report, we use publicly available data recorded in Israel to test and expand upon existing machine learning models for seismic-phase detection and arrival-time measurement. We downloaded 3-years of waveform data from Geofon, and cross referenced the waveforms to Israel bulletin picks (Schardong et al., 2021). The initial results using existing models directly generated ubiquitous false detections and that obscured detections of signals that are clearly visible in the waveforms. However, after applying transfer learning (tuning parameters in the existing ML models using one year of the Israel-network data), the results are encouraging, i.e. ML picks agree within a few tenths of a second with bulletin picks and the number of false detections is greatly reduced. The bulletin picks are a good starting point, but they cannot be considered ground-truth. To test potential improvement in picking using ML we would like to relocate the events using the ML picks to see if the events cluster more tightly at known mine locations. However, in order to constrain event locations, we need ML picks for the whole Israeli-Jordanian network, which requires waveforms that are not publicly available.

58 GEOSCIENCES↗
DOE OSTI2022

Detecting damaged buildings using real-time crowdsourced images and transfer learning

After significant earthquakes, we can see images posted on social media platforms by individuals and media agencies owing to the mass usage of smartphones these days. These images can be utilized to provide information about the shaking damage in the earthquake region both to the public and research community, and potentially to guide rescue work. This paper presents an automated way to extract the damaged buildings images after earthquakes from social media platforms such as Twitter and thus identify the particular user posts containing such images. Using transfer learning and ~ 6500 manually labelled images, we trained a deep learning model to recognize images with damaged buildings in the scene. The trained model achieved good performance when tested on newly acquired images of earthquakes at different locations and when ran in near real-time on Twitter feed after the 2020 M7.0 earthquake in Turkey. Furthermore, to better understand how the model makes decisions, we also implemented the Grad-CAM method to visualize the important regions on the images that facilitate the decision.

58 GEOSCIENCES↗
DOE OSTI2026

Hierarchical transfer learning: an agile and equitable strategy for machine-learning interatomic models

Machine-learned interatomic models are growing in popularity due to their ability to afford near quantum-accurate predictions for complex phenomena with orders-of-magnitude greater computational efficiency. However, these models struggle when applied to systems of many element types due to the approximately exponential increase in number of parameters that must be determined. To mitigate this challenge, we present a new hierarchical transfer learning approach that allows the fitting problem to be decomposed into smaller independent and reusable parameter blocks that enable development of explicitly chemically extensible ML-IAM. Application of this strategy is demonstrated for C and N mixtures under conditions ranging from nominally ambient to ~10,000 K and 200 GPa for compositions from 0 to 100% N. Ultimately, this strategy makes model generation for chemically complex systems more tractable and efficient, facilitates comprehensive model validation, and makes ML-IAM development for problems of this nature more accessible to users with limited access to extreme computing infrastructure.

Lindsey, Rebecca K. [Univ. of Michigan, Ann Arbor,↗
DOE OSTI2022

Efficient emulation of relativistic heavy ion collisions with transfer learning

Measurements from the Large Hadron Collider (LHC) and the Relativistic Heavy Ion Collider (RHIC) can be used to study the properties of quark-gluon plasma. Systematic constraints on these properties must combine measurements from different collision systems and methodically account for experimental and theoretical uncertainties. Such studies require a vast number of costly numerical simulations. While computationally inexpensive surrogate models (“emulators”) can be used to efficiently approximate the predictions of heavy ion simulations across a broad range of model parameters, training a reliable emulator remains a computationally expensive task. We use transfer learning to map the parameter dependencies of one model emulator onto another, leveraging similarities between different simulations of heavy ion collisions. By limiting the need for large numbers of simulations to only one of the emulators, this technique reduces the numerical cost of comprehensive uncertainty quantification when studying multiple collision systems and exploring different models.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗
DOE OSTI2024

Cluster expansion by transfer learning for phase stability predictions

Recent progress towards universal machine-learned interatomic potentials holds considerable promise for materials discovery. Yet the accuracy of these potentials for predicting phase stability may still be limited. In contrast, cluster expansions provide accurate phase stability predictions but are computationally demanding to parameterize from first principles, especially for structures of low dimension or with a large number of components, such as interfaces or multimetal catalysts. We overcome this trade-off via transfer learning. Using Bayesian inference, we incorporate prior statistical knowledge from machine-learned and physics-based potentials, enabling us to sample the most informative configurations and to efficiently fit first-principles cluster expansions. Furthermore, this algorithm is tested on Pt:Ni, showing robust convergence of the mixing energies as a function of sample size with reduced statistical fluctuations.

36 MATERIALS SCIENCE↗
DOE OSTI2022

Detection of multi-reference character imbalances enables a transfer learning approach for virtual high throughput screening with coupled cluster accuracy at DFT cost

Appropriately identifying and treating molecules and materials with significant multi-reference (MR) character is crucial for achieving high data fidelity in virtual high-throughput screening (VHTS). Despite development of numerous MR diagnostics, the extent to which a single value of such a diagnostic indicates the MR effect on a chemical property prediction is not well established. We evaluate MR diagnostics for over 10 000 transition-metal complexes (TMCs) and compare to those for organic molecules. We observe that only some MR diagnostics are transferable from one chemical space to another. By studying the influence of MR character on chemical properties (i.e., MR effect) that involve multiple potential energy surfaces (i.e., adiabatic spin splitting, ΔE H–L , and ionization potential, IP), we show that differences in MR character are more important than the cumulative degree of MR character in predicting the magnitude of an MR effect. Motivated by this observation, we build transfer learning models to predict CCSD(T)-level adiabatic ΔE H–L and IP from lower levels of theory. By combining these models with uncertainty quantification and multi-level modeling, we introduce a multi-pronged strategy that accelerates data acquisition by at least a factor of three while achieving coupled cluster accuracy (i.e., to within 1 kcal mol –1 MAE) for robust VHTS.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2025

Transfer learning of neural surrogates on multifidelity groundwater simulations

Multifidelity data used in the paper published in Advances in Water Resources 206 (2025) 105140, https://doi.org/10.1016/j.advwatres.2025.105140 The code used to process the data is openly available on GitHub at https://github.com/Model-Reduction-and-UQ-Group/Transfer_Learning_K_reconstruction Computationally inexpensive surrogates of process-based models, such as deep neural networks, enable ensemble-based computations used in risk assessment, data assimilation, etc. However, generation of large datasets required to train a neural network can be as expensive as the ensemble simulations themselves. We ameliorate this challenge by using data from multifidelity (MF) groundwater simulations and transfer learning (TL) to reduce data generation costs while maintaining model accuracy. As a computational example, we train a deep convolutional neural network (CNN) to reconstruct permeability fields from saturation maps derived from a multiphase flow model. Starting with very low- and low-fidelity data generated on increasingly coarse meshes, we pretrain the CNN, followed by output-layer training and fine-tuning using only a limited number of high-fidelity samples. We demonstrate the surrogate’s robustness when interpreting low-quality inputs—such as interpolated maps or data affected by noise—which has strong implications for the applicability in practical hydrogeological scenarios. This multilevel MF-TL strategy achieves a favorable trade-off between computational efficiency and predictive accuracy, significantly outperforming high-fidelity-only approaches under the same computational budget.

Chiofalo, Alessia [University of Bologna] (ORCID:0↗
DOE OSTI2021

Transfer Learning using Denoising Auto-Encoders for Cellular-Level Annotation of Tumor in Pathology Slides

Adversarial examples can produce altered classifications using only seemingly innocuous, imperceptible perturbations to the original image. The imperceptibility of adversarial perturbations suggests that the corresponding classifiers use decision criteria different than those of a human. In a medical setting, inexplicable decision criteria confound a pathologist’s willingness to trust machine-generated annotations. Here, we analyze denoising tumor detection models to see if they are robust to imperceptible adversarial perturbations. Moreover, to be more fully trusted by pathologists, we require tumor detectors that generate interpretable annotations which segment pathology slides into tumorous and normal regions at the cellular level. We therefore compare transfer learning based on two different autoencoder architectures, one derived from a deep denoising bottleneck autoencoder and one from an over-complete sparse autoencoder. Both autoencoders were first trained in an unsupervised manner on a set of pathology slides drawn from the Camelyon16 dataset. The latent representations produced by each autoencoder were then passed to separate neural networks that were trained in a supervised manner on binary tumor-normal masks generated by pathologists at cellular resolution. Both tumor detectors supported better than 90% AUC PR as measured by the area under the precision/recall curve on a held-out pathology slide. To assess the underlying decision criteria used by both tumor detectors, we constructed imperceptible adversarial examples which reduced the AUC PR of both models to less than 70%. Random noise of the same amplitude had almost no effect on the AUC PR of either model. Additionally, each tumor detector was resistant to adversarial “transfer” attacks targeting the other. The adversarial perturbations showed strong characteristic differences: the deep denoising models perturbations were a very diffuse, seemingly unrecognizable pattern while the sparse coding models perturbations showed traces of tissue cells.

47 OTHER INSTRUMENTATION↗
DOE OSTI2023

Transfer learning as a method to reproduce high-fidelity non-local thermodynamic equilibrium opacities in simulations

Simulations of high-energy density physics often need non-local thermodynamic equilibrium opacity data. These data, however, are expensive to produce at relatively low fidelity. It is even more so at high fidelity such that the opacity calculations can contribute 95 % of the total computation time. This proportion can even reach large proportions. Neural networks can be used to replace the standard calculations of low-fidelity data, and the neural networks can be trained to reproduce artificial, high-fidelity opacity spectra. In this work, it is demonstrated that a novel neural network architecture trained to reproduce high-fidelity krypton spectra through transfer learning can be used in simulations. Further, it is demonstrated that this can be done while achieving a relative per cent error of the peak radiative temperature of the hohlraum of approximately 1 % to 4 % while achieving a 19.4 $\times$ speed up.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
DOE OSTI2024

Molecular simulation using transfer-learned potentials for the disordered nanoscale structure of nitrogen-doped nanoporous carbons

Machine learning (ML)-based molecular dynamics (MD) simulations of the formation of a class of N-doped nanoporous carbons are performed to assess their disordered partially graphitized nanoscale structure. The study is motivated by the effectiveness of so-called nitrogen assembly carbons (NACs) for catalysis applications. Benchmark simulations for pure-C disordered graphitic systems reveal the importance of reliably capturing the vdW component of the potentials in order to accurately describe the tendency for layering of disordered graphene-like sheets. In our modeling, this is achieved by a transfer learning strategy incorporating features of the energetics from the optB88-vdW DFT functional into potentials initially trained with a less expensive functional, thereby providing a superior description of the pure-C systems. Generation from MD simulations of realistic partially graphitized structures is significantly more challenging for N-doped versus for pure C systems. However, such structures are achieved by a tailored MD simulation protocol mimicking the experimental synthesis process and in particular incorporating an annealing and subsequent quenching stages. Simulated PXRD patterns effectively reproduce the features of experimental observations for NACs, including the appearance of a prominent but broad (002) peak at around 25, and the development of another weaker feature associated with in-layer ordering of mixed C-N graphene-like sheets.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2023

Geolocation tracking for human identification and activity recognition using radar deep transfer learning

Abstract Human identification and activity recognition (HIAR) is crucial for many applications, such as surveillance, smart homes, and assisted living. As a sensing modality, radar has many unique characteristics including privacy protection, and contactless sensing. Single classification systems have shown to be accurate, but for long‐term solutions both human identification (ID) and human activity recognition (HAR) will need to be integrated in one system where it can be utilised simultaneously. In this article, a novel radar‐based human tracking system is presented where three classifiers are utilised to identify the subject and his/her behaviour. For any kind of motion, the system tracks the subject and detect the type of his/her motion. Based on the detected type of motion, the three classifiers are utilised for identification and activity recognition. The classifiers are built utilising deep transfer learning where three radar datasets are established to train and validate each of the deep networks. To recognise six activities and 10 human subjects, the three classifiers, namely, HAR, Gait ID, and Heart sound ID, achieve superior performance compared to the best reported results in literature with classification accuracies of 97.6%, 100%, and 41.8% respectively. Three successful examples are presented to demonstrate the introduced concept.

Alkasimi, Ahmad↗