Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “transfer learning.”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 127 records · Page 7
All sourcesNASA NTRSDOE OSTI
DOE OSTI2024

Cross-Feature Transfer Learning for Efficient Tensor Program Generation

Tuning tensor program generation involves navigating a vast search space to find optimal program transformations and measurements for a program on the target hardware. The complexity of this process is further amplified by the exponential combinations of transformations, especially in heterogeneous environments. This research addresses these challenges by introducing a novel approach that learns the joint neural network and hardware features space, facilitating knowledge transfer to new, unseen target hardware. A comprehensive analysis is conducted on the existing state-of-the-art dataset, TenSet, including a thorough examination of test split strategies and the proposal of methodologies for dataset pruning. Leveraging an attention-inspired technique, we tailor the tuning of tensor programs to embed both neural network and hardware-specific features. Notably, our approach substantially reduces the dataset size by up to 53% compared to the baseline without compromising Pairwise Comparison Accuracy (PCA). Furthermore, our proposed methodology demonstrates competitive or improved mean inference times with only 25–40% of the baseline tuning time across various networks and target hardware. The attention-based tuner can effectively utilize schedules learned from previous hardware program measurements to optimize tensor program tuning on previously unseen hardware, achieving a top-5 accuracy exceeding 90%. This research introduces a significant advancement in autotuning tensor program generation, addressing the complexities associated with heterogeneous environments and showcasing promising results regarding efficiency and accuracy.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2020

Improving radiograph analysis throughput through transfer learning and object detection

SIGN Fracture Care International partners with surgeons in low-resource hospitals worldwide to provide access to effective orthopedic care by donating educational materials and innovatively designed surgical implants. Over two decades, SIGN’s Online Surgical Database (SOSD) has grown to contain over 500,000 medical images, with radiographs holding the majority share. One challenge in working with hospitals worldwide is that both the radiographs uploaded to the SOSD and the data entry accompanying the uploads vary in quality. To improve the accuracy of data in the SOSD, we trained a model to detect surgical implants in radiographs. We first developed a tool to automatically detect radiographs, then trained an object detection model to determine the number and placement of surgical implants visible in the radiograph. Active learning was used to generate a training set containing 2,510 radiographs with screws, nails, and plates labeled by bounding boxes. Training a model to simultaneously recognize all three classes of implants gave a low average precision (AP) for the plate class, likely due to the low number of plate instances in our training set and the large variety of surgical plates used by SIGN-partnered surgeons. Applying standard image augmentation techniques to increase the plate count in our training set did not appreciably increase the AP of plate detection. To improve plate detection, we redrew the bounding boxes to account for correlations between the screw and plate classes. Training one model to detect nails and screws and a separate model to detect plates increased the AP of plate detection by 78.8 percentage points. The AP of each class was 80.7% for screws, 93.6% for nails, and 92.6% for plates; meanwhile, the sensitivity was 92% for screws, 86% for nails, and 81% for plates. We show that object detection methods can be used to detect surgical implants in radiographs of varying quality; however, the detection ability is dependent on the type of implant, and some implants, in our case plates, must be treated differently than others. Such tools can improve the throughput of radiograph analysis, assisting physicians and surgeons with the treatment of bone fractures.

60 APPLIED LIFE SCIENCES↗
DOE OSTI2024

Active and Transfer Learning of High-Dimensional Neural Network Potentials for Transition Metals

Classical molecular dynamics (MD) simulations represent a very popular and powerful tool for materials modeling and design. The predictive power of MD hinges on the ability of the interatomic potential to capture the underlying physics and chemistry. There have been decades of seminal work on developing interatomic potentials, albeit with a focus predominantly on capturing the properties of bulk materials. Such physics-based models, while extensively deployed for predicting the dynamics and properties of nanoscale systems over the past two decades, tend to perform poorly in predicting nanoscale potential energy surfaces (PESs) when compared to high-fidelity first-principles calculations. These limitations stem from the lack of flexibility in such models, which rely on a predefined functional form. Machine learning (ML) models and approaches have emerged as a viable alternative to capture the diverse size-dependent cluster geometries, nanoscale dynamics, and the complex nanoscale PESs, without sacrificing the bulk properties. Here, in this study, we introduce an ML workflow that combines transfer and active learning strategies to develop high-dimensional neural networks (NNs) for capturing the cluster and bulk properties for several different transition metals with applications in catalysis, microelectronics, and energy storage, to name a few. Our NN first learns the bulk PES from the high-quality physics-based models in literature and subsequently augments this learning via retraining with a higher-fidelity first-principles training data set to concurrently capture both the nanoscale and bulk PES. Our workflow departs from status-quo in its ability to learn from a sparsely sampled data set that nonetheless covers a diverse range of cluster configurations from near-equilibrium to highly nonequilibrium as well as learning strategies that iteratively improve the fingerprinting depending on model fidelity. All the developed models are rigorously tested against an extensive first-principles data set of energies and forces of cluster configurations as well as several properties of bulk configurations for 10 different transition metals. Our approach is material agnostic and provides a methodology to transfer and build upon the learnings from decades of seminal work in molecular simulations on to a new generation of ML-trained potentials to accelerate materials discovery and design.

36 MATERIALS SCIENCE↗
DOE OSTI2020

Transfer Learning for HVAC System Fault Detection

Faults in HVAC systems degrade thermal comfort and energy efficiency in buildings and have received significant attention from the research community, with data driven methods gaining in popularity. Yet the lack of labeled data, such as normal versus faulty operational status, has slowed the application of machine learning to HVAC systems. In addition, for any particular building, there may be an insufficient number of observed faults over a reasonable amount of time for training. To overcome these challenges, we present a transfer methodology for a novel Bayesian classifier designed to distinguish between normal operations and faulty operations. The key is to train this classifier on a building with a large amount of sensor and fault data (for example, via simulation or standard test data) then transfer the classifier to a new building using a small amount of normal operations data from the new building. We demonstrate a proof-of-concept for transferring a classifier between architecturally similar buildings in different climates and show few samples are required to maintain classification precision and recall.

transfer learning, Building HVAC, Bayesian framewo↗
DOE OSTI2025

Enhancing Cluster Identification in Atom Probe Tomography Data Using Transfer Learning

Atom probe tomography (APT) has enabled the direct visualization of solute clusters, providing valuable insights into material structures. This clustering is crucial for understanding the nanoscale composition and behavior of materials, which can significantly influence their mechanical and physical properties. However, the widely used clustering methods in the APT community face challenges such as subjective parametric selection and limited applicability, particularly in dealing with overlapping clusters, nested clusters, and artifacts across different scales, such as precipitates and dislocations. To address these challenges, we present a framework based on density-based cluster analysis that aims to be less dependent on user input, reproducible, and robust.

Density-based clustering↗
DOE OSTI2021

AtomSets as a hierarchical transfer learning framework for small and large materials datasets

Abstract Predicting properties from a material’s composition or structure is of great interest for materials design. Deep learning has recently garnered considerable interest in materials predictive tasks with low model errors when dealing with large materials data. However, deep learning models suffer in the small data regime that is common in materials science. Here we develop the AtomSets framework, which utilizes universal compositional and structural descriptors extracted from pre-trained graph network deep learning models with standard multi-layer perceptrons to achieve consistently high model accuracy for both small compositional data (<400) and large structural data (>130,000). The AtomSets models show lower errors than the graph network models at small data limits and other non-deep-learning models at large data limits. They also transfer better in a simulated materials discovery process where the targeted materials have property values out of the training data limits. The models require minimal domain knowledge inputs and are free from feature engineering. The presented AtomSets model framework can potentially accelerate machine learning-assisted materials design and discovery with less data restriction.

Chen, Chi (ORCID:0000000180087043)↗
DOE OSTI2020

Deep transfer learning for star cluster classification: I. application to the PHANGS– HST survey

ABSTRACT We present the results of a proof-of-concept experiment that demonstrates that deep learning can successfully be used for production-scale classification of compact star clusters detected in Hubble Space Telescope(HST) ultraviolet-optical imaging of nearby spiral galaxies ($D\lesssim 20\, \textrm{Mpc}$) in the Physics at High Angular Resolution in Nearby GalaxieS (PHANGS)–HST survey. Given the relatively small nature of existing, human-labelled star cluster samples, we transfer the knowledge of state-of-the-art neural network models for real-object recognition to classify star clusters candidates into four morphological classes. We perform a series of experiments to determine the dependence of classification performance on neural network architecture (ResNet18 and VGG19-BN), training data sets curated by either a single expert or three astronomers, and the size of the images used for training. We find that the overall classification accuracies are not significantly affected by these choices. The networks are used to classify star cluster candidates in the PHANGS–HST galaxy NGC 1559, which was not included in the training samples. The resulting prediction accuracies are 70 per cent, 40 per cent, 40–50 per cent, and 50–70 per cent for class 1, 2, 3 star clusters, and class 4 non-clusters, respectively. This performance is competitive with consistency achieved in previously published human and automated quantitative classification of star cluster candidate samples (70–80 per cent, 40–50 per cent, 40–50 per cent, and 60–70 per cent). The methods introduced herein lay the foundations to automate classification for star clusters at scale, and exhibit the need to prepare a standardized data set of human-labelled star cluster classifications, agreed upon by a full range of experts in the field, to further improve the performance of the networks introduced in this study.

Wei, Wei↗