Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Transfer Learning Model”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 487 records · Page 27
All sourcesNASA NTRSDOE OSTI
DOE OSTI2020

Effects of Jacobian Matrix Regularization on the Detectability of Adversarial Samples

The well-known vulnerability of Deep Neural Networks to adversarial samples has led to a rapid cycle of increasingly sophisticated attack algorithms and proposed defenses. While most contemporary defenses have been shown to be vulnerable to carefully configured attacks, methods based on gradient regularization and out-of-distribution detection have attracted much interest recently by demonstrating higher resilience to a broad range of attack algorithms. However, no study has yet investigated the effect of combining these techniques. In this paper, we consider the effect of Jacobian matrix regularization on the detectability of adversarial samples on the CIFAR-10 image benchmark dataset. We find that regularization has a significant effect on detectability, and in some cases can make an undetectable attack on a baseline model detectable. In addition, we give evidence that regularization may mitigate the known weaknesses of detectors to high-confidence adversarial samples. The defenses we consider here are highly generalizable, and we believe they will be useful for further investigations to transfer machine learning robustness to other data domains.

45 MILITARY TECHNOLOGY, WEAPONRY, AND NATIONAL DEF↗
DOE OSTI2020

On Performance Prediction of Big Data Transfer in High-performance Networks

Big data generated by large-scale scientific and industrial applications need to be transferred between different geographical locations for remote storage, processing, and analysis. High-speed dedicated connections provisioned in High-performance Networks (HPNs) are increasingly utilized to carry out such big data transfer. HPN management highly relies on an important capability of performance (mainly throughput) prediction to reserve sufficient bandwidth and meanwhile avoid over-provisioning that may result in unnecessary resource waste. This capability is critical to improving the resource (mainly bandwidth) utilization of dedicated connections and meeting various user requests for data transfer. Conventional methods conduct performance prediction by fitting prior observed transfer history with predefined loss functions, without considering unobservable latent factors such as competing loads on end hosts. Such latent factors also have a significant impact on the application-level data transfer performance, which may result in an inaccurate prediction model. In this paper, we first investigate the impact of latent factors and propose a clustering-based method to eliminate their negative impact on performance prediction. We then develop a robust machine learning-based performance predictor by: i) incorporating the proposed latent factor elimination method into data preprocessing, and ii) adopting a customized domain guided loss function. Extensive experimental results show that our predictor achieves significantly higher prediction accuracy than several other state-of-the-art methods.

Liu, Wuji↗
NASA NTRS2018

Advances in Hyperspectral Image Classification Methods for Vegetation and Agricultural Cropland Studies

Hyperspectral data are becoming more widely available via sensors on airborne and unmanned aerial vehicle (UAV) platforms, as well as proximal platforms. While space-based hyperspectral data continue to be limited in availability, multiple spaceborne Earth-observing missions on traditional platforms are scheduled for launch, and companies are experimenting with small satellites for constellations to observe the Earth, as well as for planetary missions. Land cover mapping via classification is one of the most important applications of hyperspectral remote sensing and will increase in significance as time series of imagery are more readily available. However, while the narrow bands of hyperspectral data provide new opportunities for chemistry-based modeling and mapping, challenges remain. Hyperspectral data are high dimensional, and many bands are highly correlated or irrelevant for a given classification problem. For supervised classification methods, the quantity of training data is typically limited relative to the dimension of the input space. The resulting Hughes phenomenon, often referred to as the curse of dimensionality, increases potential for unstable parameter estimates, overfitting, and poor generalization of classifiers. This is particularly problematic for parametric approaches such as Gaussian maximum likelihood–based classifiers that have been the backbone of pixel-based multispectral classification methods. This issue has motivated investigation of alternatives, including regularization of the class covariance matrices, ensembles of weak classifiers, development of feature selection and extraction methods, adoption of nonparametric classifiers, and exploration of methods to exploit unlabeled samples via semi-supervised and active learning. Data sets are also quite large, motivating computationally efficient algorithms and implementations. This chapter provides an overview of the recent advances in classification methods for mapping vegetation using hyperspectral data. Three data sets that are used in the hyperspectral classification literature (e.g., Botswana Hyperion satellite data and AVIRIS airborne data over both Kennedy Space Center and Indian Pines) are described in Section 3.2 and used to illustrate methods described in the chapter. An additional high-resolution hyperspectral data set acquired by a SpecTIR sensor on an airborne platform over the Indian Pines area is included to exemplify the use of new deep learning approaches, and a multiplatform example of airborne hyperspectral data is provided to demonstrate transfer learning in hyperspectral image classification. Classical approaches for supervised and unsupervised feature selection and extraction are reviewed in Section 3.3. In particular, nonlinearities exhibited in hyperspectral imagery have motivated development of nonlinear feature extraction methods in manifold learning, which are outlined in Section 3.3.1.4. Spatial context is also important in classification of both natural vegetation with complex textural patterns and large agricultural fields with significant local variability within fields. Approaches to exploit spatial features at both the pixel level (e.g., co-occurrence–based texture and extended morphological attribute profiles [EMAPs]) and integration of segmentation approaches (e.g., HSeg) are discussed in this context in Section 3.3.2. Recently, classification methods that leverage nonparametric methods originating in the machine learning community have grown in popularity. An overview of both widely used and newly emerging approaches, including support vector machines (SVMs), Gaussian mixture models, and deep learning based on convolutional neural networks is provided in Section 3.4. Strategies to exploit unlabeled samples, including active learning and metric learning, which combine feature extraction and augmentation of the pool of training samples in an active learning framework, are outlined in Section 3.5. Integration of image segmentation with classification to accommodate spatial coherence typically observed in vegetation is also explored, including as an integrated active learning system. Exploitation of multisensor strategies for augmenting the pool of training samples is investigated via a transfer learning framework in Section 3.5.1.2. Finally, we look to the future, considering opportunities soon to be provided by new paradigms, as hyperspectral sensing is becoming common at multiple scales from ground-based and airborne autonomous vehicles to manned aircraft and space-based platforms.

Pasolli, Edoardo↗
DOE OSTI2021

Physics-Guided Machine Learning for Prediction of Cloud Properties in Satellite-Derived Solar Data: Preprint

With over 20 years of high-resolution surface irradiance data covering most of the western hemisphere, the National Solar Radiation Database (NSRDB) is a vital public data asset. The NSRDB uses a two-step Physical Solar Model (PSM) that explicitly considers the effects of clouds and other atmospheric variables on radiative transfer. High-quality physical and optical cloud properties derived from satellite imagery are perhaps the most important data input to the PSM, representing the greatest source of radiation attenuation and scattering. However, traditional methods for cloud property retrieval have their own limitations and are unable to accurately predict cloud properties outside of nominal conditions. We introduce a physics-guided neural network that can accurately predict cloud properties when traditional methods fail or are inaccurate. Using this framework, we show reductions in relative Root Mean Square Error (RMSE) for Global Horizontal Irradiance (GHI) up to 13 percentage points for timesteps that previously had missing or low-quality cloud property data. We expect that this methodology will be effective in improving the quality of cloud property and solar irradiance data in the NSRDB.

41 EE - Solar Energy Technologies Office (EE-4S)↗
DOE OSTI2024

RuralAI in Tomato Farming: Integrated Sensor System, Distributed Computing, and Hierarchical Federated Learning for Crop Health Monitoring

Precision horticulture is evolving due to scalable sensor deployment and machine learning (ML) integration. These advancements boost the operational efficiency of individual farms, balancing the benefits of analytics with autonomy requirements. However, given concerns that affect wide geographic regions (e.g., climate change), there is a need to apply models that span farms. Federated learning (FL) has emerged as a potential solution. FL enables decentralized ML across different farms without sharing private data. Traditional FL assumes simple two-tier network topologies and, thus, falls short of operating on more complex networks found in real-world agricultural scenarios. Networks vary across crops and farms and encompass various sensor data modes, extending across jurisdictions. New hierarchical FL (HFL) approaches are needed for more efficient and context-sensitive model sharing, accommodating regulations across multiple jurisdictions. Here, we present the RuralAI architecture deployment for tomato crop monitoring, featuring sensor field units for soil, crop, and weather data collection. HFL with personalization is used to offer localized and adaptive insights. Model management, aggregation, and transfers are facilitated via a flexible approach, enabling seamless communication between local devices, edge nodes, and the cloud.

60 APPLIED LIFE SCIENCES↗
DOE OSTI2021

Analysis-Specific Fast Simulation at the LHC with Deep Learning

Abstract We present a fast-simulation application based on a deep neural network, designed to create large analysis-specific datasets. Taking as an example the generation of W + jet events produced in $$\sqrt{s}=$$ s = 13 TeV proton–proton collisions, we train a neural network to model detector resolution effects as a transfer function acting on an analysis-specific set of relevant features, computed at generation level, i.e., in absence of detector effects. Based on this model, we propose a novel fast-simulation workflow that starts from a large amount of generator-level events to deliver large analysis-specific samples. The adoption of this approach would result in about an order-of-magnitude reduction in computing and storage requirements for the collision simulation workflow. This strategy could help the high energy physics community to face the computing challenges of the future High-Luminosity LHC.

Chen, C.↗
DOE OSTI2020

Active Learning A Neural Network Model For Gold Clusters & Bulk From Sparse First Principles Training Data

Small metal clusters are of fundamental scientific interest and of tremendous significance in catalysis. These nanoscale clusters display diverse geometries and structural motifs depending on the cluster size; a knowledge of this size-dependent structural motifs and their dynamical evolution has been of longstanding interest. Given the high computational cost of first-principles calculations, molecular modeling and atomistic simulations such as molecular dynamics (MD) has proven to be an important complementary tool to aid this understanding. Classical MD typically employ predefined functional forms which limits their ability to capture such complex size-dependent structural and dynamical transformation. Neural Network (NN) based potentials represent flexible alternatives and in principle, well-trained NN potentials can provide high level of flexibility, transferability and accuracy on-par with the reference model used for training. A major challenge, however, is that NN models are interpolative and requires large quantities (similar to 10 4 or greater) of training data to ensure that the model adequately samples the energy landscape both near and far-from-equilibrium. A highly desirable goal is minimize the number of training data, especially if the underlying reference model is first-principles based and hence expensive. In this work, we introduce an active learning (AL) scheme that trains a NN model on-the-fly with minimal amount of first-principles based training data. Our AL workflow is initiated with a sparse training dataset (similar to 1 to 5 data points) and is updated on-the-fly via a Nested Ensemble Monte Carlo scheme that iteratively queries the energy landscape in regions of failure and updates the training pool to improve the network performance. Using a representative system of gold clusters, we demonstrate that our AL workflow can train a NN with similar to 500 total reference calculations. Using an extensive DFT test set of similar to 1100 configurations, we show that our AL-NN is able to accurately predict both the DFT energies and the forces for clusters of a myriad of different sizes. Our NN predictions are within 30 meV/atom and 40 meV/angstrom of the reference DFT calculations. Moreover, our AL-NN model also adequately captures the various size-dependent structural and dynamical properties of gold clusters in excellent agreement with DFT calculations and available experiments. We finally show that our AL-NN model also captures bulk properties reasonably well, even though they were not included in the training data.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2026

Sim-to-real supervised domain adaptation for radioisotope identification

Machine learning has the potential to improve the speed and reliability of radioisotope identification using gamma spectroscopy. However, meticulously labeling an experimental dataset for training is often prohibitively expensive, while training models purely on synthetic data is risky due to the domain gap between simulated and experimental measurements. In this research, we demonstrate that supervised domain adaptation can substantially improve the performance of radioisotope identification models by transferring knowledge between synthetic and experimental data domains. We consider two domain adaptation scenarios: (1) a simulation-to-simulation adaptation, where we perform multi-label proportion estimation using simulated high-purity germanium detectors, and (2) a simulation-to-experimental adaptation, where we perform multi-class, single-label classification using measured spectra from handheld lanthanum bromide (LaBr) and sodium iodide (NaI) detectors. We begin by pretraining a spectral classifier on synthetic data using a custom transformer-based neural network. After subsequent fine-tuning on just 64 labeled experimental spectra, we achieve a test accuracy of 96% in the sim-to-real scenario with a LaBr detector, far surpassing a synthetic-only baseline model (75%) and a model trained from scratch (80%) on the same 64 spectra. Furthermore, we demonstrate that domain-adapted models learn more human-interpretable features than experiment-only baseline models. Overall, our results highlight the potential for supervised domain adaptation techniques to bridge the sim-to-real gap in radioisotope identification, enabling the development of accurate and explainable classifiers even in real-world scenarios where access to experimental data is limited.

Lalor, Peter W.↗
DOE OSTI2023

CoolPINNs: A physics-informed neural network modeling of active cooling in vascular systems

Emerging technologies like hypersonic aircraft, space exploration vehicles, and batteries avail fluid circulation in embedded microvasculatures for efficient thermal regulation. Modeling is vital during the design and operational phases of these engineered systems. However, many challenges exist in developing a modeling framework. What is lacking is an accurate framework that (i) captures sharp jumps in the thermal flux across complex vasculature layouts, (ii) deals with oblique derivatives (involving tangential and normal components), (iii) handles nonlinearity because of radiative heat transfer, (iv) provides a high-speed forecast for real-time monitoring, and (v) facilitates robust inverse modeling. Here, this paper addresses these challenges by availing the power of physics-informed neural networks (PINNs). We develop a fast, reliable, and accurate Scientific Machine Learning (SciML) framework for vascular-based thermal regulation—called CoolPINNs: a PINNs-based modeling framework for active cooling. The proposed mesh-less framework elegantly overcomes all the mentioned challenges. The significance of the reported research is multi-fold. First, the framework is valuable for real-time monitoring of thermal regulatory systems because of rapid forecasting. Second, researchers can address complex thermoregulation designs since the approach is meshless. Finally, the framework facilitates systematic parameter identification and inverse modeling studies, perhaps the most significant utility of the current framework.

97 MATHEMATICS AND COMPUTING↗
NASA NTRSDate not supplied

The NASA Turbulent Heat Flux Experiments: Summary and Lessons Learned

The Turbulent Heat Flux (THX) experiments were conducted at NASA Glenn Research Center (GRC) in order to collect measurements of velocities and temperatures for computational fluid dynamics (CFD) validation of heated flows, with a focus on propulsion system components. The experiments spanned 5 phases; four of which were conducted in the GRC AeroAcoustic Propulsion Laboratory (AAPL) using the Small Hot Jet Flow Rig (SHJAR). In addition to making velocity measurements with Particle Image Velocimetry (PIV), the THX experiments introduced a new Raman-scattering based capability to measure temperatures. Computational studies were also conducted for each of the experimental configurations, in order to provide a baseline of expected CFD results and conduct an assessment of the capability of various CFD approaches for calculating flows where the turbulent transport of heat was important. Two of the collected sets of data were used for American Institute of Aeronautics and Astronautics (AIAA) Propulsion Aerodynamic Workshops (PAWs). The data set from the 5th phase, collected for heated supersonic jets, was also used to construct new validation cases for the NASA Turbulence Model Resource (TMR). This paper provides an overview of the experiments and associated computations for each of the 5 test phases. Key experimental findings are presented. Lessons learned are provided concerning the effect of computational modeling choice on accuracy of predicting turbulent flows where thermal transport is important. Emphasis is placed on comparing Reynolds- averaged Navier-Stokes approaches with large-eddy simulation approaches. The benefits of utilizing a conjugate heat transfer method in conjunction with CFD solver for film cooling is demonstrated.

RANS↗
DOE OSTI2020

Residue Specific Characterization of Electrostatics, Conformational Heterogeneity, and Dynamics of Electron Transfer Proteins with Linear and Nonlinear Infrared Spectroscopy. Final Report

The goal of this research project was to further our understanding of biological electron transfer through the development and application of new methods for studying proteins and their complexes with high spatial and temporal detail. The approach was based on incorporation of reporter bonds with distinct vibrational frequencies at specific locations in proteins and their characterization through the interaction of the bonds with pulses of infrared light. The project focused on the proteins plastocyanin (Pc) and cytochrome (cyt) f, two electron transfer partners in photosynthesis. The first thrust of the project was the incorporation of carbon deuterium bonds as non-perturbative probes at a ligand to the copper ion of Pc and their use to detect changes in the bonding due to the number of electrons at the metal and how the bonding changes upon association with cyt f. Toward developing more advanced approaches to characterize proteins, we incorporated a stronger reporter, the cyano group of cyanophenylalanine, at several locations in Pc and measured the frequency dynamics in the absence and presence of cyt f. The acquired data improved the current model for the protein complex. Notably, the data indicated that the majority of the population is involved in a loosely bound complex, providing insight into the nature of the ensembles formed by electron transfer protein complexes that is not available via other approaches. In addition to learning about the protein complex, we made strides toward developing and improving the new experimental approach to study highly dynamic behaviors of proteins.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2025

Physics-Informed Neural Network (PINN) Prediction of Mixed Mass-Heat-Crystallization Limited Methane Hydrate Formation and Dissociation in Micro-Confinement

The creation and use of Physics-Informed Neural Networks (PINNs) for simulating the dynamics of methane hydrate formation and dissociation will be presented. The PINN framework's main benefit is its capacity to impose physical consistency with only a partial comprehension of the governing equations. This makes the algorithm especially useful for systems with little experimental evidence or a lack of theoretical knowledge. A strong basis for forecasting methane hydrate behavior over the verified operating ranges of 30.0-80.9 bar pressure and 1.0-4.0 K sub-cooling conditions is provided by the combination of conductive heat transfer equations and mixed mass-transfer–crystallization kinetics. PINNs were more accurate at predicting the mixed mass-heat-crystallization limited kinetics than conventional Artificial Neural Networks (ANNs), demonstrating remarkable predictive accuracy for methane hydrate production over the ANN model. The efficiency of incorporating physical limitations from first principles into machine learning frameworks for methane hydrate crystallizations is reinforced by these findings. For hydrate-related applications in energy generation, carbon sequestration, and climate modelling, our study establishes PINNs as a computational tool that is both scalable and efficient. The proven capacity to close the gap between conventional physics-based simulations and solely data-driven models creates new opportunities for expedited hydrate research and practical applications.

Hartman, Ryan L [NYU Tandon School of Engineering]↗
NASA NTRSDate not supplied

The NASA Turbulent Heat Flux (THX) Experiments: Summary and Lessons Learned

The Turbulent Heat Flux (THX) experiments were conducted at NASA Glenn Research Center (GRC) in order to collect measurements of velocities and temperatures for computational fluid dynamics (CFD) validation of heated flows, with a focus on propulsion system components. The experiments spanned 5 phases; four of which were conducted in the GRC AeroAcoustic Propulsion Laboratory (AAPL) using the Small Hot Jet Flow Rig (SHJAR). In addition to making velocity measurements with Particle Image Velocimetry (PIV), the THX experiments introduced a new Raman-scattering based capability to measure temperatures. Computational studies were also conducted for each of the experimental configurations, in order to provide a baseline of expected CFD results and conduct an assessment of the capability of various CFD approaches for calculating flows where the turbulent transport of heat was important. Two of the collected sets of data were used for American Institute of Aeronautics and Astronautics (AIAA) Propulsion Aerodynamic Workshops (PAWs). The data set from the 5th phase, collected for heated supersonic jets, was also used to construct new validation cases for the NASA Turbulence Model Resource (TMR). This paper provides an overview of the experiments and associated computations for each of the 5 test phases. Key experimental findings are presented. Lessons learned are provided concerning the effect of computational modeling choice on accuracy of predicting turbulent flows where thermal transport is important. Emphasis is placed on comparing Reynolds-averaged Navier-Stokes approaches with large-eddy simulation approaches. The benefits of utilizing a conjugate heat transfer method in conjunction with CFD solver for film cooling is demonstrated.

RANS↗
DOE OSTI2024

High-Fidelity and High-Performance Computational Simulations for Rapid Design Optimization of Sulfur Thermal Energy Storage

Industrial process heating (IPH) accounts for approximately 70% of US manufacturing energy use and is primarily produced by fossil fuel combustion. Approximately 1500 TWht (approximately 60%) of IPH demand is in the temperature range of 100-300. Industrial applications in this temperature range include drying, hydrothermal processing, thermal enhanced oil recovery, food and beverage, bioethanol production, etc. Cost-effective thermal energy storage (TES) that increases the utilization of waste and renewable heat (solar, geothermal, etc.) could provide significant energy savings and reliable heat sources, decrease emissions, and increase US manufacturing competitiveness through reductions in fuel consumption. TES development has historically been dominated by technologies suitable for deployment with concentrating solar power (CSP). State-of-the-art thermal storage deployed commercially with power tower CSP plants uses a 60%/40% NaNO3/KNO3 molten salt and operates between temperatures of approximately 280 degrees Celsius and 570 degrees Celsius using a two-tank configuration. However, these nitrate salts are unsuitable for operation outside of this temperature range due to a high freezing point of approximately 220 degrees Celsius, and limits on high-temperature salt stability and corrosion resistance of containment alloys. Other materials being investigated for TES include those based on: (1) sensible energy storage (various molten salt compositions, inert solid particles, rocks or pebble beds, sulfur, water, concrete, graphite, etc.), (2) latent energy storage in materials that undergo solid-liquid phase change at relevant temperatures (organic materials for low-temperature applications, inorganic salts and/or metals for high-temperature applications), or (3) thermochemical energy storage (hydrides, hydroxides, carbonates, metal oxides, etc.). The application temperature and challenges pertaining to storage material and/or containment cost, energy density, long-term thermal and cyclic stability, and charge/discharge heat transfer effectiveness drive material selection for a given IPH or electricity generation application. Sulfur is a cheap commodity at $80/ton compared to $1100 - 1300/ton for conventional salts. When using a metric of storage cost per kWh, sulfur costs around 2-3 $/kWh. Previous sulfur TES development focused on high temperature (>600 degrees) concentrated solar power applications with sulfur encapsulated in pipes and flow of gaseous HTF (air) in the shell side. However, for lower-temperature IPH applications in the range of approximately 100-300 degrees Celsius Element 16 adopted a compact and scalable TES design with molten sulfur in the shell and HTF pipes submerged in the molten sulfur bath. The low-cost molten sulfur TES for dispatchable IPH has deployment potential for broad applications. The spatial and temporal evolution of the HTF and sulfur temperature is critical to the TES system performance, and thus detailed modeling can improve understanding of the performance and facilitate design improvements. Using high performance computing and computational fluid dynamics (CFD) a low-cost molten sulfur thermal energy storage (TES) system for industrial process heating (IPH) applications was developed. The unique challenges in CFD modeling of sulfur TES are the sharp property changes of sulfur relevant to the working temperatures. Above 159, liquid sulfur undergoes polymerization, and the viscosity of sulfur rapidly increases by several orders of magnitude between 159 degrees Celsius and 188 degrees Celsius, followed by a decrease in viscosity beyond 188 degrees Celsius due to thermal bound dissociation. In addition, various concentrations of H2S impurities can also modify sulfur viscosity. This numerical challenge is especially relevant to transient simulation of the sulfur TES charging and discharging processes as the extreme property variations limit the applicability of traditional heat transfer correlations. Transient CFD simulations including the temperature-dependent sulfur properties and geometric complexity of the TES design were used to predict the effect of natural convection during charging and discharging on the heat transfer process, sulfur temperature uniformity, charge/discharge rates, and performance of the storage devices. The CFD model was validated with experimental results for a full charge and discharge cycle. The work will show 3D and 2D simulation comparisons aimed to facilitate rapid design iterations and a machine learning based design optimization approach.

CFD↗
DOE OSTI2023

BuildingsBench: A Benchmark for Universal Building Load Forecasting [SWR-23-51]

The residential and commercial building stock in the United States is responsible for a significant percentage of energy consumption and greenhouse gas emissions. Electrification of end-uses, as well as decarbonizing the electrical grid through renewable energy sources such as solar and wind, constitutes the pathway to zero-emission buildings. Forecasting day-ahead building energy consumption is an integral part of this solution. Currently, specialized forecasting models are hand-made for each individual building, which is time-consuming, expensive, and leads to duplicated efforts. BuildingsBench is a Python software framework for training and comparing generalized machine learning models for universal building load forecasting. This challenge tasks a single foundational model to generalize its forecasts for a wide variety of buildings, across geographic regions, building types, weather patterns, and more. This software provide code for pre-training such models and subsequently evaluating their performance on a suite of hundreds of diverse real and synthetic buildings. BuildingsBench is a platform for: - Large-scale pretraining with the synthetic Buildings-900K dataset for short-term load forecasting (STLF). Buildings-900K is statistically representative of the entire U.S. building stock and is extracted from the NREL End-Use Load Profiles database. - Benchmarking on two tasks evaluating generalization: zero-shot STLF and transfer learning for STLF. We provide an index-based PyTorch Dataset for large-scale pretraining, easy data loading for multiple real building energy consumption datasets as PyTorch Tensors or Pandas DataFrames, simple (persistence) to advanced (transformer) baselines, metrics management, and more.

Emami, Patrick↗
DOE OSTI2025

You Only Look Once v5 and Multi-Template Matching for Small-Crack Defect Detection on Metal Surfaces

This paper compares the performance of Deep Learning (DL) and multi-template matching (MTM) models for detecting small defects. DL models extract distinguishing features of objects but require a large dataset of images. In contrast, alternative computer vision techniques like MTM need a relatively small dataset. The lack of large datasets for small metal-surface defects has inhibited the adoption of automation in small-defect detection in remanufacturing settings. This motivated this preliminary study to compare template-based approaches, like MTM, with feature-based approaches, such as DL models, for small-defect detection on an initial laboratory and remanufacturing industry dataset. This study used You Only Look Once v5 (YOLOv5) as the DL model and compared its performance against the MTM model for small-crack detection. The findings of our preliminary investigation are as follows: (i) YOLOv5 demonstrated higher performance than MTM in detecting small cracks; (ii) an extra-large variant of YOLOv5 outperformed a small-size variant; (iii) the size and object variety of the data are crucial in achieving robust pre-trained weights for use in transfer learning; and (iv) enhanced image resolution contributes to precise object detection.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗
DOE OSTI2020

Improving radiograph analysis throughput through transfer learning and object detection

SIGN Fracture Care International partners with surgeons in low-resource hospitals worldwide to provide access to effective orthopedic care by donating educational materials and innovatively designed surgical implants. Over two decades, SIGN’s Online Surgical Database (SOSD) has grown to contain over 500,000 medical images, with radiographs holding the majority share. One challenge in working with hospitals worldwide is that both the radiographs uploaded to the SOSD and the data entry accompanying the uploads vary in quality. To improve the accuracy of data in the SOSD, we trained a model to detect surgical implants in radiographs. We first developed a tool to automatically detect radiographs, then trained an object detection model to determine the number and placement of surgical implants visible in the radiograph. Active learning was used to generate a training set containing 2,510 radiographs with screws, nails, and plates labeled by bounding boxes. Training a model to simultaneously recognize all three classes of implants gave a low average precision (AP) for the plate class, likely due to the low number of plate instances in our training set and the large variety of surgical plates used by SIGN-partnered surgeons. Applying standard image augmentation techniques to increase the plate count in our training set did not appreciably increase the AP of plate detection. To improve plate detection, we redrew the bounding boxes to account for correlations between the screw and plate classes. Training one model to detect nails and screws and a separate model to detect plates increased the AP of plate detection by 78.8 percentage points. The AP of each class was 80.7% for screws, 93.6% for nails, and 92.6% for plates; meanwhile, the sensitivity was 92% for screws, 86% for nails, and 81% for plates. We show that object detection methods can be used to detect surgical implants in radiographs of varying quality; however, the detection ability is dependent on the type of implant, and some implants, in our case plates, must be treated differently than others. Such tools can improve the throughput of radiograph analysis, assisting physicians and surgeons with the treatment of bone fractures.

60 APPLIED LIFE SCIENCES↗
DOE OSTI2022

Estimating cluster masses from SDSS multiband images with transfer learning

ABSTRACT The total masses of galaxy clusters characterize many aspects of astrophysics and the underlying cosmology. It is crucial to obtain reliable and accurate mass estimates for numerous galaxy clusters over a wide range of redshifts and mass scales. We present a transfer-learning approach to estimate cluster masses using the ugriz-band images in the SDSS Data Release 12. The target masses are derived from X-ray or SZ measurements that are only available for a small subset of the clusters. We designed a semisupervised deep learning model consisting of two convolutional neural networks. In the first network, a feature extractor is trained to classify the SDSS photometric bands. The second network takes the previously trained features as inputs to estimate their total masses. The training and testing processes in this work depend purely on real observational data. Our algorithm reaches a mean absolute error (MAE) of 0.232 dex on average and 0.214 dex for the best fold. The performance is comparable to that given by redMaPPer, 0.192 dex. We have further applied a joint integrated gradient and class activation mapping method to interpret such a two-step neural network. The performance of our algorithm is likely to improve as the size of training data set increases. This proof-of-concept experiment demonstrates the potential of deep learning in maximizing the scientific return of the current and future large cluster surveys.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗