Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Transfer Learning Model”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 415 records · Page 23
All sourcesNASA NTRSDOE OSTI
DOE OSTI2023

INL Senior Project

What did my team set out to accomplish:? Can I put a custom Machine Learning Model on FPGA?? Can I analyze network traffic in real time?? Can a QSFP port be used with an FPGA?? Does a visual representation of the latent space enhance our understanding of network traffic?? What is QSFP QSFP (Quad Small Form-Factor Pluggable)? QSFP supports transfer speeds generally up to 100Gb/s? Runs 4 parallel lines running up to 28 Gb/s? Why the latent space is important to our project ?Latent space is the compressed mapping of data points in a non-linear fashion? Create an understanding of the relationship of data collected? Can represent that relationship of a single network packet in 3 points (X, Y, Z) Project Outline FPGA? Custom Xilinx Petalinux Image for the operating system? Python program to collect packets and run them through the DPU (Data Processing Unit)? The program then sends the information over a socket to a computer? Display Program? Python Program that collects the information sent from the FPGA and display it in a graph

99 GENERAL AND MISCELLANEOUS↗
DOE OSTI2024

Scaling Ensembles of Data-Intensive Quantum Chemical Calculations for Millions of Molecules

Deep learning models are efficient computational tools that can accelerate the inverse design of molecules with desired functional properties by generating predictions at a fraction of the time required by traditional quantum chemical approaches. To ensure that a model maintains accuracy and transferability across broad regions of the chemical space explored during the inverse design, it must be trained on massively large volumes of simulation data. This requires running large-scale ensemble quantum chemical calculations on high-performance computing (HPC) systems for data collection. However, the efficient execution of such large ensemble calculations and the management of large volumes of output data require tools that can judiciously utilize computational resources and manage metadata overhead on the file system. Therefore, we present a high-performance, scalable, ensemble management framework for performing data-intensive quantum chemical electronic structure calculations for organic molecules. This framework provides abstractions to plug different ab initio, first principles, and first principles-based semi-empirical methods and executes them efficiently at large scale on HPC systems. It dynamically distributes tasks to resources and uses tiered storage for managing large collections of files. We employed this framework to process over ten million organic molecules and generate open-source datasets that provide UV-vis absorption spectra by running time-dependent density-functional tight-binding calculations. It is the largest database containing molecular optical spectra that were simulated with quantum chemical methods in a consistent manner.

Mehta, Kshitij↗
DOE OSTI2024

Scaling whole-chip QAOA for higher-order ising spin glass models on heavy-hex graphs

Abstract We show that the quantum approximate optimization algorithm (QAOA) for higher-order, random coefficient, heavy-hex compatible spin glass Ising models has strong parameter concentration across problem sizes from 16 up to 127 qubits for p = 1 up to p = 5, which allows for computationally efficient parameter transfer of QAOA angles. Matrix product state (MPS) simulation is used to compute noise-free QAOA performance. Hardware-compatible short-depth QAOA circuits are executed on ensembles of 100 higher-order Ising models on noisy IBM quantum superconducting processors with 16, 27, and 127 qubits using QAOA angles learned from a single 16-qubit instance using the JuliQAOA tool. We show that the best quantum processors find lower energy solutions up to p = 2 or p = 3, and find mean energies that are about a factor of two off from the noise-free distribution. We show that p = 1 QAOA energy landscapes remain very similar as the problem size increases using NISQ hardware gridsearches with up to a 414 qubit processor.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
DOE OSTI2020

Minimal Experimental Bias on the Hydrogen Bond Greatly Improves Ab Initio Molecular Dynamics Simulations of Water

Experiment Directed Simulations (EDS) is a method within a class of techniques seeking to improve molecular simulations by minimally biasing the system Hamiltonian to reproduce certain experimental observables. In a previous application of EDS to ab initio molecular dynamics (AIMD) simulation based on electronic density functional theory (DFT), the AIMD simulations of water were biased to reproduce its experimentally derived solvation structure. In particular, by solely biasing the O-O pair correlation functions, other structural and dynamical properties that were not biased were improved. In this work, the hypothesis is tested that directly biasing the OH pair correlation (and hence the H-O∙∙∙H hydrogen bonding), will provide an even better improvement of DFT-based water properties in AIMD simulations. The logic behind this hypothesis is that for most electronic DFT descriptions of water the hydrogen bonding is known to be deficient due to anomalous charge transfer and over polarization in the DFT. Using recent advances to the EDS learning algorithm, we thus train a minimal bias on AIMD water that reproduces the O-H radial distribution function derived from the highly accurate MB-pol model of water. Finally, it is then confirmed that biasing the O-H pair correlation alone can lead to improved AIMD water properties, with structural and dynamical properties in even closer to experiment than the previous EDS-AIMD model.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2024

Investigating resource-efficient neutron/gamma classification ML models targeting eFPGAs

There has been considerable interest and resulting progress in implementing machine learning (ML) models in hardware over the last several years from the particle and nuclear physics communities. A big driver has been the release of the Python package, hls4ml, which has enabled porting models specified and trained using Python ML libraries to register transfer level (RTL) code. So far, the primary end targets have been commercial field-programmable gate arrays (FPGAs) or synthesized custom blocks on application specific integrated circuits (ASICs). However, recent developments in open-source embedded FPGA (eFPGA) frameworks now provide an alternate, more flexible pathway for implementing ML models in hardware. These customized eFPGA fabrics can be integrated as part of an overall chip design. In general, the decision between a fully custom, eFPGA, or commercial FPGA ML implementation will depend on the details of the end-use application. In this work, we explored the parameter space for eFPGA implementations of fully-connected neural network (fcNN) and boosted decision tree (BDT) models using the task of neutron/gamma classification with a specific focus on resource efficiency. We used data collected using an AmBe sealed source incident on Stilbene, which was optically coupled to an OnSemi J-series silicon photomultiplier (SiPM) to generate training and test data for this study. We investigated relevant input features and the effects of bit-resolution and sampling rate as well as trade-offs in hyperparameters for both ML architectures while tracking total resource usage. The performance metric used to track model performance was the calculated neutron efficiency at a gamma leakage of 10 -3 . The results of the study will be used to aid the specification of an eFPGA fabric, which will be integrated as part of a test chip.

47 OTHER INSTRUMENTATION↗
DOE OSTI2026

Explainable machine learning for incipient anomaly detection in compact molten salt heat exchanger with overlapping feature distributions

High-temperature molten salt-cooled reactors (MSCRs) are a promising next-generation nuclear technology option, offering efficient power conversion and inherent safety features. However, the reliability of these systems depends on the robust operation of heat exchangers (HXs), which are susceptible to failure due to temperature gradients and channel plugging caused by fluid freezing. Conventional monitoring methods, relying on inlet and outlet measurements, lack the spatial resolution needed to detect early-stage faults. We propose a novel design of a compact salt-to-salt matrix-type HX design consisting of interleaved arrays of parallel tubes, with integrated synthetic fiber optic distributed temperature sensing (DTS) to enable localized detection of incipient faults. To evaluate performance of this design, we generate high-fidelity synthetic data using heat transfer computational modeling to simulate channel plugging, and introduce sensor noise for realistic modeling of measurements. The dataset comprises of 97% normal operation and 3% anomaly cases, with each anomaly class representing 1% of the data. These early anomalies result in overlapping temperature profiles between normal and faulty channels, producing a non-separable dataset that challenges traditional classification techniques. We benchmark eight supervised machine learning (ML) models and demonstrate that XGBoost achieves the highest performance. To improve transparency, we develop an explainability framework combining Shapley values and partially ordered sets (POSETs) to quantify and structurally analyze feature importance. This approach identifies both dominant predictors and ambiguous feature relationships, enhancing trust and interpretability. Our results highlight the potential of combining DTS and explainable ML with intelligent feature selection to improve predictive maintenance and ensure operational resilience in advanced nuclear systems.

Prantikos, Konstantinos [Argonne National Laborato↗
DOE OSTI2021

Physics-Guided Machine Learning for Prediction of Cloud Properties in Satellite-Derived Solar Data

With over 20 years of high-resolution surface irradiance data covering most of the western hemisphere, the National Solar Radiation Database (NSRDB) is a vital public data asset. The NSRDB uses a two-step Physical Solar Model (PSM) that explicitly considers the effects of clouds and other atmospheric variables on radiative transfer. High-quality physical and optical cloud properties derived from satellite imagery are perhaps the most important data inputs to the PSM, representing the greatest source of radiation attenuation and scattering. However, traditional methods for cloud property retrieval have their own limitations and are unable to accurately predict cloud properties outside of nominal conditions. We introduce a physics-guided neural network that can accurately predict cloud properties when traditional methods fail or are inaccurate. Using this framework, we show reductions in relative Root Mean Square Error (RMSE) for Global Horizontal Irradiance (GHI) up to 13 percentage points for timesteps that previously had missing or low-quality cloud property data. We expect that this methodology will be effective in improving the quality of cloud property and solar irradiance data in the NSRDB.

cloud properties↗
DOE OSTI2026

Differentiable multiphase flow model for physics-informed machine learning in reservoir pressure management

Accurate subsurface reservoir pressure control is extremely challenging due to geological heterogeneity and multiphase fluid-flow dynamics. Predicting behavior in this setting relies on high-fidelity physics-based simulations that are computationally expensive. Yet, the uncertain, heterogeneous properties that control these flows make it necessary to perform many of these expensive simulations, which is often prohibitive. To address these challenges, we introduce a physics-informed machine learning workflow that couples a fully differentiable multiphase flow simulator, which is implemented in the DPFEHM framework with a convolutional neural network (CNN). The CNN learns to predict fluid extraction rates from heterogeneous permeability fields to enforce pressure limits at critical reservoir locations. By incorporating transient multiphase flow physics into the training process, our method enables more practical and accurate predictions for realistic injection-extraction scenarios compared to previous works. To speed up training, we pretrain the model on single-phase, steady-state simulations and then finetune it on full multiphase scenarios, which dramatically reduces the computational cost. We demonstrate that high-accuracy training can be achieved with fewer than three thousand full-physics multiphase flow simulations – compared to previous estimates requiring up to ten million. This drastic reduction in the number of simulations is achieved by leveraging transfer learning from much less expensive single phase simulations.

25 ENERGY STORAGE↗
DOE OSTI2020

RouteE: A Vehicle Energy Consumption Prediction Engine

The emergence of connected and automated vehicles and smart cities technologies create the opportunity for new mobility modes and routing decision tools, among many others. To achieve maximum mobility and minimum energy consumption, it is critical to understand the energy cost of decisions and optimize accordingly. The Route Energy prediction model (RouteE) enables accurate estimation of energy consumption for a variety of vehicle types over trips or sub-trips where detailed drive cycle data are unavailable. Applications include vehicle route selection, energy accounting and optimization in transportation simulation, and corridor energy analyses, among others. The software is a Python package that includes a variety of pre-trained models from the National Renewable Energy Laboratory (NREL). However, RouteE also enables users to train custom models using their own data sets, making it a robust and valuable tool for both fast calculations and rigorous, data-rich research efforts. The pre-trained RouteE models are established using NREL’s Future Automotive Systems Technology Simulator paired with approximately 1 million miles of drive cycle data from the Transportation Secure Data Center, resulting in energy consumption behavior estimates over a representative sample of driving conditions for the United States. Validations have been performed using on-road fuel consumption data for conventional and electrified vehicle powertrains. Transferring the results of the on-road validation to a larger set of real-world origin-destination pairs, it is estimated that implementing the present methodology in a green-routing application would accurately select the route that consumes the least fuel 90% of the time. The novel machine learning techniques used in RouteE make it a flexible and robust tool for a variety of transportation applications.

33 ADVANCED PROPULSION SYSTEMS↗
DOE OSTI2020

Effects of Jacobian Matrix Regularization on the Detectability of Adversarial Samples

The well-known vulnerability of Deep Neural Networks to adversarial samples has led to a rapid cycle of increasingly sophisticated attack algorithms and proposed defenses. While most contemporary defenses have been shown to be vulnerable to carefully configured attacks, methods based on gradient regularization and out-of-distribution detection have attracted much interest recently by demonstrating higher resilience to a broad range of attack algorithms. However, no study has yet investigated the effect of combining these techniques. In this paper, we consider the effect of Jacobian matrix regularization on the detectability of adversarial samples on the CIFAR-10 image benchmark dataset. We find that regularization has a significant effect on detectability, and in some cases can make an undetectable attack on a baseline model detectable. In addition, we give evidence that regularization may mitigate the known weaknesses of detectors to high-confidence adversarial samples. The defenses we consider here are highly generalizable, and we believe they will be useful for further investigations to transfer machine learning robustness to other data domains.

45 MILITARY TECHNOLOGY, WEAPONRY, AND NATIONAL DEF↗
DOE OSTI2020

On Performance Prediction of Big Data Transfer in High-performance Networks

Big data generated by large-scale scientific and industrial applications need to be transferred between different geographical locations for remote storage, processing, and analysis. High-speed dedicated connections provisioned in High-performance Networks (HPNs) are increasingly utilized to carry out such big data transfer. HPN management highly relies on an important capability of performance (mainly throughput) prediction to reserve sufficient bandwidth and meanwhile avoid over-provisioning that may result in unnecessary resource waste. This capability is critical to improving the resource (mainly bandwidth) utilization of dedicated connections and meeting various user requests for data transfer. Conventional methods conduct performance prediction by fitting prior observed transfer history with predefined loss functions, without considering unobservable latent factors such as competing loads on end hosts. Such latent factors also have a significant impact on the application-level data transfer performance, which may result in an inaccurate prediction model. In this paper, we first investigate the impact of latent factors and propose a clustering-based method to eliminate their negative impact on performance prediction. We then develop a robust machine learning-based performance predictor by: i) incorporating the proposed latent factor elimination method into data preprocessing, and ii) adopting a customized domain guided loss function. Extensive experimental results show that our predictor achieves significantly higher prediction accuracy than several other state-of-the-art methods.

Liu, Wuji↗
DOE OSTI2021

Physics-Guided Machine Learning for Prediction of Cloud Properties in Satellite-Derived Solar Data: Preprint

With over 20 years of high-resolution surface irradiance data covering most of the western hemisphere, the National Solar Radiation Database (NSRDB) is a vital public data asset. The NSRDB uses a two-step Physical Solar Model (PSM) that explicitly considers the effects of clouds and other atmospheric variables on radiative transfer. High-quality physical and optical cloud properties derived from satellite imagery are perhaps the most important data input to the PSM, representing the greatest source of radiation attenuation and scattering. However, traditional methods for cloud property retrieval have their own limitations and are unable to accurately predict cloud properties outside of nominal conditions. We introduce a physics-guided neural network that can accurately predict cloud properties when traditional methods fail or are inaccurate. Using this framework, we show reductions in relative Root Mean Square Error (RMSE) for Global Horizontal Irradiance (GHI) up to 13 percentage points for timesteps that previously had missing or low-quality cloud property data. We expect that this methodology will be effective in improving the quality of cloud property and solar irradiance data in the NSRDB.

41 EE - Solar Energy Technologies Office (EE-4S)↗
DOE OSTI2024

RuralAI in Tomato Farming: Integrated Sensor System, Distributed Computing, and Hierarchical Federated Learning for Crop Health Monitoring

Precision horticulture is evolving due to scalable sensor deployment and machine learning (ML) integration. These advancements boost the operational efficiency of individual farms, balancing the benefits of analytics with autonomy requirements. However, given concerns that affect wide geographic regions (e.g., climate change), there is a need to apply models that span farms. Federated learning (FL) has emerged as a potential solution. FL enables decentralized ML across different farms without sharing private data. Traditional FL assumes simple two-tier network topologies and, thus, falls short of operating on more complex networks found in real-world agricultural scenarios. Networks vary across crops and farms and encompass various sensor data modes, extending across jurisdictions. New hierarchical FL (HFL) approaches are needed for more efficient and context-sensitive model sharing, accommodating regulations across multiple jurisdictions. Here, we present the RuralAI architecture deployment for tomato crop monitoring, featuring sensor field units for soil, crop, and weather data collection. HFL with personalization is used to offer localized and adaptive insights. Model management, aggregation, and transfers are facilitated via a flexible approach, enabling seamless communication between local devices, edge nodes, and the cloud.

60 APPLIED LIFE SCIENCES↗
DOE OSTI2021

Analysis-Specific Fast Simulation at the LHC with Deep Learning

Abstract We present a fast-simulation application based on a deep neural network, designed to create large analysis-specific datasets. Taking as an example the generation of W + jet events produced in $$\sqrt{s}=$$ s = 13 TeV proton–proton collisions, we train a neural network to model detector resolution effects as a transfer function acting on an analysis-specific set of relevant features, computed at generation level, i.e., in absence of detector effects. Based on this model, we propose a novel fast-simulation workflow that starts from a large amount of generator-level events to deliver large analysis-specific samples. The adoption of this approach would result in about an order-of-magnitude reduction in computing and storage requirements for the collision simulation workflow. This strategy could help the high energy physics community to face the computing challenges of the future High-Luminosity LHC.

Chen, C.↗
DOE OSTI2020

Active Learning A Neural Network Model For Gold Clusters & Bulk From Sparse First Principles Training Data

Small metal clusters are of fundamental scientific interest and of tremendous significance in catalysis. These nanoscale clusters display diverse geometries and structural motifs depending on the cluster size; a knowledge of this size-dependent structural motifs and their dynamical evolution has been of longstanding interest. Given the high computational cost of first-principles calculations, molecular modeling and atomistic simulations such as molecular dynamics (MD) has proven to be an important complementary tool to aid this understanding. Classical MD typically employ predefined functional forms which limits their ability to capture such complex size-dependent structural and dynamical transformation. Neural Network (NN) based potentials represent flexible alternatives and in principle, well-trained NN potentials can provide high level of flexibility, transferability and accuracy on-par with the reference model used for training. A major challenge, however, is that NN models are interpolative and requires large quantities (similar to 10 4 or greater) of training data to ensure that the model adequately samples the energy landscape both near and far-from-equilibrium. A highly desirable goal is minimize the number of training data, especially if the underlying reference model is first-principles based and hence expensive. In this work, we introduce an active learning (AL) scheme that trains a NN model on-the-fly with minimal amount of first-principles based training data. Our AL workflow is initiated with a sparse training dataset (similar to 1 to 5 data points) and is updated on-the-fly via a Nested Ensemble Monte Carlo scheme that iteratively queries the energy landscape in regions of failure and updates the training pool to improve the network performance. Using a representative system of gold clusters, we demonstrate that our AL workflow can train a NN with similar to 500 total reference calculations. Using an extensive DFT test set of similar to 1100 configurations, we show that our AL-NN is able to accurately predict both the DFT energies and the forces for clusters of a myriad of different sizes. Our NN predictions are within 30 meV/atom and 40 meV/angstrom of the reference DFT calculations. Moreover, our AL-NN model also adequately captures the various size-dependent structural and dynamical properties of gold clusters in excellent agreement with DFT calculations and available experiments. We finally show that our AL-NN model also captures bulk properties reasonably well, even though they were not included in the training data.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2026

Sim-to-real supervised domain adaptation for radioisotope identification

Machine learning has the potential to improve the speed and reliability of radioisotope identification using gamma spectroscopy. However, meticulously labeling an experimental dataset for training is often prohibitively expensive, while training models purely on synthetic data is risky due to the domain gap between simulated and experimental measurements. In this research, we demonstrate that supervised domain adaptation can substantially improve the performance of radioisotope identification models by transferring knowledge between synthetic and experimental data domains. We consider two domain adaptation scenarios: (1) a simulation-to-simulation adaptation, where we perform multi-label proportion estimation using simulated high-purity germanium detectors, and (2) a simulation-to-experimental adaptation, where we perform multi-class, single-label classification using measured spectra from handheld lanthanum bromide (LaBr) and sodium iodide (NaI) detectors. We begin by pretraining a spectral classifier on synthetic data using a custom transformer-based neural network. After subsequent fine-tuning on just 64 labeled experimental spectra, we achieve a test accuracy of 96% in the sim-to-real scenario with a LaBr detector, far surpassing a synthetic-only baseline model (75%) and a model trained from scratch (80%) on the same 64 spectra. Furthermore, we demonstrate that domain-adapted models learn more human-interpretable features than experiment-only baseline models. Overall, our results highlight the potential for supervised domain adaptation techniques to bridge the sim-to-real gap in radioisotope identification, enabling the development of accurate and explainable classifiers even in real-world scenarios where access to experimental data is limited.

Lalor, Peter W.↗
DOE OSTI2023

CoolPINNs: A physics-informed neural network modeling of active cooling in vascular systems

Emerging technologies like hypersonic aircraft, space exploration vehicles, and batteries avail fluid circulation in embedded microvasculatures for efficient thermal regulation. Modeling is vital during the design and operational phases of these engineered systems. However, many challenges exist in developing a modeling framework. What is lacking is an accurate framework that (i) captures sharp jumps in the thermal flux across complex vasculature layouts, (ii) deals with oblique derivatives (involving tangential and normal components), (iii) handles nonlinearity because of radiative heat transfer, (iv) provides a high-speed forecast for real-time monitoring, and (v) facilitates robust inverse modeling. Here, this paper addresses these challenges by availing the power of physics-informed neural networks (PINNs). We develop a fast, reliable, and accurate Scientific Machine Learning (SciML) framework for vascular-based thermal regulation—called CoolPINNs: a PINNs-based modeling framework for active cooling. The proposed mesh-less framework elegantly overcomes all the mentioned challenges. The significance of the reported research is multi-fold. First, the framework is valuable for real-time monitoring of thermal regulatory systems because of rapid forecasting. Second, researchers can address complex thermoregulation designs since the approach is meshless. Finally, the framework facilitates systematic parameter identification and inverse modeling studies, perhaps the most significant utility of the current framework.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2020

Residue Specific Characterization of Electrostatics, Conformational Heterogeneity, and Dynamics of Electron Transfer Proteins with Linear and Nonlinear Infrared Spectroscopy. Final Report

The goal of this research project was to further our understanding of biological electron transfer through the development and application of new methods for studying proteins and their complexes with high spatial and temporal detail. The approach was based on incorporation of reporter bonds with distinct vibrational frequencies at specific locations in proteins and their characterization through the interaction of the bonds with pulses of infrared light. The project focused on the proteins plastocyanin (Pc) and cytochrome (cyt) f, two electron transfer partners in photosynthesis. The first thrust of the project was the incorporation of carbon deuterium bonds as non-perturbative probes at a ligand to the copper ion of Pc and their use to detect changes in the bonding due to the number of electrons at the metal and how the bonding changes upon association with cyt f. Toward developing more advanced approaches to characterize proteins, we incorporated a stronger reporter, the cyano group of cyanophenylalanine, at several locations in Pc and measured the frequency dynamics in the absence and presence of cyt f. The acquired data improved the current model for the protein complex. Notably, the data indicated that the majority of the population is involved in a loosely bound complex, providing insight into the nature of the ensembles formed by electron transfer protein complexes that is not available via other approaches. In addition to learning about the protein complex, we made strides toward developing and improving the new experimental approach to study highly dynamic behaviors of proteins.

59 BASIC BIOLOGICAL SCIENCES↗