Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Deep learning”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,477 records · Page 82
All sourcesNASA NTRSDOE OSTI
DOE OSTI2025

Day-Ahead Probabilistic Forecasting of Net-Load and Demand Response Potentials with High Penetration of Behind-the-Meter Solar-plus-Storage

The goal of this project is to develop advanced methods for day-ahead net-load forecasting, by leveraging the state-of-the-art machine learning techniques. The developed models produce both point and probabilistic forecasts for a variety of use cases, and are versatile to work with different types of data sets. The innovation lies in the novel design of the architectures, leveraging the most recent advances in machine learning that have not been explored in power systems, accompanied by techniques in the broader artificial intelligence fields such as fuzzy systems. This project has achieved the following accomplishments: (1) preprocessing of over 10 data sets covering varying geographical regions, time horizons, and system levels, which form a robust foundation for training and evaluating forecasting models across a wide range of realistic grid scenarios; (2) development of an interactive web app that enables exploratory analysis of load and generation data, and supports better understanding of data trends, anomalies, and correlations, facilitating model development and stakeholder engagement; (3) implementation of over 10 benchmark models for point and probabilistic forecasting, which include a mix of conventional machine learning methods and state-of-the-art deep learning approaches, providing a comprehensive baseline for performance comparison and validation of the proposed models; (4) development of a fuzzy system based gradient boosting model, tailored for small (less than 3 years) data sets, which achieves a mean absolute percentage error (MAPE) of 4% for point forecasting and a 20% improvement in average pinball loss for probabilistic forecasting; (5) development of a Transformer (a state-of-the-art deep learning architecture) based neural network model, tailored for large (3 years or more) data sets, which achieves a MAPE of 2% for point forecasting and a 20% improvement in average pinball loss for probabilistic forecasting; (6) development of a methodology for quantifying DR potential, and extensions of the previous models for multi-target forecasting of net load and DR potential, which achieve a MAPE of 10% for DR potential.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2025

DeepAndes: A Self-Supervised Vision Foundation Model for Multispectral Remote Sensing Imagery of the Andes

By mapping sites at large scales usingremotely sensed data, archaeologists can generate unique insights into long-term demographic trends, interregional social networks, and human adaptations in the past. Remote sensing surveys complement field-based approaches, and their reach can be especially great when combined with deep learning and computer vision techniques. However, conventional supervised deep learning methods face challenges in annotating fine-grained archaeological features at scale. In addition, while recent vision foundation models have shown remarkable success in learning large-scale remote sensing data with minimal annotations, most off-the-shelf solutions are designed for RGB images rather than multispectral satellite imagery, such as the eight-band data used in our study. In this article, we introduce DeepAndes, a transformer-based vision foundation model trained on three million multispectral satellite images, specifically tailored for Andean archaeology. DeepAndes incorporates a customized DINOv2 self-supervised learning algorithm optimized for eight-band multispectral imagery, marking the first foundation model designed explicitly for the Andes region. We evaluate its image understanding performance through imbalanced image classification, image instance retrieval, and pixel-level semantic segmentation tasks. Our experiments show that DeepAndes achieves superior F1 scores, mean average precision, and Dice scores in few-shot learning scenarios, significantly outperforming models trained from scratch or pretrained on smaller datasets. This underscores the effectiveness of large-scale self-supervised pretraining in archaeological remote sensing.

Guo, Junlin [Vanderbilt Univ., Nashville, TN (Unit↗
DOE OSTI2026

Machine learning enables reconstruction of past fire regimes from charcoal-derived fire intensity and fuel composition

Background Fire is a foundational ecological process that shapes ecosystem structure, diversity, and resilience. Quantifying paleofire regime attributes such as frequency, severity, and intensity is essential for understanding the historical range of variability in fire behavior and its ecological effects. While frequency and severity are often reconstructed in paleofire studies, quantitative reconstructions of fire intensity remain limited. Recent work has shown that maximum pyrolysis temperature—a proxy for fire intensity—and plant species type can be inferred from charcoal using transmission Fourier-transform infrared (FTIR) spectroscopy. However, the sample preparation for transmission FTIR is destructive and time-consuming, limiting application and reuse of materials for other analyses. We evaluated reflectance FTIR spectroscopy as a non-destructive alternative for reconstructing combustion temperature and plant species from laboratory-generated charcoal. We also examined the influence of contrasting airflow environments (ambient air versus nitrogen-rich) on pyrolysis temperature and plant species reconstruction prediction accuracies and compared predictive performance between a novel, neural network–based deep learning model with the traditional modern analogue technique (MAT) using k-nearest neighbor functions. As proof of concept, we apply our enhanced methodology to ancient charcoal to demonstrate applicability at improving long-term fire regime reconstructions and the ability to link paleofire records with contemporary fire ecology. Results Our analysis shows that transmission and reflectance FTIR spectra yield comparable spectral profiles. However, sample preparation for reflectance FTIR is minimal and non-destructive, unlike transmission FTIR which is destructive. We demonstrate that oxygen environments improved reconstruction accuracy relative to nitrogen-rich conditions. Finally, our deep learning neural network (DL) achieved testing accuracies of 98.7% for temperature and 96.2% for species identification, outperforming MAT’s k-NN approach (89.8% and 65.9%, respectively). A Shapley importance analysis identified 5 key spectral regions that greatly influenced the model’s temperature or species categorization. When applied to ancient charcoal, our results show historic fires from the most recent past primarily burned at low intensities (400–500 °C), reflective of natural fire regimes in ponderosa pine forests. Our results corroborate charcoal morphology data that suggests all ancient charcoal originated from burned woody plant types. Conclusions By combining reflectance FTIR spectroscopy with a deep learning approach, we provide the first accuracies high enough to confidently identify both species and temperature from laboratory-produced charcoal, improving quantitative reconstructions of fire intensity and fuel composition from paleofire records. This opens a wide range of research into the link between fire and larger drivers (i.e., climate or human) and greater ecological understanding of fire regimes beyond that of burn scars or recent observations. These methodological improvements have direct relevance for fire management by improving interpretation of historical fire behavior, informing fuel–fire relationships, and providing a scalable analytical framework applicable to both long-term ecological studies and contemporary fire science.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2021

Identifying Genomic Islands with Deep Neural Networks

Background Horizontal gene transfer is the main source of adaptability for bacteria, through which genes are obtained from different sources including bacteria, archaea, viruses, and eukaryotes. This process promotes the rapid spread of genetic information across lineages, typically in the form of clusters of genes referred to as genomic islands (GIs). Different types of GIs exist, and are often classified by the content of their cargo genes or their means of integration and mobility. While various computational methods have been devised to detect different types of GIs, no single method is capable of detecting all types. Results We propose a method, which we call Shutter Island, that uses a deep learning model (Inception V3, widely used in computer vision) to detect genomic islands. The intrinsic value of deep learning methods lies in their ability to generalize. Via a technique called transfer learning, the model is pre-trained on a large generic dataset and then re-trained on images that we generate to represent genomic fragments. We demonstrate that this image-based approach generalizes better than the existing tools. Conclusions We used a deep neural network and an image-based approach to detect the most out of the correct GI predictions made by other tools, in addition to making novel GI predictions. The fact that the deep neural network was re-trained on only a limited number of GI datasets and then successfully generalized indicates that this approach could be applied to other problems in the field where data is still lacking or hard to curate.

Computer Vision↗
DOE OSTI2019

Phase Segmentation in Atom-Probe Tomography Using Deep Learning-Based Edge Detection

Abstract Atom-probe tomography (APT) facilitates nano- and atomic-scale characterization and analysis of microstructural features. Specifically, APT is well suited to study the interfacial properties of granular or heterophase systems. Traditionally, the identification of the interface between, for precipitate and matrix phases, in APT data has been obtained either by extracting iso-concentration surfaces based on a user-supplied concentration value or by manually perturbing the concentration value until the iso-concentration surface qualitatively matches the interface. These approaches are subjective, not scalable, and may lead to inconsistencies due to local composition inhomogeneities. We introduce a digital image segmentation approach based on deep neural networks that transfer learned knowledge from natural images to automatically segment the data obtained from APT into different phases. This approach not only provides an efficient way to segment the data and extract interfacial properties but does so without the need for expensive interface labeling for training the segmentation model. We consider here a system with a precipitate phase in a matrix and with three different interface modalities—layered, isolated, and interconnected—that are obtained for different relative geometries of the precipitate phase. We demonstrate the accuracy of our segmentation approach through qualitative visualization of the interfaces, as well as through quantitative comparisons with proximity histograms obtained by using more traditional approaches.

47 OTHER INSTRUMENTATION↗
DOE OSTI2024

TNet: A Model-Constrained Tikhonov Network Approach for Inverse Problems

Deep learning (DL), in particular deep neural networks, by default is purely data-driven and in general does not require physics. This is the strength of DL but also one of its key limitations when applied to science and engineering problems in which underlying physical properties—such as stability, conservation, and positivity—and accuracy are required. DL methods in their original forms are often not capable of respecting the underlying mathematical models or achieving desired accuracy even in big-data regimes. On the other hand, many data-driven science and engineering problems, such as inverse problems, typically have limited experimental or observational data, and DL would overfit the data in this case. Leveraging information encoded in the underlying mathematical models, we argue, not only compensates for missing information in low data regimes but also provides opportunities to equip DL methods with the underlying physics, hence promoting better generalization. This paper develops a model-constrained DL approach and its variant TNet—a Tikhonov neural network—which are capable of learning not only information hidden in the training data but also in the underlying mathematical models to solve inverse problems governed by partial differential equations in low data regimes. We provide the constructions and some theoretical results for the proposed approaches for both linear and nonlinear inverse problems. Since TNet is designed to learn inverse solutions with Tikhonov regularization, it is interpretable: in fact it recovers Tikhonov solutions for linear cases while potentially approximating Tikhonov solutions for nonlinear inverse problems. We also prove that data randomization can enhance not only the smoothness of the networks but also their generalizations. Comprehensive numerical results confirm the theoretical findings and show that with even as little as 1 training data sample for one-dimensional (1D) deconvolution, 5 for an inverse 2D heat conductivity problem, 100 for inverse initial conditions for a time-dependent 2D Burgers’s equation, and 50 for inverse initial conditions for 2D Navier–Stokes equations, TNet solutions can be as accurate as Tikhonov solutions while being several orders of magnitude faster. Furthermore, this is possible owing to the model-constrained term, replications, and randomization.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2025

A physics-constrained neural ordinary differential equations approach for robust learning of stiff chemical kinetics

The high computational cost associated with solving for detailed chemistry poses a significant challenge for predictive computational fluid dynamics (CFD) simulations of turbulent reacting flows. While deep learning techniques have been explored to develop faster surrogate models, they often fail to integrate reliably with CFD solvers. This instability arises because traditional deep learning approaches optimize for training error without ensuring compatibility with ordinary differential equation (ODE) solvers, resulting in accumulation of errors over time. Recently, neuralODE (NODE) based approaches have been shown to be a promising technique to emulate and accelerate detailed chemistry computations. Here, in the present work, we extend this NODE framework for stiff chemical kinetics by incorporating mass conservation constraints directly into the loss function during training. This ensures that the total mass as well as the individual elemental species masses are conserved in an a-posteriori manner. Proof-of-concept studies are performed with the novel physics-constrained NODE (PC-NODE) approach for homogeneous autoignition of hydrogen-air mixture over a range of composition and thermodynamic conditions. It is demonstrated that the PC-NODE framework not only improves the physical consistency of the resulting data-driven model with respect to mass conservation criteria, but also improves training efficiency. PC-NODE is shown to achieve 2–100× speedup relative to the hydrogen-air detailed chemical mechanism depending on the type of the ODE solver (implicit or explicit) used during autoregressive inference tests. Lastly, a-posteriori studies are performed wherein the trained PC-NODE model is coupled with a CFD solver. It is shown that higher accuracy is achieved with PC-NODE relative to the purely data-driven NODE approach. Moreover, PC-NODE also exhibits robustness and generalizability to unseen initial conditions from within (interpolative capability) as well as outside (extrapolative capability) the training regime.

computational combustion↗
DOE OSTI2024

DDStore

DDStore is designed to addresses random, read-oriented, global shuffle operations for data-intensive deep learning applications. By preserving global shuffling, DDStore ensures that deep learning model maintains high accuracy and generalizability when it is trained with large volume of data using HPC facilities.

Choi, Jong Youl [Oak Ridge National Laboratory (OR↗
DOE OSTI2020

Conditional Generative Adversarial Networks for Solving Heat Transfer Problems

Generative Adversarial Networks (GANs) have been used as a deep learning approach to solving physics and engineering problems. Using deep learning for these problems is attractive in that reasonably accurate models can be inferred from only raw data, eliminating the need to define the exact physical equations governing a problem. We expand on previous work using GANs to generate steady-state solutions to the two-dimensional heat equation. Using a basic conditional GAN (cGAN), we generate accurate solutions for rectangular domains conditioned on four edge boundary conditions (MAE < 0.5%). For finding steady-state solutions over arbitrary two-dimensional domains (not constrained to rectangles), we use a cGAN designed for image-to-image translation. We train this GAN on various types of geometric domains (circles, squares, triangles, shapes with one circular or rectangular hole), achieving accurate results on test data made up of geometries similar to those in training (MAE < 1%). For both of these GANs, we experiment with different loss function terms, showing that a term using the gradients of solution images significantly improves the basic cGAN but not the image-to-image GAN. Lastly, we show that the image-to-image GAN performs poorly when applied to two-dimensional geometries that vary in structure from training data (MAE < 8% for shapes with multiple holes or different shaped holes). This demonstrates the cGAN's lack of generalizability. While the cGAN is an accurate and computationally efficient method when trained and tested on similarly structured data, it is a much less reliable method when applied to data that is slightly different in structure from the training data.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2021

Cosmic Ray Background Removal With Deep Neural Networks in SBND

In liquid argon time projection chambers exposed to neutrino beams and running on or near surface levels, cosmic muons, and other cosmic particles are incident on the detectors while a single neutrino-induced event is being recorded. In practice, this means that data from surface liquid argon time projection chambers will be dominated by cosmic particles, both as a source of event triggers and as the majority of the particle count in true neutrino-triggered events. In this work, we demonstrate a novel application of deep learning techniques to remove these background particles by applying deep learning on full detector images from the SBND detector, the near detector in the Fermilab Short-Baseline Neutrino Program. We use this technique to identify, on a pixel-by-pixel level, whether recorded activity originated from cosmic particles or neutrino interactions.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2020

Deep learning-based model for progress variable dissipation rate in turbulent premixed flames

A deep neural network (DNN) based large eddy simulation (LES) model for progress variable dissipation rate in turbulent premixed flames is presented. The DNN model is trained using filtered data from direct numerical simulations (DNS) of statistically planar turbulent premixed flames with n-heptane as fuel. Training data was comprised of flames with varying turbulence levels leading to a range of Karlovitz numbers. Through a-priori tests the DNN model is shown to predict the subfilter contribution to progress variable dissipation rate accurately over a range of filter widths and for all Karlovitz numbers examined in this study. Superior performance of the DNN model relative to an established physics-based model is also demonstrated. Additionally, transferability of the DNN model is highlighted by a-priori evaluation of the model using filtered DNS data from multiple cases with different Karlovitz numbers and fuel species than those that were used for training the model.

33 ADVANCED PROPULSION SYSTEMS↗
DOE OSTI2024

Reinforcement Learning as a Parsimonious Alternative to Prediction Cascades: A Case Study on Image Segmentation

Deep learning architectures have achieved state-of-the-art (SOTA) performance on computer vision tasks such as object detection and image segmentation. This may be attributed to the use of over-parameterized, monolithic deep learning architectures executed on large datasets. Although such large architectures lead to increased accuracy, this is usually accompanied by a larger increase in computation and memory requirements during inference. While this is a non-issue in traditional machine learning (ML) pipelines, the recent confluence of machine learning and fields like the Internet of Things (IoT) has rendered such large architectures infeasible for execution in low-resource settings. For some datasets, large monolithic pipelines may be overkill for simpler inputs. To address this problem, previous efforts have proposed decision cascades where inputs are passed through models of increasing complexity until the desired performance is achieved. However, we argue that cascaded prediction leads to sub-optimal throughput and increased computational cost due to wasteful intermediate computations. To address this, we propose PaSeR (Parsimonious Segmentation with Reinforcement Learning) a non-cascading, cost-aware learning pipeline as an efficient alternative to cascaded decision architectures. Through experimental evaluation on both real-world and standard datasets, we demonstrate that PaSeR achieves better accuracy while minimizing computational cost relative to cascaded models. Further, we introduce a new metric IoU/GigaFlop to evaluate the balance between cost and performance. On the real-world task of battery material phase segmentation, PaSeR yields 179% improvement over SOTA MatPhase model and a 196% improvement over IDK Cascades under the IoU/GigaFlop metric. We also demonstrate PaSeR’s adaptability to complementary models trained on a noisy MNIST dataset, where it outperforms all baselines on IoU/GigaFlop by an average of 44%.

Srikshan, Bharat↗
DOE OSTI2021

Machine Learning for Turbulence

New Deep Learning architectures that embed physics as hard constraints with numerical methods, increasing accuracy and reducing computational cost.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
DOE OSTI2021

Improvement of the NOvA Near Detector Event Reconstruction and Primary Vertexing through the Application of Machine Learning Methods

The purpose of this work is to examine the application of a deep learning model in event reconstruction of neutrino interactions. The challenges faced in event reconstruction include the placement of an accurate primary neutrino interaction vertex which is used to support the particle track and prong algorithms. The result of accurate primary vertex ensures all particles involved in a neutrino interaction are included. We propose a regression-based Convolutional Neural Network (CNN) method to predict the primary vertex of a particle interaction. We show that with raw two-dimensional pixel map views as input, the regression-based CNN can predict the primary vertex in all three coordinates. This work is applied as part of the NOvA (NuMI Off-axis $\nu_e$ Appearance) near detector reconstruction efforts. The primary vertex predicted by the regression-based CNN model shows promising results for future applications. This deep learning method can be extended to secondary vertexing through a Kernel Density Estimate algorithm discussed in this work.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗
DOE OSTI2021

An Overview of the Usefulness of Machine Learning Techniques on Network Packet Data

Understanding the health and behavior of a computer network allows for better network efficiency and security. We present an overview of various machine learning techniques for classifying network packet data via packet metadata. While some classical machine learning approaches achieve reasonable results, the most accurate classification can be achieved with deep learning. On the four data sets studied herein, a basic deep learning model achieved at or near 100\% classification accuracy. We also propose a method for determining variable importance as a means for potential transfer learning applications to classifying yet unseen network packet data.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2023

De novo design of protein structure and function with RFdiffusion

Abstract There has been considerable recent progress in designing new proteins using deep-learning methods 1–9 . Despite this progress, a general deep-learning framework for protein design that enables solution of a wide range of design challenges, including de novo binder design and design of higher-order symmetric architectures, has yet to be described. Diffusion models 10,11 have had considerable success in image and language generative modelling but limited success when applied to protein modelling, probably due to the complexity of protein backbone geometry and sequence–structure relationships. Here we show that by fine-tuning the RoseTTAFold structure prediction network on protein structure denoising tasks, we obtain a generative model of protein backbones that achieves outstanding performance on unconditional and topology-constrained protein monomer design, protein binder design, symmetric oligomer design, enzyme active site scaffolding and symmetric motif scaffolding for therapeutic and metal-binding protein design. We demonstrate the power and generality of the method, called RoseTTAFold diffusion (RFdiffusion), by experimentally characterizing the structures and functions of hundreds of designed symmetric assemblies, metal-binding proteins and protein binders. The accuracy of RFdiffusion is confirmed by the cryogenic electron microscopy structure of a designed binder in complex with influenza haemagglutinin that is nearly identical to the design model. In a manner analogous to networks that produce images from user-specified inputs, RFdiffusion enables the design of diverse functional proteins from simple molecular specifications.

Science & Technology - Other Topics↗
DOE OSTI2020

Completion design improvement using a deep convolutional network

Maximizing stimulated natural and hydraulic fracture network is one of the primary hydraulic fracturing concerns for economic production from a horizontal shale gas well. Geomechanical facies and preexisting fractures in each stage are identified based on similarities in formation characteristics to optimize the locations of perforation clusters. This often requires analyzing large volumes of drilling, Logging While Drilling (LWD) and Measurement While Drilling (MWD) data. In this paper, we develop a methodology that calculates the mechanical specific energy (MSE) using real-time drill string acceleration signals directly from its definition. High resolution vibration signals have been collected using a tri-axial accerlometer, which was an auxiliary tool included in acoustic borehole imager. This technique provides a cost-efficient solution for engineered completion design. Furthermore, we adopt deep Convolutional Neural Network (CNN) with signal processing to build a data pipeline that effectively extracts patterns from dynamic acceleration signals for rock lateral MSE classification. First, we apply discrete wavelet transform and Short-Time Fourier Transform (STFT) for signal denoising and pattern recognition. Then we construct an image dataset using multi-scale image fusion at pixel level from 3 sensor channels, including axial, lateral acceleration spectrograms and zero-padded revolutions per minute (RPM). The resulted RGB image dataset includes 4,000 images of 5 MSE ranges with various rock strength conditions. Our results demonstrate that the proposed deep learning model can achieve more than 90% classification accuracy. The deep learning results, as a reference source, were applied in selected Marcellus Shale Energy and Environmental Lab (MSEEL) wells engineered completion located in the Marcellus shale gas site.

03 NATURAL GAS↗