SEARCH · Engineering Papers

Results for “deep neural network”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 865 records · Page 48

All sources NASA NTRS DOE OSTI

DOE OSTI2021

Deep learning for Alzheimer's disease: Mapping large-scale histological tau protein for neuroimaging biomarker validation

Abnormal tau inclusions are hallmarks of Alzheimer's disease and predictors of clinical decline. Several tau PET tracers are available for neurodegenerative disease research, opening avenues for molecular diagnosis in vivo. However, few have been approved for clinical use. Understanding the neurobiological basis of PET signal validation remains problematic because it requires a large-scale, voxel-to-voxel correlation between PET and (immuno) histological signals. Large dimensionality of whole human brains, tissue deformation impacting co-registration, and computing requirements to process terabytes of information preclude proper validation. We developed a computational pipeline to identify and segment particles of interest in billion-pixel digital pathology images to generate quantitative, 3D density maps. The proposed convolutional neural network for immunohistochemistry samples, IHCNet, is at the pipeline's core. We have successfully processed and immunostained over 500 slides from two whole human brains with three phospho-tau antibodies (AT100, AT8, and MC1), spanning several terabytes of images. Our artificial neural network estimated tau inclusion from brain images, which performs with ROC AUC of 0.87, 0.85, and 0.91 for AT100, AT8, and MC1, respectively. Introspection studies further assessed the ability of our trained model to learn tau-related features. We present an end-to-end pipeline to create terabytes-large 3D tau inclusion density maps co-registered to MRI as a means to facilitate validation of PET tracers.

60 APPLIED LIFE SCIENCES↗

DOE OSTI2024

Data-Efficient Dimensionality Reduction and Surrogate Modeling of High-Dimensional Stress Fields

Tensor datatypes representing field variables like stress, displacement, velocity, etc., have increasingly become a common occurrence in data-driven modeling and analysis of simulations. Numerous methods [such as convolutional neural networks (CNNs)] exist to address the meta-modeling of field data from simulations. As the complexity of the simulation increases, so does the cost of acquisition, leading to limited data scenarios. Modeling of tensor datatypes under limited data scenarios remains a hindrance for engineering applications. Here, in this article, we introduce a direct image-to-image modeling framework of convolutional autoencoders enhanced by information bottleneck loss function to tackle the tensor data types with limited data. The information bottleneck method penalizes the nuisance information in the latent space while maximizing relevant information making it robust for limited data scenarios. The entire neural network framework is further combined with robust hyperparameter optimization. We perform numerical studies to compare the predictive performance of the proposed method with a dimensionality reduction-based surrogate modeling framework on a representative linear elastic ellipsoidal void problem with uniaxial loading. The data structure focuses on the low-data regime (fewer than 100 data points) and includes the parameterized geometry of the ellipsoidal void as the input and the predicted stress field as the output. The results of the numerical studies show that the information bottleneck approach yields improved overall accuracy and more precise prediction of the extremes of the stress field. Additionally, an in-depth analysis is carried out to elucidate the information compression behavior of the proposed framework.

artificial intelligence↗

DOE OSTI2025

An efficient surrogate model of secondary electron formation and evolution

This work extends the adjoint-deep learning framework for runaway electron (RE) evolution, developed by McDevitt et al. [Phys. Plasmas 32, 042503 (2025)], to account for large-angle collisions. By incorporating large-angle collisions, the framework allows the avalanche of REs to be captured, an essential component of RE dynamics. This extension is accomplished by using a Rosenbluth–Putvinski approximation to estimate the distribution of secondary electrons generated by large-angle collisions. By evolving both the primary and multiple generations of secondary electrons, the present formulation can capture both the detailed temporal evolution of a RE population beginning from an arbitrary initial momentum space distribution, along with providing approximations to the saturated growth and decay rates of the RE population. Predictions of the adjoint-deep learning framework are verified against a traditional RE solver, with good agreement present across a broad range of parameters.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗

DOE OSTI2021

Molecular dynamics simulation of metallic Al–Ce liquids using a neural network machine learning interatomic potential

Al-rich Al-Ce alloys have the possibility of replacing heavier steel and cast-irons for use in high-temperature applications. Knowledge about the structures and properties of Al-Ce alloys at liquid state is vital for optimizing the manufacture process to produce desired allows. However, reliable molecular dynamics simulation of Al-Ce alloy systems remains a great challenge due to the lack of accurate Al-Ce interatomic potential. In this work, an artificial neural network (ANN) deep machine learning (ML) method is used to develop a reliable interatomic potential for Al-Ce alloy. Ab initio molecular dynamics (AIMD) simulation data on Al-Ce liquid with small unit cell (~200 atoms) and on the known Al-Ce crystalline compounds are collected to train the interatomic potential using ANN-ML. The obtained ANN-ML model reproduces well the energies, forces, and atomic structure of Al 90 Ce 10 liquid and crystalline phases of Al-Ce compounds in comparison with ab initio results. The developed ANN-ML potential is applied in molecular dynamics simulations to study the structures and properties of metallic Al 90 Ce 10 liquid, which would provide useful insight for guiding experimental process to produce desired Al-Ce allows.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

DOE OSTI2026

Explaining System-Level Prognostics with Established Machine Learning Methods

System-level prognostics is crucial for ensuring reliability and enabling predictive maintenance in complex systems with interconnected components. This study presents a framework that integrates data-driven methods to predict the remaining useful life (RUL) of a subsystem under multiple and concurrent faults within a nuclear power plant system with explainable artificial intelligence (XAI). A nuclear power plant (NPP) operation was simulated to model the degradation behavior of NPP components, and four machine learning models—Gradient Boosting Regressor (GBR), Support Vector Regressor (SVR), Fully Connected Neural Network (FCNN), and Long Short-Term Memory (LSTM)—were evaluated for prognostics with a novel system RUL parameter. The LSTM model demonstrated potential superior repeatability, while SHAP (SHapley Additive exPlanations) for explainability provided consistent and trustworthy global explanations. In contrast, LIME (Local Interpretable Model-agnostic Explanations) offered localized interpretability but showed reduced stability for sequential data. Key findings include the interplay between component-level degradation and system-wide performance, with LSTM effectively capturing these dynamics through sequence-level predictions. The XAI techniques enhanced transparency by identifying critical features influencing model predictions and aligning with domain knowledge. Furthermore, this framework has significant implications for improving trust and understanding in predictive maintenance, particularly in safety-critical industries like nuclear energy.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗

DOE OSTI2021

Solving Inverse Stochastic Problems from Discrete Particle Observations Using the Fokker--Planck Equation and Physics-Informed Neural Networks

The Fokker--Planck (FP) equation governing the evolution of the probability density function (PDF) is applicable to many disciplines, but it requires specification of the coefficients for each case, which can be functions of space-time and not just constants and hence require the development of a data-driven modeling approach. When the data available is directly on the PDF, there exist methods for inverse problems that can be employed to infer the coefficients and thus determine the FP equation and subsequently obtain its solution. Herein, we address a more realistic scenario, where only sparse data are given on the particles' positions at a few time instants, which are not sufficient to accurately construct directly the PDF even at those times from existing methods, e.g., kernel estimation algorithms. To this end, we develop a general framework based on physics-informed neural networks (PINNs) that introduces a new loss function using the Kullback--Leibler divergence to connect the stochastic samples with the FP equation to simultaneously learn the equation and infer the multidimensional PDF at all times. In particular, we consider two types of inverse problems, type I, where the FP equation is known but the initial PDF is unknown, and type II, in which, in addition to the unknown initial PDF, the drift and diffusion terms are also unknown. In both cases, we investigate problems with either Brownian or Lévy noise or a combination of both. Here, we demonstrate the new PINN framework in detail in the one-dimensional (1D) case, but we also provide results for up to five dimensions demonstrating that we can infer both the FP equation and dynamics simultaneously at all times with high accuracy using only very few discrete observations of the particles.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2021

A deep learning-accelerated data assimilation and forecasting workflow for commercial-scale geologic carbon storage

Fast assimilation of monitoring data to update forecasts of pressure buildup and carbon dioxide (CO 2 ) plume migration under geologic uncertainties is a challenging problem in geologic carbon storage. The high computational cost of data assimilation with a high-dimensional parameter space impedes fast decision-making for commercial-scale reservoir management. We propose to leverage physical understandings of porous medium flow behavior with deep learning techniques to develop a fast data assimilation-reservoir response forecasting workflow. Applying an Ensemble Smoother Multiple Data Assimilation (ES-MDA) framework, the workflow updates geologic properties and predicts reservoir performance with quantified uncertainty from pressure history and CO 2 plumes interpreted through seismic inversion. As the most computationally expensive component in such a workflow is reservoir simulation, we developed surrogate models to predict dynamic pressure and CO 2 plume extents under multi-well injection. The surrogate models employ deep convolutional neural networks, specifically, a wide residual network and a residual U-Net. The workflow is validated against a flat threedimensional reservoir model representative of a clastic shelf depositional environment. Intelligent treatments are applied to bridge between quantities in a true-3D reservoir model and those in a single-layer reservoir model. The workflow can complete history matching and reservoir forecasting with uncertainty quantification in less than one hour on a mainstream personal workstation.

25 ENERGY STORAGE↗

DOE OSTI2022

A comparison of deep-learning-based inpainting techniques for experimental X-ray scattering

The implementation is proposed of image inpainting techniques for the reconstruction of gaps in experimental X-ray scattering data. The proposed methods use deep learning neural network architectures, such as convolutional autoencoders, tunable U-Nets, partial convolution neural networks and mixed-scale dense networks, to reconstruct the missing information in experimental scattering images. In particular, the recovered pixel intensities are evaluated against their corresponding ground-truth values using the mean absolute error and the correlation coefficient metrics. The results demonstrate that the proposed methods achieve better performance than traditional inpainting algorithms such as biharmonic functions. Overall, tunable U-Net and mixed-scale dense network architectures achieved the best reconstruction performance among all the tested algorithms, with correlation coefficient scores greater than 0.9980.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2023

A co-kurtosis PCA based dimensionality reduction with nonlinear reconstruction using neural networks

For turbulent reacting flow systems, identification of low-dimensional representations of the thermo-chemical state space is vitally important, primarily to significantly reduce the computational cost of device-scale simulations. Principal component analysis (PCA), and its variants, are a widely employed class of methods. Recently, an alternative technique that focuses on higher-order statistical interactions, co-kurtosis PCA (CoK-PCA), has been shown to effectively provide a low-dimensional representation by capturing the stiff chemical dynamics associated with spatiotemporally localized reaction zones. While its effectiveness has only been demonstrated based on a priori analyses with linear reconstruction, in this work, we employ nonlinear techniques to reconstruct the full thermo-chemical state and evaluate the efficacy of CoK-PCA compared to PCA. Specifically, we combine a CoK-PCA-/PCA-based dimensionality reduction (encoding) with an artificial neural network (ANN) based reconstruction (decoding) and examine, a priori, the reconstruction errors of the thermo-chemical state. In addition, we evaluate the errors in species production rates and heat release rates, which are nonlinear functions of the reconstructed state, as a measure of the overall accuracy of the dimensionality reduction technique. We employ four datasets to assess CoK-PCA/PCA coupled with ANN-based reconstruction: zero-dimensional (homogeneous) reactor for autoignition of an ethylene/air mixture that has conventional single-stage ignition kinetics, a dimethyl ether (DME)/air mixture which has two-stage (low and high temperature) ignition kinetics, a one-dimensional freely propagating premixed ethylene/air laminar flame, and a two-dimensional dataset representing turbulent autoignition of ethanol in a homogeneous charge compression ignition (HCCI) engine. Finally, results from the analyses demonstrate the robustness of the CoK-PCA based low-dimensional manifold with ANN reconstruction in accurately capturing the data, specifically from the reaction zones.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

DOE OSTI2022

Autoencoder Neural Network for chemically reacting systems

Incorporating detailed chemical kinetic models is critical for accurate simulations of reacting flows. However, detailed models involve a large number of thermochemical (TC) state variables. Solving the governing equations to evolve these TC variables becomes impractical for real-world applications. In this work, we propose an autoencoder (AE) neural network (NN)-based reduced model to accelerate such simulations. The AE NN is first trained to find a low-dimensional latent representation of the TC states. Then, the evolving state of a chemical system can be tracked by solving the equations of the latent variables instead of the original TC equations. We demonstrate the reduced model in a syngas CO/H 2 combustion system, using training data collected from canonical perfectly stirred reactors (PSRs). It is found that the AE model can reduce the dimension of the combustion system from 12 to 2 while maintaining low reconstruction error and excellent elemental mass conservation for the test dataset. In the a posteriori test, the combustion states obtained from solving the two latent equations are compared to those from solving the 12 equations of the full model. The AE reduced method is found to be able to capture the diverse combustion states on the top two branches of the S-curve well including the extinction turning point, but with higher prediction errors for states near the ignition turning point.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2023

A Novel Deep Reinforcement Learning Approach to Traffic Signal Control with Connected Vehicles

The advent of connected vehicle (CV) technology offers new possibilities for a revolution in future transportation systems. With the availability of real-time traffic data from CVs, it is possible to more effectively optimize traffic signals to reduce congestion, increase fuel efficiency, and enhance road safety. The success of CV-based signal control depends on an accurate and computationally efficient model that accounts for the stochastic and nonlinear nature of the traffic flow. Without the necessity of prior knowledge of the traffic system’s model architecture, reinforcement learning (RL) is a promising tool to acquire the control policy through observing the transition of the traffic states. In this paper, we propose a novel data-driven traffic signal control method that leverages the latest in deep learning and reinforcement learning techniques. By incorporating a compressed representation of the traffic states, the proposed method overcomes the limitations of the existing methods in defining the action space to include more practical and flexible signal phases. The simulation results demonstrate the convergence and robust performance of the proposed method against several existing benchmark methods in terms of average vehicle speeds, queue length, wait time, and traffic density.

42 ENGINEERING↗

DOE OSTI2023

Bayesian Inference with Latent Hamiltonian Neural Networks (L-HNNs)

When sampling for Bayesian inference, one popular approach is to use Hamiltonian Monte Carlo (HMC) and the No-U-Turn Sampler (NUTS). However, HMC and NUTS can require numerous numerical gradients of the target density and can prove slow in practice. We propose Hamiltonian neural networks (HNNs) with HMC and NUTS for solving Bayesian inference problems [1, 2]. Once trained, HNNs do not require gradients of the target density while sampling. Moreover, they satisfy important properties such as perfect time reversibility and Hamiltonian conservation, making them well suited for use within HMC and NUTS because stationarity can be shown. We also propose an HNN extension called latent HNNs (L-HNNs), which predict latent variable outputs. Compared to HNNs, L-HNNs offer improved expressivity and a reduction in integration errors. Finally, we propose employing L-HNNs in NUTS with an online error monitoring scheme to prevent degeneracy of the sampling in regions of low probability density. We demonstrate L-HNNs in NUTS with online error monitoring by using several example cases involving complex, heavy-tailed, and high local curvature probability densities. Overall, L-HNNs in NUTS with online error monitoring satisfactorily inferred these probability densities. Compared to traditional NUTS, L-HNNs in NUTS with online error monitoring improved the effective sample size (ESS) per gradient by an order of magnitude.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2021

Scalable balanced training of conditional generative adversarial neural networks on image data

Here, we propose a distributed approach to train deep convolutional generative adversarial neural network (DC-CGANs) models. Our method reduces the imbalance between generator and discriminator by partitioning the training data according to data labels, and enhances scalability by performing a parallel training where multiple generators are concurrently trained, each one of them focusing on a single data label. Performance is assessed in terms of inception score, Fréchet inception distance, and image quality on MNIST, CIFAR10, CIFAR100, and ImageNet1k datasets, showing a significant improvement in comparison to state-of-the-art techniques to training DC-CGANs. Weak scaling is attained on all the four datasets using up to 1000 processes and 2000 NVIDIA V100 GPUs on the OLCF supercomputer Summit.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2020

PPINN: Parareal physics-informed neural network for time-dependent PDEs

Physics-informed neural networks (PINNs) encode physical conservation laws and prior physical knowledge into the neural networks, ensuring the correct physics is represented accurately while alleviating the need for supervised learning to a great degree. While effective for relatively short-term time integration, when long time integration of the time-dependent PDEs is sought, the time–space domain may become arbitrarily large and hence training of the neural network may become prohibitively expensive. To this end, we develop a parareal physics-informed neural network (PPINN), hence decomposing a long-time problem into many independent short-time problems supervised by an inexpensive/fast coarse-grained (CG) solver. In particular, the serial CG solver is designed to provide approximate predictions of the solution at discrete times, while initiate many fine PINNs simultaneously to correct the solution iteratively. There is a two-fold benefit from training PINNs with small-data sets rather than working on a large-data set directly, i.e., training of individual PINNs with small-data is much faster, while training the fine PINNs can be readily parallelized. Consequently, compared to the original PINN approach, the proposed PPINN approach may achieve a significant speed-up for long-time integration of PDEs, assuming that the CG solver is fast and can provide reasonable predictions of the solution, hence aiding the PPINN solution to converge in just a few iterations. To investigate the PPINN performance on solving time-dependent PDEs, we first apply the PPINN to solve the Burgers equation, and subsequently we apply the PPINN to solve a two-dimensional nonlinear diffusion–reaction equation. Furthermore, our results demonstrate that PPINNs converge in a few iterations with significant speed-ups proportional to the number of time-subdomains employed.

42 ENGINEERING↗

DOE OSTI2024

Deep learning with mixup augmentation for improved pore detection during additive manufacturing

In additive manufacturing (AM), process defects such as keyhole pores are difficult to anticipate, affecting the quality and integrity of the AM-produced materials. Hence, considerable efforts have aimed to predict these process defects by training machine learning (ML) models using passive measurements such as acoustic emissions. This work considered a dataset in which keyhole pores of a laser powder bed fusion (LPBF) experiment were identified using X-ray radiography and then registered both in space and time to acoustic measurements recorded during the LPBF experiment. Due to AM’s intrinsic process controls, where a pore-forming event is relatively rare, the acoustic datasets collected during monitoring include more non-pores than pores. In other words, the dataset for ML model development is imbalanced. Moreover, this imbalanced and sparse data phenomenon remains ubiquitous across many AM monitoring schemes since training data is nontrivial to collect. Hence, we propose a machine learning approach to improve this dataset imbalance and enhance the prediction accuracy of pore-labeled data. Specifically, we investigate how data augmentation helps predict pores and non-pores better. This imbalance is improved using recent advances in data augmentation called Mixup, a weak-supervised learning method. Convolutional neural networks (CNNs) are trained on original and augmented datasets, and an appreciable increase in performance is reported when testing on five different experimental trials. When ML models are trained on original and augmented datasets, they achieve an accuracy of 95% and 99% on test datasets, respectively. We also provide information on how dataset size affects model performance. Lastly, we investigate the optimal Mixup parameters for augmentation in the context of CNN performance.

36 MATERIALS SCIENCE↗

DOE OSTI2024

Deep learning with mixup augmentation for improved pore detection during additive manufacturing

42 ENGINEERING↗

DOE OSTI2024

Deep learning-based predictive models for laser direct drive at the Omega Laser Facility

The rich and complex physics of inertial confinement fusion provides a unique and challenging space for high-fidelity first-principles modeling. Consequently, simulation codes that are used to design experiments are computationally expensive and lack the predictive capability required for extensive parameter exploration in search of a high-performing design for laser direct drive. In this article, we present two deep-learning-based predictive models intended to address these difficulties. The first model (TL DNN) acts as a fast emulator of simulations as well as experiments at the Omega Laser Facility. This model is trained on a simulation database and subsequently calibrated on experimental data using transfer learning. To facilitate the development of this model, an autoencoder is developed to reduce the dimensionality of the input space by compressing the laser pulse input. The model predicts key experimental scalar observables of Omega experiments with high accuracy and minimal computational cost. This deep neural net enables rapid exploration of a high-dimensional input parameter space for an optimal implosion design. The second model (DNN SM+) aims to extend the statistical modeling work of Lees et al. [Phys. Rev. Lett. 127, 105001 (2021)], by increasing the complexity of the model space and allowing for coupling between degradation terms. Since the model capacity of DNN SM+ is higher than the model of Lees et al., DNN SM+ can potentially provide an improvement in predictive capability, and we use this model to provide insight into complicated degradation dependencies.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗

DOE OSTI2023

DeepAstroUDA: semi-supervised universal domain adaptation for cross-survey galaxy morphology classification and anomaly detection

Abstract Artificial intelligence methods show great promise in increasing the quality and speed of work with large astronomical datasets, but the high complexity of these methods leads to the extraction of dataset-specific, non-robust features. Therefore, such methods do not generalize well across multiple datasets. We present a universal domain adaptation method, DeepAstroUDA , as an approach to overcome this challenge. This algorithm performs semi-supervised domain adaptation (DA) and can be applied to datasets with different data distributions and class overlaps. Non-overlapping classes can be present in any of the two datasets (the labeled source domain, or the unlabeled target domain), and the method can even be used in the presence of unknown classes. We apply our method to three examples of galaxy morphology classification tasks of different complexities (three-class and ten-class problems), with anomaly detection: (1) datasets created after different numbers of observing years from a single survey (Legacy Survey of Space and Time mock data of one and ten years of observations); (2) data from different surveys (Sloan Digital Sky Survey (SDSS) and DECaLS); and (3) data from observing fields with different depths within one survey (wide field and Stripe 82 deep field of SDSS). For the first time, we demonstrate the successful use of DA between very discrepant observational datasets. DeepAstroUDA is capable of bridging the gap between two astronomical surveys, increasing classification accuracy in both domains (up to 40 % on the unlabeled data), and making model performance consistent across datasets. Furthermore, our method also performs well as an anomaly detection algorithm and successfully clusters unknown class samples even in the unlabeled target dataset.

79 ASTRONOMY AND ASTROPHYSICS↗