Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Deep Neural Networks”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,171 records · Page 65

TH-CD-206-05: Machine-Learning Based Segmentation of Organs at Risks for Head and Neck Radiotherapy Planning

Purpose: Accurate and efficient delineation of tumor target and organs-at-risks is essential for the success of radiotherapy. In reality, despite of decades of intense research efforts, auto-segmentation has not yet become clinical practice. In this study, we present, for the first time, a deep learning-based classification algorithm for autonomous segmentation in head and neck (HaN) treatment planning. Methods: Fifteen HN datasets of CT, MR and PET images with manual annotation of organs-at-risk (OARs) including spinal cord, brainstem, optic nerves, chiasm, eyes, mandible, tongue, parotid glands were collected and saved in a library of plans. We also have ten super-resolution MR images of the tongue area, where the genioglossus and inferior longitudinalis tongue muscles are defined as organs of interest. We applied the concepts of random forest- and deep learning-based object classification for automated image annotation with the aim of using machine learning to facilitate head and neck radiotherapy planning process. In this new paradigm of segmentation, random forests were used for landmark-assisted segmentation of super-resolution MR images. Alternatively to auto-segmentation with random forest-based landmark detection, deep convolutional neural networks were developed for voxel-wise segmentation of OARs in single and multi-modal images. The network consisted of three pairs of convolution and pooing layer, one RuLU layer and a softmax layer. Results: We present a comprehensive study on using machine learning concepts for auto-segmentation of OARs and tongue muscles for the HaN radiotherapy planning. An accuracy of 81.8% in terms of Dice coefficient was achieved for segmentation of genioglossus and inferior longitudinalis tongue muscles. Preliminary results of OARs regimentation also indicate that deep-learning afforded an unprecedented opportunities to improve the accuracy and robustness of radiotherapy planning. Conclusion: A novel machine learning framework has been developed for image annotation and structure segmentation. Our results indicate the great potential of deep learning in radiotherapy treatment planning.

60 APPLIED LIFE SCIENCES↗

Physics-Informed Machine Learning of Dynamical Systems for Efficient Bayesian Inference

Although the no-u-turn sampler (NUTS) is a widely adopted method for performing Bayesian inference, it requires numerous posterior gradients which can be expensive to compute in practice. Recently, there has been a significant interest in physics-based machine learning of dynamical (or Hamiltonian) systems and Hamiltonian neural networks (HNNs) is a noteworthy architecture. But these types of architectures have not been applied to solve Bayesian inference problems efficiently. We propose the use of HNNs for performing Bayesian inference efficiently without requiring numerous posterior gradients. We introduce latent variable outputs to HNNs (L-HNNs) for improved expressivity and reduced integration errors. We integrate L-HNNs in NUTS and further propose an online error monitoring scheme to prevent sampling degeneracy in regions where L-HNNs may have little training data. We demonstrate L-HNNs in NUTS with online error monitoring consider several complex high-dimensional posterior densities and compare its performance to NUTS.

97 MATHEMATICS AND COMPUTING↗

Self-Supervised Cloud Classification

Abstract Low-level marine clouds play a pivotal role in Earth’s weather and climate through their interactions with radiation, heat and moisture transport, and the hydrological cycle. These interactions depend on a range of dynamical and microphysical processes that result in a broad diversity of cloud types and spatial structures, and a comprehensive understanding of cloud morphology is critical for continued improvement of our atmospheric modeling and prediction capabilities moving forward. Deep learning has recently accelerated our ability to study clouds using satellite remote sensing, and machine learning classifiers have enabled detailed studies of cloud morphology. A major limitation of deep learning approaches to this problem, however, is the large number of hand-labeled samples that are required for training. This work applies a recently developed self-supervised learning scheme to train a deep convolutional neural network (CNN) to map marine cloud imagery to vector embeddings that capture information about mesoscale cloud morphology and can be used for satellite image classification. The model is evaluated against existing cloud classification datasets and several use cases are demonstrated, including training cloud classifiers with very few labeled samples, interrogation of the CNN’s learned internal feature representations, cross-instrument application, and resilience against sensor calibration drift and changing scene brightness. The self-supervised approach learns meaningful internal representations of cloud structures and achieves comparable classification accuracy to supervised deep learning methods without the expense of creating large hand-annotated training datasets. Significance Statement Marine clouds heavily influence Earth’s weather and climate, and improved understanding of marine clouds is required to improve our atmospheric modeling capabilities and physical understanding of the atmosphere. Recently, deep learning has emerged as a powerful research tool that can be used to identify and study specific marine cloud types in the vast number of images collected by Earth-observing satellites. While powerful, these approaches require hand-labeling of training data, which is prohibitively time intensive. This study evaluates a recently developed self-supervised deep learning method that does not require human-labeled training data for processing images of clouds. We show that the trained algorithm performs competitively with algorithms trained on hand-labeled data for image classification tasks. We also discuss potential downstream uses and demonstrate some exciting features of the approach including application to multiple satellite instruments, resilience against changing image brightness, and its learned internal representations of cloud types. The self-supervised technique removes one of the major hurdles for applying deep learning to very large atmospheric datasets.

54 ENVIRONMENTAL SCIENCES↗

Deep Gaze Velocity Analysis During Mammographic Reading for Biometric Identification of Radiologists

Several studies have confirmed that the gaze velocity of the human eye can be utilized as a behavioral biometric or personalized biomarker. In this study, we leverage the local feature representation capacity of convolutional neural networks (CNNs) for eye gaze velocity analysis as the basis for biometric identification of radiologists performing breast cancer screening. Using gaze data collected from 10 radiologists reading 100 mammograms of various diagnoses, we compared the performance of a CNN-based classification algorithm with two deep learning classifiers, deep neural network and deep belief network, and a previously presented hidden Markov model classifier. The study showed that the CNN classifier is superior compared to alternative classification methods based on macro F1-scores derived from 10-fold cross-validation experiments. Our results further support the efficacy of eye gaze velocity as a biometric identifier of medical imaging experts.

59 BASIC BIOLOGICAL SCIENCES↗

Post-hazard condition assessment of nuclear piping-equipment systems: Novel approach to feature extraction and deep learning

Over the past decade, the use of artificial intelligence techniques in the field of health-monitoring has gained significant interest, especially for structures such as building and bridges. However, applications to industrial systems such as equipment-piping systems in nuclear plants have not been explored. Here, in this paper, it is shown that the existing techniques developed for buildings and bridges cannot be extended directly to equipment-piping systems as the response of such systems is governed by multiple localized modes unlike that in buildings and bridges. This paper proposes a new approach that consists of three key aspects: (i) a novel vector of degradation-sensitive features extracted from measured data, (ii) using a deep Artificial Neural Network (ANN) for diagnosis of degradation location and degradation severity, and (iii) consideration of uncertainty in degradation severity when training the ANN. Degradation in piping-equipment systems can occur due to flow-accelerated erosion and corrosion. These locations can potentially exhibit damage such as localized yielding or initiation of cracking due to an external event such as an earthquake. Moreover, such locations can at times go undetected by current inspection techniques. Therefore, a robust framework is needed for detection of degradation after a seismic event. This manuscript proposes a proof-of-concept framework, which utilizes data collected from sensors to generate a deep ANN database for predicting degraded locations and severity in a piping-equipment system. Degradation severity is classified as minor, moderate, and severe. In the suggested methodology, a novel vector of degradation-sensitive features is extracted from the sensor data to train the ANN. A simple piping-equipment system is selected to demonstrate feature extraction as a means to simplify pattern recognition, explore the design and parameters of an ANN, and develop a sensor placement strategy. The effectiveness of the proposed framework is demonstrated on a realistic primary safety system of a two-loop nuclear reactor. It is shown that the proposed post-hazard condition assessment framework is able to detect degraded locations along with the severity levels, including minor degradation, with considerably higher accuracy.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗

Machine Learning-Assisted Distribution System Network Reconfiguration Problem

High penetration from volatile renewable energy resources in the grid and the varying nature of loads raise the need for frequent line switching to ensure the efficient operation of electrical distribution networks. Operators must ensure maximum load delivery, reduced losses, and the operation between voltage limits. However, computations to decide the optimal feeder configuration are often computationally expensive and intractable, making it unfavorable for real-time operations. This is mainly due to the existence of binary variables in the network reconfiguration optimization problem. To tackle this issue, we have devised an approach that leverages machine learning techniques to reshape distribution networks featuring multiple substations. This involves predicting the substation responsible for serving each part of the network. Hence, it leaves simple and more tractable Optimal Power Flow problems to be solved. This method can produce accurate results in a significantly faster time, as demonstrated using the IEEE 37-bus distribution feeder. Compared to the traditional optimization-based approaches, a feasible solution is achieved approximately ten times faster for all the tested scenarios.

deep neural networks↗

Deep learning for Alzheimer's disease: Mapping large-scale histological tau protein for neuroimaging biomarker validation

Abnormal tau inclusions are hallmarks of Alzheimer's disease and predictors of clinical decline. Several tau PET tracers are available for neurodegenerative disease research, opening avenues for molecular diagnosis in vivo. However, few have been approved for clinical use. Understanding the neurobiological basis of PET signal validation remains problematic because it requires a large-scale, voxel-to-voxel correlation between PET and (immuno) histological signals. Large dimensionality of whole human brains, tissue deformation impacting co-registration, and computing requirements to process terabytes of information preclude proper validation. We developed a computational pipeline to identify and segment particles of interest in billion-pixel digital pathology images to generate quantitative, 3D density maps. The proposed convolutional neural network for immunohistochemistry samples, IHCNet, is at the pipeline's core. We have successfully processed and immunostained over 500 slides from two whole human brains with three phospho-tau antibodies (AT100, AT8, and MC1), spanning several terabytes of images. Our artificial neural network estimated tau inclusion from brain images, which performs with ROC AUC of 0.87, 0.85, and 0.91 for AT100, AT8, and MC1, respectively. Introspection studies further assessed the ability of our trained model to learn tau-related features. We present an end-to-end pipeline to create terabytes-large 3D tau inclusion density maps co-registered to MRI as a means to facilitate validation of PET tracers.

60 APPLIED LIFE SCIENCES↗

Data-Efficient Dimensionality Reduction and Surrogate Modeling of High-Dimensional Stress Fields

Tensor datatypes representing field variables like stress, displacement, velocity, etc., have increasingly become a common occurrence in data-driven modeling and analysis of simulations. Numerous methods [such as convolutional neural networks (CNNs)] exist to address the meta-modeling of field data from simulations. As the complexity of the simulation increases, so does the cost of acquisition, leading to limited data scenarios. Modeling of tensor datatypes under limited data scenarios remains a hindrance for engineering applications. Here, in this article, we introduce a direct image-to-image modeling framework of convolutional autoencoders enhanced by information bottleneck loss function to tackle the tensor data types with limited data. The information bottleneck method penalizes the nuisance information in the latent space while maximizing relevant information making it robust for limited data scenarios. The entire neural network framework is further combined with robust hyperparameter optimization. We perform numerical studies to compare the predictive performance of the proposed method with a dimensionality reduction-based surrogate modeling framework on a representative linear elastic ellipsoidal void problem with uniaxial loading. The data structure focuses on the low-data regime (fewer than 100 data points) and includes the parameterized geometry of the ellipsoidal void as the input and the predicted stress field as the output. The results of the numerical studies show that the information bottleneck approach yields improved overall accuracy and more precise prediction of the extremes of the stress field. Additionally, an in-depth analysis is carried out to elucidate the information compression behavior of the proposed framework.

artificial intelligence↗

An efficient surrogate model of secondary electron formation and evolution

This work extends the adjoint-deep learning framework for runaway electron (RE) evolution, developed by McDevitt et al. [Phys. Plasmas 32, 042503 (2025)], to account for large-angle collisions. By incorporating large-angle collisions, the framework allows the avalanche of REs to be captured, an essential component of RE dynamics. This extension is accomplished by using a Rosenbluth–Putvinski approximation to estimate the distribution of secondary electrons generated by large-angle collisions. By evolving both the primary and multiple generations of secondary electrons, the present formulation can capture both the detailed temporal evolution of a RE population beginning from an arbitrary initial momentum space distribution, along with providing approximations to the saturated growth and decay rates of the RE population. Predictions of the adjoint-deep learning framework are verified against a traditional RE solver, with good agreement present across a broad range of parameters.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗

Molecular dynamics simulation of metallic Al–Ce liquids using a neural network machine learning interatomic potential

Al-rich Al-Ce alloys have the possibility of replacing heavier steel and cast-irons for use in high-temperature applications. Knowledge about the structures and properties of Al-Ce alloys at liquid state is vital for optimizing the manufacture process to produce desired allows. However, reliable molecular dynamics simulation of Al-Ce alloy systems remains a great challenge due to the lack of accurate Al-Ce interatomic potential. In this work, an artificial neural network (ANN) deep machine learning (ML) method is used to develop a reliable interatomic potential for Al-Ce alloy. Ab initio molecular dynamics (AIMD) simulation data on Al-Ce liquid with small unit cell (~200 atoms) and on the known Al-Ce crystalline compounds are collected to train the interatomic potential using ANN-ML. The obtained ANN-ML model reproduces well the energies, forces, and atomic structure of Al 90 Ce 10 liquid and crystalline phases of Al-Ce compounds in comparison with ab initio results. The developed ANN-ML potential is applied in molecular dynamics simulations to study the structures and properties of metallic Al 90 Ce 10 liquid, which would provide useful insight for guiding experimental process to produce desired Al-Ce allows.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

Explaining System-Level Prognostics with Established Machine Learning Methods

System-level prognostics is crucial for ensuring reliability and enabling predictive maintenance in complex systems with interconnected components. This study presents a framework that integrates data-driven methods to predict the remaining useful life (RUL) of a subsystem under multiple and concurrent faults within a nuclear power plant system with explainable artificial intelligence (XAI). A nuclear power plant (NPP) operation was simulated to model the degradation behavior of NPP components, and four machine learning models—Gradient Boosting Regressor (GBR), Support Vector Regressor (SVR), Fully Connected Neural Network (FCNN), and Long Short-Term Memory (LSTM)—were evaluated for prognostics with a novel system RUL parameter. The LSTM model demonstrated potential superior repeatability, while SHAP (SHapley Additive exPlanations) for explainability provided consistent and trustworthy global explanations. In contrast, LIME (Local Interpretable Model-agnostic Explanations) offered localized interpretability but showed reduced stability for sequential data. Key findings include the interplay between component-level degradation and system-wide performance, with LSTM effectively capturing these dynamics through sequence-level predictions. The XAI techniques enhanced transparency by identifying critical features influencing model predictions and aligning with domain knowledge. Furthermore, this framework has significant implications for improving trust and understanding in predictive maintenance, particularly in safety-critical industries like nuclear energy.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗

Solving Inverse Stochastic Problems from Discrete Particle Observations Using the Fokker--Planck Equation and Physics-Informed Neural Networks

The Fokker--Planck (FP) equation governing the evolution of the probability density function (PDF) is applicable to many disciplines, but it requires specification of the coefficients for each case, which can be functions of space-time and not just constants and hence require the development of a data-driven modeling approach. When the data available is directly on the PDF, there exist methods for inverse problems that can be employed to infer the coefficients and thus determine the FP equation and subsequently obtain its solution. Herein, we address a more realistic scenario, where only sparse data are given on the particles' positions at a few time instants, which are not sufficient to accurately construct directly the PDF even at those times from existing methods, e.g., kernel estimation algorithms. To this end, we develop a general framework based on physics-informed neural networks (PINNs) that introduces a new loss function using the Kullback--Leibler divergence to connect the stochastic samples with the FP equation to simultaneously learn the equation and infer the multidimensional PDF at all times. In particular, we consider two types of inverse problems, type I, where the FP equation is known but the initial PDF is unknown, and type II, in which, in addition to the unknown initial PDF, the drift and diffusion terms are also unknown. In both cases, we investigate problems with either Brownian or Lévy noise or a combination of both. Here, we demonstrate the new PINN framework in detail in the one-dimensional (1D) case, but we also provide results for up to five dimensions demonstrating that we can infer both the FP equation and dynamics simultaneously at all times with high accuracy using only very few discrete observations of the particles.

97 MATHEMATICS AND COMPUTING↗

A deep learning-accelerated data assimilation and forecasting workflow for commercial-scale geologic carbon storage

Fast assimilation of monitoring data to update forecasts of pressure buildup and carbon dioxide (CO 2 ) plume migration under geologic uncertainties is a challenging problem in geologic carbon storage. The high computational cost of data assimilation with a high-dimensional parameter space impedes fast decision-making for commercial-scale reservoir management. We propose to leverage physical understandings of porous medium flow behavior with deep learning techniques to develop a fast data assimilation-reservoir response forecasting workflow. Applying an Ensemble Smoother Multiple Data Assimilation (ES-MDA) framework, the workflow updates geologic properties and predicts reservoir performance with quantified uncertainty from pressure history and CO 2 plumes interpreted through seismic inversion. As the most computationally expensive component in such a workflow is reservoir simulation, we developed surrogate models to predict dynamic pressure and CO 2 plume extents under multi-well injection. The surrogate models employ deep convolutional neural networks, specifically, a wide residual network and a residual U-Net. The workflow is validated against a flat threedimensional reservoir model representative of a clastic shelf depositional environment. Intelligent treatments are applied to bridge between quantities in a true-3D reservoir model and those in a single-layer reservoir model. The workflow can complete history matching and reservoir forecasting with uncertainty quantification in less than one hour on a mainstream personal workstation.

25 ENERGY STORAGE↗

A comparison of deep-learning-based inpainting techniques for experimental X-ray scattering

The implementation is proposed of image inpainting techniques for the reconstruction of gaps in experimental X-ray scattering data. The proposed methods use deep learning neural network architectures, such as convolutional autoencoders, tunable U-Nets, partial convolution neural networks and mixed-scale dense networks, to reconstruct the missing information in experimental scattering images. In particular, the recovered pixel intensities are evaluated against their corresponding ground-truth values using the mean absolute error and the correlation coefficient metrics. The results demonstrate that the proposed methods achieve better performance than traditional inpainting algorithms such as biharmonic functions. Overall, tunable U-Net and mixed-scale dense network architectures achieved the best reconstruction performance among all the tested algorithms, with correlation coefficient scores greater than 0.9980.

97 MATHEMATICS AND COMPUTING↗

A co-kurtosis PCA based dimensionality reduction with nonlinear reconstruction using neural networks

For turbulent reacting flow systems, identification of low-dimensional representations of the thermo-chemical state space is vitally important, primarily to significantly reduce the computational cost of device-scale simulations. Principal component analysis (PCA), and its variants, are a widely employed class of methods. Recently, an alternative technique that focuses on higher-order statistical interactions, co-kurtosis PCA (CoK-PCA), has been shown to effectively provide a low-dimensional representation by capturing the stiff chemical dynamics associated with spatiotemporally localized reaction zones. While its effectiveness has only been demonstrated based on a priori analyses with linear reconstruction, in this work, we employ nonlinear techniques to reconstruct the full thermo-chemical state and evaluate the efficacy of CoK-PCA compared to PCA. Specifically, we combine a CoK-PCA-/PCA-based dimensionality reduction (encoding) with an artificial neural network (ANN) based reconstruction (decoding) and examine, a priori, the reconstruction errors of the thermo-chemical state. In addition, we evaluate the errors in species production rates and heat release rates, which are nonlinear functions of the reconstructed state, as a measure of the overall accuracy of the dimensionality reduction technique. We employ four datasets to assess CoK-PCA/PCA coupled with ANN-based reconstruction: zero-dimensional (homogeneous) reactor for autoignition of an ethylene/air mixture that has conventional single-stage ignition kinetics, a dimethyl ether (DME)/air mixture which has two-stage (low and high temperature) ignition kinetics, a one-dimensional freely propagating premixed ethylene/air laminar flame, and a two-dimensional dataset representing turbulent autoignition of ethanol in a homogeneous charge compression ignition (HCCI) engine. Finally, results from the analyses demonstrate the robustness of the CoK-PCA based low-dimensional manifold with ANN reconstruction in accurately capturing the data, specifically from the reaction zones.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

ADVANCED-STEM-BASED DEEP LEARNING FOR SEMANTIC SEGMENTATION OF DEFECTS IN STEELS

In this work, we developed a novel deep convolutional neural network (DCNN) model, called DefectNet, for robust and automated semantic segmentation of three crystallographic defects including line dislocations, precipitates and voids commonly observed in structural metals and alloys. In previous work, we established an experimental protocol for a diffraction contrast imaging scanning transmission electron microscopy (DCI STEM) technique and tailored it specifically for imaging defects in popular iron-based structural alloy. Thus, the DefectNet was trained over a small but high-quality DCI STEM defect image sets of a HT-9 martensitic steel before and after neutron irradiation for 111.8 dpa at 412°C. For the defect quantification that typically takes at least half an hour even for an expert, with a good model the machine learning algorithm can produce more reproducible and reliable quantification in a few seconds. This work demonstrates the feasibility of using deep learning algorithms for fast and accurate defect quantitative analysis.

Machine Learning, Radiation effects, Image analysi↗

Autoencoder Neural Network for chemically reacting systems

Incorporating detailed chemical kinetic models is critical for accurate simulations of reacting flows. However, detailed models involve a large number of thermochemical (TC) state variables. Solving the governing equations to evolve these TC variables becomes impractical for real-world applications. In this work, we propose an autoencoder (AE) neural network (NN)-based reduced model to accelerate such simulations. The AE NN is first trained to find a low-dimensional latent representation of the TC states. Then, the evolving state of a chemical system can be tracked by solving the equations of the latent variables instead of the original TC equations. We demonstrate the reduced model in a syngas CO/H 2 combustion system, using training data collected from canonical perfectly stirred reactors (PSRs). It is found that the AE model can reduce the dimension of the combustion system from 12 to 2 while maintaining low reconstruction error and excellent elemental mass conservation for the test dataset. In the a posteriori test, the combustion states obtained from solving the two latent equations are compared to those from solving the 12 equations of the full model. The AE reduced method is found to be able to capture the diverse combustion states on the top two branches of the S-curve well including the extinction turning point, but with higher prediction errors for states near the ignition turning point.

97 MATHEMATICS AND COMPUTING↗

A Novel Deep Reinforcement Learning Approach to Traffic Signal Control with Connected Vehicles

The advent of connected vehicle (CV) technology offers new possibilities for a revolution in future transportation systems. With the availability of real-time traffic data from CVs, it is possible to more effectively optimize traffic signals to reduce congestion, increase fuel efficiency, and enhance road safety. The success of CV-based signal control depends on an accurate and computationally efficient model that accounts for the stochastic and nonlinear nature of the traffic flow. Without the necessity of prior knowledge of the traffic system’s model architecture, reinforcement learning (RL) is a promising tool to acquire the control policy through observing the transition of the traffic states. In this paper, we propose a novel data-driven traffic signal control method that leverages the latest in deep learning and reinforcement learning techniques. By incorporating a compressed representation of the traffic states, the proposed method overcomes the limitations of the existing methods in defining the action space to include more practical and flexible signal phases. The simulation results demonstrate the convergence and robust performance of the proposed method against several existing benchmark methods in terms of average vehicle speeds, queue length, wait time, and traffic density.

42 ENGINEERING↗