Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Deep Neural Networks”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,567 records · Page 87

Deep learning approaches to semantic segmentation of fatigue cracking within cyclically loaded nickel superalloy

Improvements to synchrotron-based micro-computed tomography scanning capabilities have gifted researchers the ability to characterize 4D material thermomechanical responses more thoroughly than ever before. These advancements, however, have brought about new challenges in analyzing the resulting deluge of data. We report on a nickel-based superalloy specimen imaged 26 times in-situ during cyclic loading at Argonne National Laboratory Advanced Photon Source beamline 1ID, in order to monitor crack growth within the microstructure. Therefore, several deep learning approaches which utilize convolutional neural networks are implemented to segment crack features from reconstructed tomography scans. U-Net architecture implementations are found to be especially effective, achieving IoU = 0.995 +/- 0.004 and Matthews correlation coefficient scores of Φ = 0.826 +/- 0.085. These advancements broaden possibilities for scientists seeking to automate segmentation analyses of similar large datasets.

36 MATERIALS SCIENCE↗

Approximating Trajectory Constraints With Machine Learning – Microgrid Islanding With Frequency Constraints

Here, we introduce deep earning aided constraint encoding to tackle the frequency-constraint microgrid scheduling problem. The nonlinear function between system operating condition and frequency nadir is approximated by using a neural network, which admits an exact mixed-integer formulation (MIP). This formulation is then integrated with the scheduling problem to encode the frequency constraint. With the stronger representation power of the neural network, the resulting commands can ensure adequate frequency response in a realistic setting in addition to islanding success. The proposed method is validated on a modified 33-node system. Successful islanding with a secure response is simulated under the scheduled commands using a detailed three-phase model in Simulink. The advantages of our model are particularly remarkable when the inertia emulation functions from wind turbine generators are considered.

24 POWER TRANSMISSION AND DISTRIBUTION↗

Graph neural networks predict energetic and mechanical properties for models of solid solution metal alloy phases

Here, we developed a PyTorch-based architecture called HydraGNN that implements graph convolutional neural networks (GCNNs) to predict the formation energy and the bulk modulus for models of solid solution alloys for various atomic crystal structures and relaxed volumes. We trained the GCNN surrogate model on a dataset for nickel–niobium (NiNb) generated by the embedded atom model (EAM) empirical interatomic potential for demonstration purposes. The dataset was generated by calculating the formation energy and the bulk modulus as a prototypical elastic property for optimized geometries starting from initial body-centered cubic (BCC), face-centered cubic (FCC), and hexagonal compact packed (HCP) crystal structures, with configurations spanning the possible compositional range for each of the three types of initial crystal structures. Numerical results show that the GCNN model effectively predicts both the formation energy and the bulk modulus as function of the optimized crystal structure, relaxed volume, and configurational entropy of the model structures for solid solution alloys.

36 MATERIALS SCIENCE↗

Machine Learning-Based Identification of the Interface Regions for Coupling Local and Nonlocal Models

Local-nonlocal coupling approaches provide a means to combine the computational efficiency of local models and the accuracy of nonlocal models. However, the coupling process can be challenging, requiring expertise to identify the interface between local and nonlocal regions. Here, this study introduces a machine learning-based approach to automatically detect the regions in which the local and nonlocal models should be used. The method uses loading functions evaluated at grid points to decide the model selection at those points. Training of the networks is based on datasets provided by classes of loading functions for which reference coupling configurations are computed using accurate coupled solutions, where accuracy is measured in terms of the relative error between the solution to the coupling approach and the solution to the nonlocal model. We study two approaches that vary in data structure. The first, the full-domain input data approach, uses the entire load vector and outputs a complete label vector, performing a global classification. The second, a window-based approach, processes loads into windows and addresses the problem as a node-wise classification where each window's central point is classified individually. The classification problems are solved via deep learning algorithms based on convolutional neural networks. The performance of these approaches is studied on one-dimensional numerical examples using F1-scores and accuracy metrics. Notably, the windowing approach achieves an accuracy of 0.96 and an F1-score of 0.97, highlighting its potential to automate coupling processes effectively and enhance computational efficiency in material science applications.

97 MATHEMATICS AND COMPUTING↗

Skip-Connected Self-Recurrent Spiking Neural Networks with Joint Intrinsic Parameter and Synaptic Weight Training

As an important class of spiking neural networks (SNNs), recurrent spiking neural networks (RSNNs) possess great computational power and have been widely used for processing sequential data like audio and text. However, most RSNNs suffer from two problems. 1. Due to the lack of architectural guidance, random recurrent connectivity is often adopted, which does not guarantee good performance. 2. Training of RSNNs is in general challenging, bottlenecking achievable model accuracy. To address these problems, we propose a new type of RSNNs called Skip-Connected Self-Recurrent SNNs (ScSr-SNNs). Recurrence in ScSr-SNNs is introduced in a stereotyped manner by adding self-recurrent connections to spiking neurons. The SNNs with self-recurrent connections can realize recurrent behaviors similar to those of more complex RSNNs while the error gradients can be more straightforwardly calculated due to the mostly feedforward nature of the network. The network dynamics is enriched by skip connections between nonadjacent layers. Moreover, we propose a new backpropagation (BP) method called backpropagated intrinsic plasticity (BIP) to further boost the performance of ScSr-SNNs by training intrinsic model parameters. Unlike standard intrinsic plasticity rules that adjust the neuron's intrinsic parameters according to neuronal activity, the proposed BIP method optimizes intrinsic parameters based on the backpropagated error gradient of a well-defined global loss function in addition to synaptic weight training. Here, based on challenging speech, neuromorphic speech, and neuromorphic image datasets, the proposed ScSr-SNNs can boost performance by up to 2.85% compared with other types of RSNNs trained by state-of-the-art BP methods.

97 MATHEMATICS AND COMPUTING↗

A Tailored Convolutional Neural Network for Nonlinear Manifold Learning of Computational Physics Data Using Unstructured Spatial Discretizations

In this work, we propose a nonlinear manifold learning technique based on deep convolutional autoencoders that is appropriate for model order reduction of physical systems in complex geometries. Convolutional neural networks have proven to be highly advantageous for compressing data arising from systems demonstrating a slow-decaying Kolmogorov n-width. However, these networks are restricted to data on structured meshes. Unstructured meshes are often required for performing analyses of real systems with complex geometry. Our custom graph convolution operators based on the available differential operators for a given spatial discretization effectively extend the application space of deep convolutional autoencoders to systems with arbitrarily complex geometry that are typically discretized using unstructured meshes. We propose sets of convolution operators based on the spatial derivative operators for the underlying spatial discretization, making the method particularly well suited to data arising from the solution of partial differential equations. We demonstrate the method using examples from heat transfer and fluid mechanics and show better than an order of magnitude improvement in accuracy over linear methods.

97 MATHEMATICS AND COMPUTING↗

Autonomous nondestructive evaluation of resistance spot welded joints

The application of non-destructive evaluation approaches has attracted strong interests in modern automotive industries. Here, we present an autonomous deep-computing framework to analyze raw videos from infrared systems and to predict weld nugget shape and size with unprecedented accuracy and speed. In a comprehensive training and testing experiment with 90 videos (seven sets of welding material stack-ups), a new method was developed to assemble sufficient datasets for neural network training. Our framework successfully predicts all the nugget shapes with F1 scores that range from 0.84 to 0.92. The total training time on Nvidia DGX station takes less than 10 min for each set of welding material stack-up. The real inference time of an individual dataset (with 30 video frames) takes about 0.005 s. The procedure and methods developed in the study can be applied to other image-based weld property prediction, as well as other manufacturing processes. Furthermore, our well-trained neural networks take limited memory resources (2.3 MB) and are suitable for embedded microprocessors for in-situ welding quality control as edge computing within an intelligent welding framework.

42 ENGINEERING↗

Recurrent and convolutional neural networks for sequential multispectral optoacoustic tomography ( MSOT ) imaging

Abstract Multispectral optoacoustic tomography (MSOT) is a beneficial technique for diagnosing and analyzing biological samples since it provides meticulous details in anatomy and physiology. However, acquiring high through‐plane resolution volumetric MSOT is time‐consuming. Here, we propose a deep learning model based on hybrid recurrent and convolutional neural networks to generate sequential cross‐sectional images for an MSOT system. This system provides three modalities (MSOT, ultrasound, and optoacoustic imaging of a specific exogenous contrast agent) in a single scan. This study used ICG‐conjugated nanoworms particles (NWs‐ICG) as the contrast agent. Instead of acquiring seven images with a step size of 0.1 mm, we can receive two images with a step size of 0.6 mm as input for the proposed deep learning model. The deep learning model can generate five other images with a step size of 0.1 mm between these two input images meaning we can reduce acquisition time by approximately 71%.

Juhong, Aniwat↗

Dynamic Validation of CNN-Based Surrogate Models for Inverter-Based Resources in Open-Source Solvers

Traditionally, distribution system planning has focused on steady-state analyses, with limited consideration of dynamic behavior. However, as large or medium-scale inverter-based resources (IBRs), particularly grid-following (GFL) inverters in commercial or industry buildings, become more prevalent, understanding their dynamic impact is essential for grid planning and operation. This article presents an innovative deep-learning (DL)-approach using convolutional neural networks technique to model the GFL inverters. Developed from real grid-tied commercial IBR transient data, these dynamic DL models overcome proprietary constraints by requiring minimal knowledge of internal converter physics while maintaining high accuracy and flexibility. To demonstrate their applicability, the models were incorporated into GridLAB-D, an open-source, three-phase distribution analysis tool. This integration enables dynamic simulations of large-scale distribution networks with high IBR penetration stability analysis. Rigorous testing and validation, aligned with industry standards, confirmed the reliability and efficiency of this approach, paving the way for enhanced planning and operational assessments of modern power systems.

Deep-learning↗

Deep-learning based artificial intelligence tool for melt pools and defect segmentation

Accelerating fabrication of additively manufactured components with precise microstructures is important for quality and qualification of built parts, as well as for a fundamental understanding of process improvement. Accomplishing this requires fast and robust characterization of melt pool geometries and structural defects in images. This paper proposes a pragmatic approach based on implementation of deep learning models and self-consistent workflow that enable systematic segmentation of defects and melt pools in optical images. Deep learning is based on an image-to-image translation–conditional generative adversarial neural network architecture. An artificial intelligence (AI) tool based on this deep learning model enables fast and incrementally more accurate predictions of the prevalent geometric features, including melt pool boundaries and printing-induced structural defects. We present statistical analysis of geometric features that is enabled by the AI tool, showing strong spatial correlation of defects and the melt pool boundaries. The correlations of widths and heights of melt pools with dataset processing parameters show the highest sensitivity to thermal influences resulting from laser passes in adjacent and subsequent layer passes. The presented models and tools are demonstrated on the aluminum alloy and datasets produced with different sets of processing parameters. However, they have universal quality and could easily be adapted to different material compositions. The method can be easily generalized to microstructural characterizations other than optical microscopy.

additive manufacturing↗

Neural network methods for radiation detectors and imaging

Recent advances in image data proccesing through deep learning allow for new optimization and performance-enhancement schemes for radiation detectors and imaging hardware. This enables radiation experiments, which includes photon sciences in synchrotron and X-ray free electron lasers as a subclass, through data-endowed artificial intelligence. We give an overview of data generation at photon sources, deep learning-based methods for image processing tasks, and hardware solutions for deep learning acceleration. Most existing deep learning approaches are trained offline, typically using large amounts of computational resources. However, once trained, DNNs can achieve fast inference speeds and can be deployed to edge devices. A new trend is edge computing with less energy consumption (hundreds of watts or less) and real-time analysis potential. While popularly used for edge computing, electronic-based hardware accelerators ranging from general purpose processors such as central processing units (CPUs) to application-specific integrated circuits (ASICs) are constantly reaching performance limits in latency, energy consumption, and other physical constraints. These limits give rise to next-generation analog neuromorhpic hardware platforms, such as optical neural networks (ONNs), for high parallel, low latency, and low energy computing to boost deep learning acceleration (LA-UR-23-32395).

edge computing↗

Solving high-dimensional partial differential equations using deep learning

Developing algorithms for solving high-dimensional partial differential equations (PDEs) has been an exceedingly difficult task for a long time, due to the notoriously difficult problem known as the “curse of dimensionality.” This paper introduces a deep learning-based approach that can handle general high-dimensional parabolic PDEs. To this end, the PDEs are reformulated using backward stochastic differential equations and the gradient of the unknown solution is approximated by neural networks, very much in the spirit of deep reinforcement learning with the gradient acting as the policy function. Numerical results on examples including the nonlinear Black–Scholes equation, the Hamilton–Jacobi–Bellman equation, and the Allen–Cahn equation suggest that the proposed algorithm is quite effective in high dimensions, in terms of both accuracy and cost. Furthermore, this opens up possibilities in economics, finance, operational research, and physics, by considering all participating agents, assets, resources, or particles together at the same time, instead of making ad hoc assumptions on their interrelationships.

97 MATHEMATICS AND COMPUTING↗

Using Explainable Artificial Intelligence to Predict Perovskite Solar Cell Electrical Metastability from Operando Photoluminescence Images in Accelerated Stress Testing

Metal halide perovskite (MHP) solar cells exhibit a metastable response to bias governed by coupled ionic–electronic processes, complicating the conventional reciprocity relation between luminescence intensity and device open-circuit voltage (V oc ). This limits the use of luminescence as a diagnostic for device screening or accelerated stress testing, motivating new approaches that can interpret photoluminescence (PL) signals under nonequilibrium conditions. From the artificial intelligence perspective, we develop an explainable deep learning framework that integrates convolutional neural networks (CNN), long short-term memory (LSTM) layers, and an attention mechanism to learn spatiotemporal features from operando photoluminescence PL image sequences. The model achieves a mean absolute error of ±0.027 V in predicting open-circuit voltage transients and reduces extreme-tail errors by up to 78% compared to physics-based reciprocity calculations. Gradient-weighted Class Activation Mapping (Grad-CAM) provides interpretability by highlighting physically meaningful regions such as electrode edges and emergent defect features. From the engineering application perspective, this framework enables accurate, contactless prediction of device V oc and identification of degradation-relevant features during accelerated aging of perovskite solar cells. This approach demonstrates how explainable AI can enhance operando diagnostics and reliability analysis in photovoltaic devices under nonequilibrium conditions.

14 SOLAR ENERGY↗

Geometry-aware framework for deep energy method: An application to structural mechanics with hyperelastic materials

Here, in this work, we introduce a novel physics-informed framework named the Geometry-Aware Deep Energy Method (GADEM) for solving structural mechanics problems on different geometries. As the weak form of the physical system equation (or the energy-based approach) has demonstrated clear advantages compared to the strong form for solving solid mechanics problems, GADEM employs the weak form and aims to infer the solution on multiple shapes of geometries. Integrating a geometry-aware framework into an energy-based method results in an effective physics-informed deep learning model in terms of accuracy and computational cost. Different ways to represent the geometric information and to encode the geometric latent vectors are investigated in this work. We introduce a loss function of GADEM which is minimized based on the potential energy of all considered geometries. An adaptive learning method is also employed for the sampling of collocation points to enhance the performance of GADEM. We present some applications of GADEM to solve solid mechanics problems, including a loading simulation of a toy tire involving contact mechanics and large deformation hyperelasticity. The numerical results of this work demonstrate the remarkable capability of GADEM to infer the solution on various and new shapes of geometries using only one trained model.

97 MATHEMATICS AND COMPUTING↗

DeePMD-kit v2: A software package for deep potential models

DeePMD-kit is a powerful open-source software package that facilitates molecular dynamics simulations using machine learning potentials known as Deep Potential (DP) models. This package, which was released in 2017, has been widely used in the fields of physics, chemistry, biology, and material science for studying atomistic systems. The current version of DeePMD-kit offers numerous advanced features, such as DeepPot-SE, attention-based and hybrid descriptors, the ability to fit tensile properties, type embedding, model deviation, DP-range correction, DP long range, graphics processing unit support for customized operators, model compression, non-von Neumann molecular dynamics, and improved usability, including documentation, compiled binary packages, graphical user interfaces, and application programming interfaces. This article presents an overview of the current major version of the DeePMD-kit package, highlighting its features and technical details. Additionally, this article presents a comprehensive procedure for conducting molecular dynamics as a representative application, benchmarks the accuracy and efficiency of different models, and discusses ongoing developments.

97 MATHEMATICS AND COMPUTING↗

Multi-fidelity information fusion with concatenated neural networks

Recently, computational modeling has shifted towards the use of statistical inference, deep learning, and other data-driven modeling frameworks. Although this shift in modeling holds promise in many applications like design optimization and real-time control by lowering the computational burden, training deep learning models needs a huge amount of data. This big data is not always available for scientific problems and leads to poorly generalizable data-driven models. This gap can be furnished by leveraging information from physics-based models. Exploiting prior knowledge about the problem at hand, this study puts forth a physics-guided machine learning (PGML) approach to build more tailored, effective, and efficient surrogate models. For our analysis, without losing its generalizability and modularity, we focus on the development of predictive models for laminar and turbulent boundary layer flows. In particular, we combine the self-similarity solution and power-law velocity profile (low-fidelity models) with the noisy data obtained either from experiments or computational fluid dynamics simulations (high-fidelity models) through a concatenated neural network. We illustrate how the knowledge from these simplified models results in reducing uncertainties associated with deep learning models applied to boundary layer flow prediction problems. The proposed multi-fidelity information fusion framework produces physically consistent models that attempt to achieve better generalization than data-driven models obtained purely based on data. While we demonstrate our framework for a problem relevant to fluid mechanics, its workflow and principles can be adopted for many scientific problems where empirical, analytical, or simplified models are prevalent. In line with grand demands in novel PGML principles, this work builds a bridge between extensive physics-based theories and data-driven modeling paradigms and paves the way for using hybrid physics and machine learning modeling approaches for next-generation digital twin technologies.

42 ENGINEERING↗

HomPINNs: Homotopy physics-informed neural networks for learning multiple solutions of nonlinear elliptic differential equations

Physics-informed neural networks (PINNs) based machine learning is an emerging framework for solving nonlinear differential equations. However, due to the implicit regularity of neural network structure, PINNs can only find the flattest solution in most cases by minimizing the loss functions. In this paper, we combine PINNs with the homotopy continuation method, a classical numerical method to compute isolated roots of polynomial systems, and propose a new deep learning framework, named homotopy physics-informed neural networks (HomPINNs), for solving multiple solutions of nonlinear elliptic differential equations. The implementation of an HomPINN is a homotopy process that is composed of the training of a fully connected neural network, named the starting neural network, and training processes of several PINNs with different tracking parameters. The starting neural network is to approximate a starting function constructed by the trivial solutions, while other PINNs are to minimize the loss functions defined by boundary condition and homotopy functions, varying with different tracking parameters. These training processes are regraded as different steps of a homotopy process, and a PINN is initialized by the well-trained neural network of the previous step, while the first starting neural network is initialized using the default initialization method. Finally, several numerical examples are presented to show the efficiency of our proposed HomPINNs, including reaction-diffusion equations with a heart-shaped domain.

97 MATHEMATICS AND COMPUTING↗

Understanding important features of deep learning models for segmentation of high-resolution transmission electron microscopy images

Cutting edge deep learning techniques allow for image segmentation with great speed and accuracy. However, application to problems in materials science is often difficult since these complex models may have difficultly learning meaningful image features that would enable extension to new datasets. In situ electron microscopy provides a clear platform for utilizing automated image analysis. In this work, we consider the case of studying coarsening dynamics in supported nanoparticles, which is important for understanding, for example, the degradation of industrial catalysts. By systematically studying dataset preparation, neural network architecture, and accuracy evaluation, we describe important considerations in applying deep learning to physical applications, where generalizable and convincing models are required. With a focus on unique challenges that arise in high-resolution images, we propose methods for optimizing performance of image segmentation using convolutional neural networks, critically examining the application of complex deep learning models in favor of motivating intentional process design.

36 MATERIALS SCIENCE↗