Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “deep learning method”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,117 records · Page 62

Probabilistic Programming for Transportable Source Characterization and Uncertainty Quantification of the North Korean Nuclear Tests 2006–2017

Here, we introduce a transportable technique to determine the yield and depth of burial (DOB) from seismic source spectra of underground nuclear explosions. We demonstrate this technique on the six declared North Korean nuclear tests. This approach derives source spectra in absolute units from regional phase (Pg) amplitudes by correcting the observations for geometric spreading, attenuation, and site amplification. We couple the source spectra and explosion source models with a probabilistic programming framework that integrates deep learning techniques and Bayesian modeling. This approach permits the exchange of information across various data categories to quantify both the data and model uncertainty. This technique stands out as an innovative use of broad‐area propagation models, making it transportable across various geologic settings. This method proves to be effective in scenarios with diverse and/or limited observational data, even when the source depth is unknown. We present new independent estimates of absolute yield and DOB that are consistent with the prior assessments, underscoring the potential of this method in enhancing transportable nuclear explosion monitoring capabilities.

58 GEOSCIENCES↗

Automatic Building Feature Extraction Toolkit (AutoBFE) v1.0

In recent years advances in the performance of machine learning methods have considerably improved the ability to accurately extract information from images. Identification of building characteristics from imagery is an example of an application that is time consuming and costly to perform manually, and where automatic feature extraction holds great promise. A major contribution of the work conducted in this project is the provision of open, transparent, and replicable solutions for buildings features extraction from images (e.g., satellite/aerial images). In this first version of the toolkit we are focussing on extracting building footprints from satellite/aerial images using state of the art deep learning image segmentation algorithms.

Touzani, Samir↗

Why it is Unfortunate that Linear Machine Learning “Works” so well in Electromechanical Switching of Ferroelectric Thin Films

Machine learning (ML) is relied on for materials spectroscopy. It is challenging to make ML models fail because statistical correlations can mimic the physics without causality. Here, using a benchmark band-excitation piezoresponse force microscopy polarization spectroscopy (BEPS) dataset the pitfalls of the so-called “better”, “faster”, and “less-biased” ML of electromechanical switching are demonstrated and overcome. Using a toy and real experimental dataset, it is demonstrated how linear nontemporal ML methods result in physically reasonable embedding (eigenvalues) while producing nonsensical eigenvectors and generated spectra, promoting misleading interpretations. A new method of unsupervised multimodal hyperspectral analysis of BEPS is demonstrated using long-short-term memory (LSTM) β-variational autoencoders (β-VAEs) . By including LSTM neurons, the ordinal nature of ferroelectric switching is considered. Further, to improve the interpretability of the latent space, a variational Kullback–Leibler-divergency regularization is imposed . Finally, regularization scheduling of β as a disentanglement metric is leveraged to reduce user bias. Combining these experiment-inspired modifications enables the automated detection of ferroelectric switching mechanisms, including a complex two-step, three-state one. Ultimately, this work provides a robust ML method for the rapid discovery of electromechanical switching mechanisms in ferroelectrics and is applicable to other multimodal hyperspectral materials spectroscopies.

36 MATERIALS SCIENCE↗

Multi deep learning-based stochastic microstructure reconstruction and high-fidelity micromechanics simulation of time-dependent ceramic matrix composite response

A multi deep learning-based framework is developed for efficient, automated microstructure reconstruction and generation of stochastic representative volume elements (SRVEs) with periodic boundary conditions (PBCs) for accurate modeling of ceramic matrix composite (CMC) response. The methodology comprises a convolutional neural network coupled with regression layers to act as a vanilla regression network for semantic segmentation of the microstructure, allowing accurate characterization of the phases and their distributions at the microscale. Scanning electron microscope and confocal microscope are used to obtain C/SiNC and SiC/SiNC CMCs micrographs for vanilla regression testing. Microstructure variability in terms of fiber volume fraction and porosity are quantified through the output regression layer, ensuring accurate representation of material variability in SRVE construction. Generative adversarial network (GAN) and its variants are designed to produce high-fidelity SRVE, spanning CMCs microstructure variability space. A circular padding algorithm is developed to generate SRVEs with PBCs during training of GANs. The accuracy of the generated SRVEs is established through micromechanics simulations, where an efficient formulation of the high-fidelity generalized methods of cells (HFGMC) approach is used to compute the effective mechanical properties. Furthermore, an iterative algorithm is implemented in the HFGMC solver to simulate time-dependent deformation of SiC/SiNC subjected to creep loading conditions.

36 MATERIALS SCIENCE↗

Understanding Adsorption and Reactions at Aqueous Oxide Interfaces with Neural Network Potential Molecular Dynamics

Chemical processes at metal oxide−water interfaces are of central importance in geochemistry, biology, and energy technologies. A better understanding of these processes would allow us to make a significant step toward optimizing and controlling them, which could in turn lead to broader impacts. Computational modeling is indispensable to accomplishing this task because complexity and disorder often make it difficult to extract atomistic information from experiments. Balancing computational cost and accuracy, simulation schemes based on efficient machine learning representations of the potential energy surface (PES) predicted by ab initio calculations have become increasingly popular over the past decade. In particular, several studies have demonstrated the ability of machine learning models to accurately reproduce the complex ab initio PESs of aqueous oxide interfaces, allowing simulations of systems and processes that are not accessible with ab initio methods. In this Account, we review our recent efforts to understand adsorption processes and reactions at aqueous oxide interfaces using deep potential molecular dynamics (DPMD), a simulation scheme employing deep neural networks (DNNs), which has proven to be quite successful in accurately describing many different systems in the condensed phase. After summarizing the DPMD methodology, we first review our work on the acid−base chemistry of oxide surfaces in contact with water, a fundamental characteristic that controls proton transfer and surface charge at the interface. We focus on the aqueous interface of rutile IrO 2 , an oxide material thus far considered the best catalyst for the oxygen evolution reaction (OER). We show that this interface is characterized by a large fraction of dissociated water and a strong Brønsted acidity of the surface sites, in good agreement with the experimentally measured value of the point of zero proton charge. In our second example, we investigate how the adsorption of organic species from ambient air or water affects the structure and wettability of the aqueous interfaces of TiO 2 , a prototypical photocatalytic material. This is a question that is relevant to understanding the UV-induced hydrophilicity of TiO 2 surfaces, a property at the basis of self-cleaning windows and related applications. Specifically focusing on formic and acetic acids, the two most common atmospheric organic acids, our simulations reveal that these acids control the wettability of TiO 2 largely through acid−base chemistry at the interface rather than chemisorption on the oxide surface, a finding that could help improve the design of self-cleaning surfaces and photocatalytic devices. Finally, we review our recent study of methanol at TiO 2 −water interfaces, a system whose interest is largely motivated by the role of methanol in enhancing photocatalytic hydrogen evolution on TiO 2 . Our simulations provide mechanistic insights into the coupled roles of the organic adsorbate and water at the TiO 2 interface, with implications for how methanol enhances the activity of H 2 evolution.

adsorption↗

A Quantum-Classical Collaborative Training Architecture Based on Quantum State Fidelity

Recent advancements have highlighted the limitations of current quantum systems, particularly the restricted number of qubits available on near-term quantum devices. This constraint greatly inhibits the range of applications that can leverage quantum computers. Moreover, as the available qubits increase, the computational complexity grows exponentially, posing additional challenges. Consequently, there is an urgent need to use qubits efficiently and mitigate both present limitations and future complexities. To address this, existing quantum applications attempt to integrate classical and quantum systems in a hybrid framework. In this study, we concentrate on quantum deep learning and introduce a collaborative classical-quantum architecture called co-TenQu. The classical component employs a tensor network for compression and feature extraction, enabling higher-dimensional data to be encoded onto logical quantum circuits with limited qubits. On the quantum side, we propose a quantum-state-fidelity-based evaluation function to iteratively train the network through a feedback loop between the two sides. co-TenQu has been implemented and evaluated with both simulators and the IBM-Q platform. Compared to state-of-the-art approaches, co-TenQu enhances a classical deep neural network by up to 41.72% in a fair setting. Additionally, it outperforms other quantum-based methods by up to 1.9 times and achieves similar accuracy while utilizing 70.59% fewer qubits.

42 ENGINEERING↗

GRinding Automated Classification Engine

This work is an ML-driven framework for automated surface analysis of microscopy images. We create a training dataset by imaging stainless steel samples to benchmark four developed deep neural network architectures. These models, based on a YOLOv8n-cls backend, integrate image features and process metadata using various fusion methods to distinguish between acceptable and unacceptable surface finishes. This code is associated with publication "Classifying Alloy Surface Preparation Quality with Metadata-Infused Machine Learning for Rapid Alloy Discovery" for project APEX LDRD-ER (25-ERD-039)

Gongora, AldairE [Lawrence Livermore National Labo↗

Oracle-Preserving Latent Flows

A fundamental task in data science is the discovery, description, and identification of any symmetries present in the data. We developed a deep learning methodology for the simultaneous discovery of multiple non-trivial continuous symmetries across an entire labeled dataset. The symmetry transformations and the corresponding generators are modeled with fully connected neural networks trained with a specially constructed loss function, ensuring the desired symmetry properties. The two new elements in this work are the use of a reduced-dimensionality latent space and the generalization to invariant transformations with respect to high-dimensional oracles. The method is demonstrated with several examples on the MNIST digit dataset, where the oracle is provided by the 10-dimensional vector of logits of a trained classifier. We find classes of symmetries that transform each image from the dataset into new synthetic images while conserving the values of the logits. We illustrate these transformations as lines of equal probability (“flows”) in the reduced latent space. These results show that symmetries in the data can be successfully searched for and identified as interpretable non-trivial transformations in the equivalent latent space.

97 MATHEMATICS AND COMPUTING↗

Quantifying uncertainty for deep learning based forecasting and flow-reconstruction using neural architecture search ensembles

Classical problems in computational physics such as data-driven forecasting and signal reconstruction from sparse sensors have recently seen an explosion in deep neural network (DNN) based algorithmic approaches. However, most DNN models do not provide uncertainty estimates, which are crucial for establishing the trustworthiness of these techniques in downstream decision making tasks and scenarios. In recent years, ensemble-based methods have achieved significant success for the uncertainty quantification in DNNs on a number of benchmark problems. However, their performance on real-world applications remains under-explored. In this work, we present an automated approach to DNN discovery and demonstrate how this may also be utilized for ensemble-based uncertainty quantification. Specifically, we propose the use of a scalable neural and hyperparameter architecture search for discovering an ensemble of DNN models for complex dynamical systems. We highlight how the proposed method not only discovers high-performing neural network ensembles for our tasks, but also quantifies uncertainty seamlessly. This is achieved by using genetic algorithms and Bayesian optimization for sampling the search space of neural network architectures and hyperparameters. Subsequently, a model selection approach is used to identify candidate models for an ensemble set construction. Afterwards, a variance decomposition approach is used to estimate the uncertainty of the predictions from the ensemble. We demonstrate the feasibility of this framework for two tasks — forecasting from historical data and flow reconstruction from sparse sensors for the sea-surface temperature. In conclusion, we demonstrate superior performance from the ensemble in contrast with individual high-performing models and other benchmarks.

Deep ensembles↗

Safe Deep Reinforcement Learning for Active Distribution System Model Predictive Control with EVs and DERs

The temporal and spatial mismatch between PV generation and electric vehicle (EV) charging and discharging may cause voltage violations in active distribution networks. Despite the widespread use of deep reinforcement learning (DRL) in power system optimization and control, it lacks guarantees on constraint satisfaction during both training and deployment. This paper proposes a Lagrangian-based safe DRL approach for model predictive control (MPC) of active distribution systems with large-scale integration of PVs, EVs, and energy storage systems (ESSs). A Transformer-LSTM time-series model is proposed to forecast EV charging demand, which is then formulated as a constraint to ensure charging requirements are met. Using this prediction, a Lagrangian-based safe soft actor-critic (SAC) framework is developed for real-time control in a three-phase unbalanced distribution system, enforcing voltage safety constraints while optimizing the cumulative net reward. By integrating the forecasting model with multi-period constraints, the proposed framework jointly coordinates PV systems, EV charging and discharging, and ESS scheduling within the MPC horizon. Numerical experiments on a modified IEEE 123-bus system with real-world data show that, under a high PV penetration scenario, the proposed method increases the net reward by 30.74% and reduces average voltage violations from 0.0011 p.u. to 0.0002 p.u. compared with standard SAC. Compared with the optimal power flow (OPF) approach, it achieves similar voltage security while yielding lower line losses. It also maintains real-time control capability, reducing operation latency to 53.21 ms per 15-minute control interval. The proposed method remains effective under varying PV/EV penetrations and load conditions.

24 POWER TRANSMISSION AND DISTRIBUTION↗

Identifying build orientation of 3D-printed materials using convolutional neural networks

The advent of additive manufacturing (AM) processes brought with it intense research into various materials and manufacturing processes. At the same time, the need for validation of material properties, as well as study and forecasting of aging, has arisen. Modern imaging techniques, like X-ray computed tomography (XCT), are a convenient vehicle for such studies; however, the large datasets they produce require novel analysis techniques to efficiently extract critical information. Here, we present our work on developing a 3D extension of the ResNet architecture to distinguish between two build orientations of tensile bars produced by AM. Using only information from XCT, our method achieves a 99.3% correct classification at a misidentification of 1%.

36 MATERIALS SCIENCE↗

A Method of Ground-Based Cloud Motion Predict: CCLSTM + SR-Net

Ground-based cloud images can provide information on weather and cloud conditions, which play an important role in cloud cover monitoring and photovoltaic power generation forecasting. However, the cloud motion prediction of ground-based cloud images still lacks advanced and complete methods, and traditional technologies based on image processing and motion vector calculation are difficult to predict cloud morphological changes. In this paper, we propose a cloud motion prediction method based on Cascade Causal Long Short-Term Memory (CCLSTM) and Super-Resolution Network (SR-Net). Firstly, CCLSTM is used to estimate the shape and speed of cloud motion. Secondly, the Super-Resolution Network is built based on perceptual losses to reconstruct the result of CCLSTM and, finally, make it clearer. We tested our method on Atmospheric Radiation Measurement (ARM) Climate Research Facility TSI (total sky imager) images. The experiments showed that the method is able to predict the sky cloud changes in the next few steps.

54 ENVIRONMENTAL SCIENCES↗

State predictive information bottleneck

We report the ability to make sense of the massive amounts of high-dimensional data generated from molecular dynamics simulations is heavily dependent on the knowledge of a low-dimensional manifold (parameterized by a reaction coordinate or RC) that typically distinguishes between relevant metastable states, and which captures the relevant slow dynamics of interest. Methods based on machine learning and artificial intelligence have been proposed over the years to deal with learning such low-dimensional manifolds, but they are often criticized for a disconnect from more traditional and physically interpretable approaches. To deal with such concerns, in this work we propose a deep learning based state predictive information bottleneck approach to learn the RC from high-dimensional molecular simulation trajectories. We demonstrate analytically and numerically how the RC learnt in this approach is connected to the committor in chemical physics and can be used to accurately identify transition states. A crucial hyperparameter in this approach is the time delay or how far into the future the algorithm should make predictions about. Through careful comparisons for benchmark systems, we demonstrate that this hyperparameter choice gives useful control over how coarse-grained we want the metastable state classification of the system to be. We thus believe that this work represents a step forward in systematic application of deep learning based ideas to molecular simulations.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

The PAU Survey: Photometric redshifts using transfer learning from simulations

In this paper, we introduce the DEEPZ deep learning photometric redshift (photo-z) code. As a test case, we apply the code to the PAU survey (PAUS) data in the COSMOS field. DEEPZ reduces the σ68 scatter statistic by 50 percent at iAB = 22.5 compared to existing algorithms. This improvement is achieved through various methods, including transfer learning from simulations where the training set consists of simulations as well as observations, which reduces the need for training data. The redshift probability distribution is estimated with a mixture density network (MDN), which produces accurate redshift distributions. Our code includes an autoencoder to reduce noise and extract features from the galaxy SEDs. It also benefits from combining multiple networks, which lowers the photo-z scatter by 10 percent. Furthermore, training with randomly constructed coadded fluxes adds information about individual exposures, reducing the impact of photometric outliers. In addition to opening up the route for higher redshift precision with narrow bands, these machine learning techniques can also be valuable for broad-band surveys.

79 ASTRONOMY AND ASTROPHYSICS↗

REV-INR: Regularized Evidential Implicit Neural Representation for Uncertainty-Aware Volume Visualization

Applications of Implicit Neural Representations (INRs) have emerged as a promising deep learning approach for compactly representing large volumetric datasets. These models can act as surrogates for volume data, enabling efficient storage and on-demand reconstruction via model predictions. However, conventional deterministic INRs only provide value predictions without insights into the model’s prediction uncertainty or the impact of inherent noisiness in the data. This limitation can lead to unreliable data interpretation and visualization due to prediction inaccuracies in the reconstructed volume. Identifying erroneous results extracted from model-predicted data may be infeasible, as raw data may be unavailable due to its large size. To address this challenge, we introduce REV-INR, Regularized Evidential Implicit Neural Representation, which learns to predict data values accurately along with the associated coordinate-level data uncertainty and model uncertainty using only a single forward pass of the trained REV-INR during inference. By comprehensively comparing and contrasting REV-INR with existing well-established deep uncertainty estimation methods, we show that REV-INR achieves the best volume reconstruction quality with robust data (aleatoric) and model (epistemic) uncertainty estimates using the fastest inference time. Consequently, we demonstrate that REV-INR facilitates assessment of the reliability and trustworthiness of the extracted isosurfaces and volume visualization results, enabling analyses to be solely driven by model-predicted data.

Saklani, Shanu [Indian Institute of Technology, Ka↗

Machine learning models for PDE constrained optimization

Partial differential equation (PDE)-constrained optimization problems arise in a variety of scientific and engineering applications, such as topology optimization, electrodynamics, fluid dynamics, and structural dynamics. However, these problems are often challenging and computationally expensive to solve, due to the need to solve the PDEs within the optimization loop. One approach to reducing the computational cost of these methods while providing convergence guarantees is through inexact trust region methods; this method uses lower fidelity solutions of the PDE at early stages of the optimization and adjusts the required accuracy of inexact PDE solvers as the optimization progresses. In this work, we explore the use of machine learning based surrogate models with these inexact trust region methods. We first demonstrate the potential of this approach by using Gaussian processes as the surrogate model and test this on a simple PDE-constrained optimization problem. We then document explorations into improving the computational costs of evolutional deep neural network / neural Galerkin methods, with the eventual goal of using these methods with the inexact trust region algorithms. We are able to speed up these approaches, albeit at the cost of lower accuracy.

97 MATHEMATICS AND COMPUTING↗

Sensor Fault Detection in Smart Extraterrestrial Habitats Using Unsupervised Learning

Various types of sensors are needed to monitor the health state of smart deep-space habitats. However, measured data can be affected by sensor faults, which influence the health management system and consequently the decision-making. In this paper, an unsupervised learning approach based on convolutional autoencoders (CAEs) is developed to detect anomalies in temperature and pressure sensors. The proposed method is systematically investigated using a habitat simulator (HabSim). Several illustrative examples are demonstrated in the nominal and hazardous states of the habitat, including micrometeorite impact and fire scenarios. The performance of the proposed method using CAEs is compared with that of existing methods using auto-associative neural networks (AANNs) and variational autoencoders. This comparison is based on typical evaluation metrics, including precision, recall, F1 score, training time, and testing time. The effect of temperature–pressure coupling on the detection performance of CAEs and AANNs is explored by training different data-driven models, including one with temperature sensors, one with pressure sensors, and one with both temperature and pressure sensors. The effect of the number of faulty sensors on the performance of CAEs is studied, as with an increase in the number of faulty sensors, redundant information among the sensors is reduced. The capability of CAEs to change the number of sensors without redesigning the network architecture and retraining the neural network is investigated and demonstrated. The capabilities and limitations of the proposed solution are discussed.

Zixin Wang↗

Physics-Guided, Physics-Informed, and Physics-Encoded Neural Networks and Operators in Scientific Computing: Fluid and Solid Mechanics

Abstract Advancements in computing power have recently made it possible to utilize machine learning and deep learning to push scientific computing forward in a range of disciplines, such as fluid mechanics, solid mechanics, materials science, etc. The incorporation of neural networks is particularly crucial in this hybridization process. Due to their intrinsic architecture, conventional neural networks cannot be successfully trained and scoped when data are sparse, which is the case in many scientific and engineering domains. Nonetheless, neural networks provide a solid foundation to respect physics-driven or knowledge-based constraints during training. Generally speaking, there are three distinct neural network frameworks to enforce the underlying physics: (i) physics-guided neural networks (PgNNs), (ii) physics-informed neural networks (PiNNs), and (iii) physics-encoded neural networks (PeNNs). These methods provide distinct advantages for accelerating the numerical modeling of complex multiscale multiphysics phenomena. In addition, the recent developments in neural operators (NOs) add another dimension to these new simulation paradigms, especially when the real-time prediction of complex multiphysics systems is required. All these models also come with their own unique drawbacks and limitations that call for further fundamental research. This study aims to present a review of the four neural network frameworks (i.e., PgNNs, PiNNs, PeNNs, and NOs) used in scientific computing research. The state-of-the-art architectures and their applications are reviewed, limitations are discussed, and future research opportunities are presented in terms of improving algorithms, considering causalities, expanding applications, and coupling scientific and deep learning solvers.

Computer Science↗