Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “explainable neural networks”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 127 records · Page 7

Super Resolving Unrolled Neural Networks for Remote Sensing

In remote sensing systems, the capabilities of the system are constrained by the complex interactions between size, weight, and power (SWAP) of potential designs. In electro-optical (EO) systems, examples of these critical parameters include the system’s sensitivity and resolution. Those parameters can be increased by ever larger optical apertures and focal planes but at the cost of more SWAP. Multi-image super resolution (MISR) techniques allow resolution to be enhanced via computation rather than more sophisticated optical hardware. These algorithms combine multiple images together into a single, higher resolution image, trading temporal resolution and computation for spatial resolution. Fielded MISR techniques, such as Drizzle, can require several hundred images to create a single super resolved image, implying reduced temporal resolution, increased data acquisition load, and limiting mission applications. Iterative techniques, such as model-based image reconstruction and compressive sensing, have been shown to create super resolved images using fewer images than Drizzle. They do this by posing an optimization problem that balances accuracy between a highly accurate physical model and an image model. In the case of super resolution, the physical model is defined by the relation between low resolution input images and the desired high resolution output image. The image model encodes some assumptions about the super resolved image. These assumptions are meant to suppress reconstruction artifacts that arise due to deterministic physical model error, stochastic measurement noise, and potential undersampling. In practice, the performance of iterative methods are limited by imaging models compatible with optimization. Deep learning-based methods can effectively learn image models of arbitrary complexity, but lack the theoretical explainability and robustness of iterative techniques. Consensus equilibrium (CE) generalizes the iterative techniques beyond optimization, enabling blackbox algorithms such as traditional and neural image denoisers to be used as the image model. CE-based approaches retain much of the explainability and robustness of iterative techniques while allowing the expressiveness of machine learning image models to be used. Additionally, by unrolling iterations of CE with an embedded image denoiser, the image denoiser can be further trained and specialized to the specific application with potentially higher quality reconstructions. Under this project, we demonstrated the feasibility of training an unrolled neural network based upon CE. While we didn’t train one, we showed that the CE process is differentiable and its gradient can be tractably computed. We also explored the usage of a variants of CE akin to generative neural works. Most importantly, we applied the CE framework to a number of problems including non-blind deconvolution, upsampling, single-image super resolution, MISR, event-based sensing, and saturated deconvolution. Our MISR prototype creates high quality reconstructions with an order of magnitude fewer images than previous approaches and, critically, produces these reconstructions fast enough for practical usage.

47 OTHER INSTRUMENTATION↗

A Deep Gravity model for mobility flows generation

The movements of individuals within and among cities influence critical aspects of our society, such as well-being, the spreading of epidemics, and the quality of the environment. When information about mobility flows is not available for a particular region of interest, we must rely on mathematical models to generate them. In this work, we propose Deep Gravity, an effective model to generate flow probabilities that exploits many features (e.g., land use, road network, transport, food, health facilities) extracted from voluntary geographic data, and uses deep neural networks to discover non-linear relationships between those features and mobility flows. Our experiments, conducted on mobility flows in England, Italy, and New York State, show that Deep Gravity achieves a significant increase in performance, especially in densely populated regions of interest, with respect to the classic gravity model and models that do not use deep neural networks or geographic data. Deep Gravity has good generalization capability, generating realistic flows also for geographic areas for which there is no data availability for training. Finally, we show how flows generated by Deep Gravity may be explained in terms of the geographic features and highlight crucial differences among the three considered countries interpreting the model’s prediction with explainable AI techniques.

97 MATHEMATICS AND COMPUTING↗

Identifying precursors of daily to seasonal hydrological extremes over the USA using deep learning techniques and climate model ensembles

Focal Area(s): We focus on two areas of crosscutting interest for DOE: 1) predictability of extreme precipitation and drought in the USA and 2) the integration of climate models with new AI tools, such as convolutional neural networks (CNN) and methods to understand their output (e.g. layer-wise relevance propagation; LRP). This project fits into focus area 3 of this call for white paper using AI to gain insight from complex data, including explainable AI tools. Science Challenge: Predicting hydrological extremes is important due to their impacts on people, agriculture and infrastructure. This prediction is difficult due to the infrequent occurrence of extremes and their complexity. However, extreme events can be related to more predictable conditions in the ocean, such as El Nino, long-term soil moisture or large scale modes of climate variability, such as the North Atlantic Oscillation (NAO).

54 ENVIRONMENTAL SCIENCES↗

Enhanced Oblique Decision Tree Enabled Policy Extraction for Deep Reinforcement Learning in Power System Emergency Control

Deep reinforcement learning (DRL) algorithms have successfully solved many challenging problems in various power system control scenarios. However, their decision-making process is usually regarded as black-boxes. Furthermore, how DRL models interact with human intelligence remains an open problem. Thus, this paper proposes a policy extraction framework to extract a complex DRL model into an explainable policy. This framework includes three parts: 1) DRL training and data generation. We train an agent for a specific control task and generate data, which contains the control policy of the agent. 2) Policy extraction. We propose an information gain rate based weighted oblique decision tree (IGR-WODT) for DRL policy extraction. 3) Policy evaluation. We define three metrics to evaluate the performance of the proposed approach. A case study for the under-voltage load shedding problem shows that the IGR-WODT presents a performance enhancement compared with DRL, weighted oblique decision tree, and univariate decision tree. The proposed policy extraction method could provide an intuitive explanation of the neural network decision-making process to the dispatchers when making final decisions on power grid operation. Also, the resulted rule-based controller could replace the deep neural network-based controller in many field edge devices with limited computing resources, providing comparable performance.

deep reinforcement learning↗

Explainable and Differentiable Reinforcement Learning for Multi-objective Optimization in Particle Accelerators

Operating particle accelerators involves optimizing multiple goals simultaneously, which can be challenging due to trade-offs among objectives. While evolutionary algorithms like the genetic algorithm (GA) have been used for various Multi-Objective Optimization (MOO) tasks, they are not inherently suited for complex control problems. This talk highlights two variations of Reinforcement Learning (RL) for concurrently optimizing heat load and trip rates at the Continuous Electron Beam Accelerator Facility (CEBAF). The problem involves strict constraints on individual states, actions, and overall energy requirements of the beam. First, this talk highlights how differentiability can be harnessed through a Deep Differentiable Reinforcement Learning (DDRL) approach to address MOO issues within particle accelerators. We examine the DDRL method alongside Model Free Reinforcement Learning (MFRL), GA, and Bayesian Optimization (BO). The performance of these methods is assessed by generating a Pareto-front for two objectives. Our findings indicate that DDRL excels in handling high-dimensional problems more effectively than MFRL, BO, and GA. Next, we will show integration of explainable physics-based constraints into RL algorithms to enhance trans- parency and trust in decision-making processes by enabling users to verify that agents adhere to established physical principles. This surrogate function can be modeled using neural networks or sparse dictionary mod- els. By examining the mathematical form of the learned constraint function, we are able to confirm the agent has learned to use the established physics of each environment provided but the surrogate model. In addi- tion, we find that the introduction of a mathematical functional dictionary based surrogate model enables our reinforcement learning algorithms to reliably converge for difficult high-dimensional accelerator controls environments.

Rajput, Kishansingh [Thomas Jefferson National Acc↗

Runway Sign Classifier: A DAL C Certifiable Machine Learning System

In recent years, the remarkable progress of Machine Learning (ML) technologies within the domain of Artificial Intelligence (AI) systems has presented unprecedented opportunities for the aviation industry, paving the way for further advancements in automation, including the potential for single pilot or fully autonomous operation of large commercial airplanes. However, ML technology faces major incompatibilities with existing airborne certification standards, such as ML model traceability and explainability issues or the inadequacy of traditional coverage metrics. Certification of ML-based airborne systems using current standards is problematic due to these challenges. This paper presents a case study of an airborne system utilizing a Deep Neural Network (DNN) for airport sign detection and classification. Building upon our previous work, which demonstrates compliance with Design Assurance Level (DAL) ”D”, we upgrade the system to meet the more stringent requirements of Design Assurance Level ”C”. To achieve DAL C, we employ an established architectural mitigation technique involving two redundant and dissimilar Deep Neural Networks. The application of novel ML-specific data management techniques further enhances this approach. This work is intended to illustrate how the certification challenges of ML-based systems can be addressed for medium criticality airborne applications.

Flight Software↗

Explainable machine learning reveals that local structural motifs encode the thermodynamic state across the CuZr metallic glass-forming range

Metallic glasses derive their properties from the statistics of local atomic motifs rather than from long-range order, yet a quantitative, chemistry-specific link between motif populations and the underlying glassy state has remained elusive. In this work we combine large-scale molecular dynamics, Voronoi tessellation, deep neural networks, and SHapley Additive exPlanations (SHAP) to identify which local structural motifs define the glassy state of Cu—Zr metallic glasses. A dataset of 17,180 atomistic configurations spanning ten compositions (Cu 20 Zr 80 –Cu 80 Zr 20 ) and four quench rates (10 9 –10 12 K/s) is used to train a feed-forward neural network that regresses temperature across the 50–2000 K liquid–supercooled–glass range, achieving a mean absolute error of 19.89 K and R 2 = 0.9974, confirming that the local structural state is faithfully encoded in motif-level structure. SHAP analysis then reveals that a tightly coupled near-icosahedral family of motifs (coordination numbers (CN) 11–13, including the full icosahedron 001200 and its single-atom-perturbation sibling 10930) collectively encodes the thermodynamic state of the system across the full glass-forming range. The CN = 11–13 ordered members carry negative SHAP values at high populations, tracking the most deeply-quenched configurations, while 10930 shows the reversed signature consistent with its role as a soft-spot host whose population shrinks as the icosahedral network deepens. The analysis demonstrates that explainable machine learning can isolate the minimal motif vocabulary defining the glassy state and recovers the near-icosahedral building blocks previously identified by data-driven analyses of Cu—Zr. The approach provides a general, chemistry-specific route for characterizing the structural state of disordered materials.

36 MATERIALS SCIENCE↗

Deep learning Hamiltonians from disordered image data in quantum materials

The capabilities of image probe experiments are rapidly expanding, providing new information about quantum materials on unprecedented length- and timescales. Many such materials feature inhomogeneous electronic properties with intricate pattern formation on the observable surface. This rich spatial structure contains information about interactions, dimensionality, and disorder—a spatial encoding of the Hamiltonian driving the pattern formation. Image recognition techniques from machine learning are an excellent tool for interpreting information encoded in the spatial relationships in such images. Here, we develop a deep learning framework for using the rich information available in these spatial correlations in order to discover the underlying Hamiltonian driving the patterns. We first vet the method on a known case, scanning near-field optical microscopy on a thin film of V⁢O 2 . We then apply our trained convolutional neural network architecture to new optical microscope images of a different V⁢O 2 film as it goes through the metal-insulator transition. We find that a two-dimensional Hamiltonian with both interactions and random field disorder is required to explain the intricate, fractal intertwining of metal and insulator domains during the transition. This detailed knowledge about the underlying Hamiltonian paves the way for using the model to control the pattern formation via, e.g., tailored hysteresis protocols. Finally, we also introduce a distribution-based confidence measure on the results of a multilabel classifier, which does not rely on adversarial training. In addition, we propose a machine-learning-based criterion for diagnosing a physical system's proximity to criticality.

75 CONDENSED MATTER PHYSICS, SUPERCONDUCTIVITY AND↗

Towards Geospatial Knowledge Graph Infused Neuro-Symbolic AI for Remote Sensing Scene Understanding

Deep learning has proven its effectiveness in numerous tasks for remote sensing scene understanding. However there is an increasing interest to explore fusion of domain-specific background information to the deep neural network to further improve its performance. Remote sensing researchers are also working towards developing models that generalize and adapt to multiple applications. Generalization challenges coupled with the scarcity of large corpora of high-quality noise-free labelled data, have together fueled an interest for leveraging background information. Knowledge graphs serve as excellent choice to represent domain-specific information in a structured, standardized and extensible manner. Integrating symbolic knowledge representations in the form of Knowledge Graph Embedding (KGE) to perform neuro-symbolic reasoning is an emerging research direction promising significant impacts. This vision paper seeks to position ideas and provoke early thoughts toward advancing neuro-symbolic artificial intelligence in the context of geospatial challenges. Specifically, it conceptualizes and elaborates on an architecture for infusing geospatial knowledge from knowledge graph in a deep neural network pipeline. As guiding case studies - land-use land-cover classification, object detection and instance segmentation can benefit from infusing spatio-contextual information with remote sensing imagery. The discussion further reflects on and articulates the challenges and explainable AI opportunities anticipated when scaling and maintaining large-scale geospatial knowledge graphs.

Potnis, Abhishek↗

Edge ML for CAN bus intrusion detection in AVs

Autonomous Vehicles (AVs) are revolutionizing transportation, but their reliance on interconnected cyber-physical systems exposes them to unprecedented cybersecurity risks. This study addresses the critical challenge of detecting real-time cyber intrusions in self-driving vehicles by leveraging a dataset from the Udacity self-driving car project. We simulate four high-impact attack vectors, Denial of Service (DoS), spoofing, replay, and fuzzy attacks, by injecting noise into spatial features (e.g., bounding box coordinates) to replicate adversarial scenarios. We develop and evaluate two lightweight neural network architectures (NN-1 and NN-2) alongside a logistic regression baseline (LG-1) for intrusion detection. The models achieve exceptional performance, with NN-2 attaining an AUC score of 93.15% and 93.15% accuracy, demonstrating their suitability for edge deployment in AV environments. Through explainable AI techniques, we uncover unique forensic fingerprints of each attack type, such as spatial corruption in fuzzy attacks and temporal anomalies in replay attacks, offering actionable insights for feature engineering and proactive defense. Visual analytics, including confusion matrices, ROC curves, and feature importance plots, validate the models' robustness and interpretability. This research sets a new benchmark for AV cybersecurity, delivering a scalable, field-ready toolkit for Original Equipment Manufacturers (OEMs) and policymakers. By aligning intrusion fingerprints with SAE J3061 automotive security standards, we provide a pathway for integrating machine learning into safety-critical AV systems. Our findings underscore the urgent need for security-by-design AI, ensuring that AVs not only drive autonomously but also defend autonomously. This work bridges the gap between theoretical cybersecurity and life-preserving engineering, offering a leap toward safer, more secure autonomous transportation.

97 MATHEMATICS AND COMPUTING↗

Signatures of a liquid–liquid transition in an ab initio deep neural network model for water

Significance Water is central across much of the physical and biological sciences and exhibits physical properties that are qualitatively distinct from those of most other liquids. Understanding the microscopic basis of water’s peculiar properties remains an active area of research. One intriguing hypothesis is that liquid water can separate into metastable high- and low-density liquid phases at low temperatures and high pressures, and the existence of this liquid–liquid transition could explain many of water’s anomalous properties. We used state-of-the-art approaches in computational quantum chemistry, statistical mechanics, and machine learning and obtained evidence consistent with a liquid–liquid transition, supporting the argument for the existence of this phenomenon in real water.

36 MATERIALS SCIENCE↗

Explainable physics-based constraints on reinforcement learning for accelerator optimization

We present a reinforcement learning (RL) framework for optimizing particle accelerator experiments that builds explainable physics-based constraints on agent behavior. The goal is to increase transparency and trust by letting users verify that the agent’s decision-making process incorporates suitable physics. Our algorithm uses a learnable surrogate function for physical observables, such as energy, and uses them to fine-tune how actions are chosen. This surrogate can be represented by a neural network or by an interpretable sparse dictionary model. We test our algorithm on a range of particle accelerator optimization environments designed to emulate the Continuous Electron Beam Accelerator Facility at Jefferson Lab. By examining the mathematical form of the learned constraint function, we are able to confirm the agent has learned to use the established physics of each environment. In addition, we find that the introduction of a physics-based surrogate enables our RL algorithms to reliably converge for difficult high-dimensional accelerator optimization environments.

explainability↗

Discovering a reaction–diffusion model for Alzheimer’s disease by combining PINNs with symbolic regression

Misfolded tau proteins play a critical role in the progression and pathology of Alzheimer's disease. Recent studies suggest that the spatio-temporal pattern of misfolded tau follows a reaction-diffusion type equation. However, the precise mathematical model and parameters that characterize the progression of misfolded protein across the brain remain incompletely understood. Here, we use deep learning and artificial intelligence to discover a mathematical model for the progression of Alzheimer's disease using longitudinal tau positron emission tomography from the Alzheimer's Disease Neuroimaging Initiative database. Specifically, we integrate physics informed neural networks (PINNs) and symbolic regression to discover a reaction-diffusion type partial differential equation for tau protein misfolding and spreading. First, we demonstrate the potential of our model and parameter discovery on synthetic data. Then, we apply our method to discover the best model and parameters to explain tau imaging data from 46 individuals who are likely to develop Alzheimer's disease and 30 healthy controls. Our symbolic regression discovers different misfolding models f(c) for two groups, with a faster misfolding for the Alzheimer's group, f(c) = 0.23c 3 – 1.34c 2 + 1.11c, than for the healthy control group, f(c) = –c 3 + 0.62c 2 + 0.39c. Our results suggest that PINNs, supplemented by symbolic regression, can discover a reaction-diffusion type model to explain misfolded tau protein concentrations in Alzheimer's disease. Furthermore, we expect our study to be the starting point for a more holistic analysis to provide image-based technologies for early diagnosis, and ideally early treatment of neurodegeneration in Alzheimer's disease and possibly other misfolding-protein based neurodegenerative disorders.

60 APPLIED LIFE SCIENCES↗

Comparative study of machine learning techniques for post-combustion carbon capture systems

Computational analysis of countercurrent flows in packed absorption columns, often used in solvent-based post-combustion carbon capture systems (CCSs), is challenging. Typically, computational fluid dynamics (CFD) approaches are used to simulate the interactions between a solvent, gas, and column's packing geometry while accounting for the thermodynamics, kinetics, heat, and mass transfer effects of the absorption process. These simulations can then be used explain a column's hydrodynamic characteristics and evaluate its CO 2 -capture efficiency. However, these approaches are computationally expensive, making it difficult to evaluate numerous designs and operating conditions to improve efficiency at industrial scales. In this work, we comprehensively explore the application of statistical ML methods, convolutional neural networks (CNNs), and graph neural networks (GNNs) to aid and accelerate the scale-up and design optimization of solvent-based post-combustion CCSs. We apply these methods to CFD datasets of countercurrent flows in absorption columns with structured packings characterized by several geometric parameters. We train models to use these parameters, inlet velocity conditions, and other model-specific representations of the column to estimate key determinants of CO 2 -capture efficiency without having to simulate additional CFD datasets. We also evaluate the impact of different input types on the accuracy and generalizability of each model. We discuss the strengths and limitations of each approach to further elucidate the role of CNNs, GNNs, and other machine learning approaches for CO 2 -capture property prediction and design optimization.

97 MATHEMATICS AND COMPUTING↗

A Self-learning Neural Network Approach for Radio Frequency Interference Detection and Removal in Radio Astronomy

In this work, we present a novel neural network (NN) method for the detection and removal of radio frequency interference (RFI) from the raw digitized signal in the signal processing chain of a typical radio astronomy experiment. The main advantage of our method is that it does not require a training set. Instead, our method relies on the fact that the true signal of interest coming from astronomical sources is thermal and therefore described as a Gaussian random process, which cannot be compressed. We employ a variational encoder/decoder network to find the compressible information in the data stream that can explain the most variance with the fewest degrees of freedom. We demonstrate it on a set of toy problems and stored ring buffers from the Baryon Mapping eXperiment prototype. We find that the RFI subtraction is effective at cleaning simulated time streams: While we find that the power spectra of the RFI-cleaned time streams output by the NN suffer from extra signal consistent with additive noise, we find that it is generally around percent level across the band and sub 10% in contaminated spectral channels even when RFI power is an order of magnitude larger than the signal. We discuss the advantages and limitations of this method and possible implementation in the front end of future radio experiments.

79 ASTRONOMY AND ASTROPHYSICS↗

Geometric Measures of Trustworthiness for Machine Learning Predictions

his report details the findings from the research and investigation of Geometric Measures of Trustworthiness for Machine Learning Predictions. We explored the trustworthiness of machine learning (ML) models’ predictions using geometric measures to quantify the similarity of a query point with the training data. Predictive uncertainty in ML can originate from at least three sources: (1) Model uncertainty, which represents the uncertainty in model form (e.g. decision tree, vs neural network) and estimating the model parameters from the training data, (2) Data uncertainty, which represents the natural complexities of the data such as class overlap and inherent noise, and (3) Distributional uncertainty, which represents the mismatch between the training and operational distributions. The proposed measures focus on measuring and explaining the data and distributional uncertainties by measuring the relationships of operational data with the training data.

97 MATHEMATICS AND COMPUTING↗

Integrating Deep Learning and Hydrodynamic Modeling to Improve the Great Lakes Forecast

The Laurentian Great Lakes, one of the world’s largest surface freshwater systems, pose a modeling challenge in seasonal forecast and climate projection. While physics-based hydrodynamic modeling is a fundamental approach, improving the forecast accuracy remains critical. In recent years, machine learning (ML) has quickly emerged in geoscience applications, but its application to the Great Lakes hydrodynamic prediction is still in its early stages. This work is the first one to explore a deep learning approach to predicting spatiotemporal distributions of the lake surface temperature (LST) in the Great Lakes. Our study shows that the Long Short-Term Memory (LSTM) neural network, trained with the limited data from hypothetical monitoring networks, can provide consistent and robust performance. The LSTM prediction captured the LST spatiotemporal variabilities across the five Great Lakes well, suggesting an effective and efficient way for monitoring network design in assisting the ML-based forecast. Furthermore, we employed an explainable artificial intelligence (XAI) technique named SHapley Additive exPlanations (SHAP) to uncover how the features impact the LSTM prediction. Our XAI analysis shows air temperature is the most influential feature for predicting LST in the trained LSTM. The relatively large bias in the LSTM prediction during the spring and fall was associated with substantial heterogeneity of air temperature during the two seasons. In contrast, the physics-based hydrodynamic model performed better in spring and fall yet exhibited relatively large biases during the summer stratification period. Finally, we developed a statistical integration of the hydrodynamic modeling and deep learning results based on the Best Linear Unbiased Estimator (BLUE). The integration further enhanced prediction accuracy, suggesting its potential for next-generation Great Lakes forecast systems.

Xue, Pengfei (ORCID:000000025702421X)↗

Defect Properties, Anion Ordering, and Photochromic Mechanism in Yttrium Oxyhydride

Yttrium oxyhydride (YHO) undergoes a reversible photochromic transition when exposed to ultraviolet light. However, the mechanism for this transformation is not fully understood, and the structure and precise chemical composition of YHO remain under debate. Here, we use first-principles density functional theory calculations with a hybrid functional to study the structure, chemical stability, and point defect properties of YHO. As experiments have shown, we find that YHO prefers a cubic structure, with H and O anions present in equal concentrations and located on tetrahedral sites. Stoichiometric and ordered YHO is chemically stable, but it has a wide band gap of 5.01 eV, considerably larger than that measured in experiments (2.4–3.8 eV). On the other hand, Y4H10O has a smaller band gap of 2.97 eV and also has a region of chemical stability; thus, the actual material may include some fraction of this H-rich structure. The defect chemistry of YHO is dominated by anionic antisite species (H O and O H ), with hydrogen interstitials (H i ) and vacancies (V H ) also present in reasonably high concentrations. We show that antisite disorder lowers the band gap relative to the perfectly ordered structure, bringing the magnitude of the gap into closer agreement with experiment. Based on our calculations of defect migration and the positions of defect states relative to the band edges, we link the onset of photochromic behavior to the reaction H O – → V O 0 + H i – , which follows photoexcitation of a H O + defect. H i – can subsequently migrate away and be trapped by additional H O + defects, contributing to the persistence of the reaction, while the resultant oxygen vacancy, V O 0 , introduces an occupied defect state that leads to optical absorption at visible wavelengths. Our results can explain reported discrepancies between experimental and computational results for YHO, and they allow us to propose specific atomic-scale processes that can lead to photochromism. In conclusion, understanding these mechanisms is key for unlocking YHO’s application in devices ranging from smart windows and optoelectronics to electrochemical synapses for neural networks.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗