SEARCH · Engineering Papers

Results for “explainable”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 55 records · Page 3

All sources NASA NTRS DOE OSTI

DOE OSTI2022

Training calibration-based counterfactual explainers for deep learning models in medical image analysis

The rapid adoption of artificial intelligence methods in healthcare is coupled with the critical need for techniques to rigorously introspect models and thereby ensure that they behave reliably. This has led to the design of explainable AI techniques that uncover the relationships between discernible data signatures and model predictions. In this context, counterfactual explanations that synthesize small, interpretable changes to a given query while producing desired changes in model predictions have become popular. This under-constrained, inverse problem is vulnerable to introducing irrelevant feature manipulations, particularly when the model’s predictions are not well-calibrated. Hence, in this paper, we propose the TraCE (training calibration-based explainers) technique, which utilizes a novel uncertainty-based interval calibration strategy for reliably synthesizing counterfactuals. Given the wide-spread adoption of machine-learned solutions in radiology, our study focuses on deep models used for identifying anomalies in chest X-ray images. Using rigorous empirical studies, we demonstrate the superiority of TraCE explanations over several state-of-the-art baseline approaches, in terms of several widely adopted evaluation metrics. Our findings show that TraCE can be used to obtain a holistic understanding of deep models by enabling progressive exploration of decision boundaries, to detect shortcuts, and to infer relationships between patient attributes and disease severity.

62 RADIOLOGY AND NUCLEAR MEDICINE↗

DOE OSTI2026

Explainable machine learning for incipient anomaly detection in compact molten salt heat exchanger with overlapping feature distributions

High-temperature molten salt-cooled reactors (MSCRs) are a promising next-generation nuclear technology option, offering efficient power conversion and inherent safety features. However, the reliability of these systems depends on the robust operation of heat exchangers (HXs), which are susceptible to failure due to temperature gradients and channel plugging caused by fluid freezing. Conventional monitoring methods, relying on inlet and outlet measurements, lack the spatial resolution needed to detect early-stage faults. We propose a novel design of a compact salt-to-salt matrix-type HX design consisting of interleaved arrays of parallel tubes, with integrated synthetic fiber optic distributed temperature sensing (DTS) to enable localized detection of incipient faults. To evaluate performance of this design, we generate high-fidelity synthetic data using heat transfer computational modeling to simulate channel plugging, and introduce sensor noise for realistic modeling of measurements. The dataset comprises of 97% normal operation and 3% anomaly cases, with each anomaly class representing 1% of the data. These early anomalies result in overlapping temperature profiles between normal and faulty channels, producing a non-separable dataset that challenges traditional classification techniques. We benchmark eight supervised machine learning (ML) models and demonstrate that XGBoost achieves the highest performance. To improve transparency, we develop an explainability framework combining Shapley values and partially ordered sets (POSETs) to quantify and structurally analyze feature importance. This approach identifies both dominant predictors and ambiguous feature relationships, enhancing trust and interpretability. Our results highlight the potential of combining DTS and explainable ML with intelligent feature selection to improve predictive maintenance and ensure operational resilience in advanced nuclear systems.

Prantikos, Konstantinos [Argonne National Laborato↗

DOE OSTI2026

Explainable tokamak-agnostic forecasting of fusion plasma instability via megahertz turbulent fluctuations

Scientific applications of artificial intelligence (AI) often remain limited by device-specific training and unexplained “black-box” approaches, creating fundamental barriers to cross-system generalization. This challenge is critical for nuclear fusion, where future reactors will have limited operational data for AI training. Here, we demonstrate that our neural network, trained solely on megahertz-scale turbulence measurements from one machine (DIII-D), forecasts Type-I edge localized mode (ELM) onsets in a different tokamak (KSTAR) through zero-shot weight transfer following physics-consistent preprocessing without device-specific retraining. Through an explainable AI framework combining gradient-weighted class activation mapping with physics validation, we reveal that our network can internalize physics relationships governing the ELM instabilities rather than memorizing device-specific patterns. The network perceives spatiotemporal features that correlate consistently with independently calculated instability growth rates, magnetohydrodynamic stability limits, and pedestal structure dynamics. Statistical analyses of dimensionally-reduced saliency features reveal the identical triangular features between the saliency representations, instability growth rates, and prediction probability across tokamaks, providing evidence that our forecasting system can show tokamak-agnostic generalization. This work contributes to a foundation for explainable scientific AI systems, where cross-system developments are essential for transcending traditional domain-specific constraints.

AI↗

DOE OSTI2026

Explaining System-Level Prognostics with Established Machine Learning Methods

System-level prognostics is crucial for ensuring reliability and enabling predictive maintenance in complex systems with interconnected components. This study presents a framework that integrates data-driven methods to predict the remaining useful life (RUL) of a subsystem under multiple and concurrent faults within a nuclear power plant system with explainable artificial intelligence (XAI). A nuclear power plant (NPP) operation was simulated to model the degradation behavior of NPP components, and four machine learning models—Gradient Boosting Regressor (GBR), Support Vector Regressor (SVR), Fully Connected Neural Network (FCNN), and Long Short-Term Memory (LSTM)—were evaluated for prognostics with a novel system RUL parameter. The LSTM model demonstrated potential superior repeatability, while SHAP (SHapley Additive exPlanations) for explainability provided consistent and trustworthy global explanations. In contrast, LIME (Local Interpretable Model-agnostic Explanations) offered localized interpretability but showed reduced stability for sequential data. Key findings include the interplay between component-level degradation and system-wide performance, with LSTM effectively capturing these dynamics through sequence-level predictions. The XAI techniques enhanced transparency by identifying critical features influencing model predictions and aligning with domain knowledge. Furthermore, this framework has significant implications for improving trust and understanding in predictive maintenance, particularly in safety-critical industries like nuclear energy.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗

DOE OSTI2026

NuGraph2 with explainability: post-hoc explanations for geometric neural network predictions

With the growing popularity of artificial intelligence (AI) used for scientific applications, the ability of attribute a result to a reasoning process from the network is in high demand for robust scientific generalizations to hold. In this work we aim to motivate the need for and demonstrate the use of post-hoc explainability methods when applied to AI methods used in scientific applications. To this end, we introduce explainability add-ons to the existing graph neural network (GNN) for neutrino tagging, NuGraph2. The explanations take the form of a suite of techniques examining the output of the network (node classifications) and the edge connections between them, and probing of the latent space using novel general-purpose tools applied to this network. We show how none of these methods are singularly sufficient to show network ‘understanding’, but together can give insights into the processes used in classification. While these methods are tested on the NuGraph2 application, they can be applied to a broad range of networks, not limited to GNNs. The code for this work is publicly available on GitHub at https://github.com/voetberg/XNuGraph.

Voetberg, Margaret [Fermilab] (ORCID:0009000527154↗

DOE OSTI2025

SetBERT: the deep learning platform for contextualized embeddings and explainable predictions from high-throughput sequencing

MOTIVATION: High-throughput sequencing (HTS) is a modern sequencing technology used to profile microbiomes by sequencing thousands of short genomic fragments from the microorganisms within a given sample. This technology presents a unique opportunity for artificial intelligence to comprehend the underlying functional relationships of microbial communities. However, due to the unstructured nature of HTS data, nearly all computational models are limited to processing DNA sequences individually. This limitation causes them to miss out on key interactions between microorganisms, significantly hindering our understanding of how these interactions influence the microbial communities as a whole. Furthermore, most computational methods rely on post-processing of samples which could inadvertently introduce unintentional protocol-specific bias. RESULTS: Addressing these concerns, we present SetBERT, a robust pre-training methodology for creating generalized deep learning models for processing HTS data to produce contextualized embeddings and be fine-tuned for downstream tasks with explainable predictions. By leveraging sequence interactions, we show that SetBERT significantly outperforms other models in taxonomic classification with genus-level classification accuracy of 95%. Furthermore, we demonstrate that SetBERT is able to accurately explain its predictions autonomously by confirming the biological-relevance of taxa identified by the model. AVAILABILITY AND IMPLEMENTATION: All source code is available at https://github.com/DLii-Research/setbert. SetBERT may be used through the q2-deepdna QIIME 2 plugin whose source code is available at https://github.com/DLii-Research/q2-deepdna.

Ludwig, David W↗

DOE OSTI2024

Genetic and Functional Diversity Help Explain Pathogenic, Weakly Pathogenic, and Commensal Lifestyles in the Genus Xanthomonas

The genus Xanthomonas has been primarily studied for pathogenic interactions with plants. However, besides host and tissue-specific pathogenic strains, this genus also comprises nonpathogenic strains isolated from a broad range of hosts, sometimes in association with pathogenic strains, and other environments, including rainwater. Based on their incapacity or limited capacity to cause symptoms on the host of isolation, nonpathogenic xanthomonads can be further characterized as commensal and weakly pathogenic. This study aimed to understand the diversity and evolution of nonpathogenic xanthomonads compared to their pathogenic counterparts based on their cooccurrence and phylogenetic relationship and to identify genomic traits that form the basis of a life history framework that groups xanthomonads by ecological strategies. We sequenced genomes of 83 strains spanning the genus phylogeny and identified eight novel species, indicating unexplored diversity. While some nonpathogenic species have experienced a recent loss of a type III secretion system, specifically the hrp2 cluster, we observed an apparent lack of association of the hrp2 cluster with lifestyles of diverse species. We performed association analysis on a large data set of 337 Xanthomonas strains to explain how xanthomonads may have established association with the plants across the continuum of lifestyles from commensals to weak pathogens to pathogens. Presence of distinct transcriptional regulators, distinct nutrient utilization and assimilation genes, transcriptional regulators, and chemotaxis genes may explain lifestyle-specific adaptations of xanthomonads.

59 BASIC BIOLOGICAL SCIENCES↗

DOE OSTI2023

Making Corgis Important for Honeycomb Classification: Adversarial Attacks on Concept-based Explainability Tools

Methods for model explainability have become increasingly critical for testing the fairness and soundness of deep learning. Concept-based interpretability techniques, which use a small set of human-interpretable concept exemplars in order to measure the influence of a concept on a model's internal representation of input, are an important thread in this line of research. In this work we show that these explainability methods can suffer the same vulnerability to adversarial attacks as the models they are meant to analyze. We demonstrate this phenomenon on two well-known concept-based interpretability methods: TCAV and faceted feature visualization. We show that by leveraging the geometry of the problem and carefully perturbing the examples of the concept that is being investigated, we can radically change the output of the interpretability method. The attacks that we propose can either induce positive interpretations (polka dots are an important concept for a model when classifying zebras) or negative interpretations (stripes are not an important factor in identifying images of a zebra). Our work highlights the fact that in safety-critical applications, there is need for security around not only the machine learning pipeline but also the model interpretation process.

Brown, Davis R.↗

DOE OSTI2022

Explainable Bayesian Neural Network for Probabilistic Transient Stability Analysis Considering Wind Energy

While several data-driven models have been developed for transient stability assessment, how to consider the uncertainties from load and renewable generations and provide interpretation of data-driven assessment results are still open. This paper proposes an explainable Bayesian Neural Network (BNN) for probabilistic transient stability assessment (TSA). By extracting the uncertainties from loads and wind farms, the BNN model can make a reliable prediction and quantify the prediction uncertainties. We also develop the Gradient Shap algorithm to make the global and local explanations for the probabilistic TSA model, a significant advantage over existing black-box data-driven methods. Numerical results on the modified IEEE 39-bus system show that the proposed method outperforms the existing methods in terms of prediction accuracy and uncertainty quantification capabilities. The explainability of the proposed method allows system operators to design preventive controls for enhancing system stability.

Bayesian Neural Network↗

DOE OSTI2025

Patchy burn severity explains heterogeneous soil viral and prokaryotic responses to fire in a mixed conifer forest

ABSTRACT Effects of fire on soil viruses and virus–host dynamics are largely unexplored, despite known microbial contributions to biogeochemical processes and ecosystem recovery. Here, we assessed how viral and prokaryotic communities responded to a prescribed burn in a mixed conifer forest. We sequenced 91 viral-size fraction metagenomes (viromes) and 115 16S rRNA gene amplicon libraries from 120 samples: four samples at five timepoints (two before fire and three after fire) at six sites (four treatment, two control). We hypothesized that compositional differences would be most significant between burned and unburned soils, but instead, plot location best distinguished viral communities, more than treatment (burned or not), depth (0–3 or 3–6 cm), or timepoint. For both viruses and prokaryotes, some burned communities resembled unburned controls, while others were significantly different, revealing heterogeneous responses to fire. These patterns were explained by burn severity, here defined by soil chemistry. Viral but not prokaryotic richness decreased significantly with burn severity, and low viromic DNA yields indicated substantial loss of viral biomass at higher severity. The relative abundances of Firmicutes, Actinobacteriota, and the viruses predicted to infect them increased significantly with burn severity, suggesting survival and viral infection of these fire-responsive and potentially spore-forming taxa. The degree of burn severity experienced by each patch of soil, rather than burn status alone, differed over mere meters in the same fire. Therefore, our analyses highlight the importance of high-resolution, paired biogeochemical data to explain soil community responses to fire. IMPORTANCE The impact of fire on the soil microbiome, particularly on understudied soil viral communities, warrants investigation, given known microbial contributions to biogeochemical processes and ecosystem recovery. Here, we collected 120 soil samples before and after a prescribed burn in a mixed conifer forest to assess the impacts of this disturbance on soil viral and prokaryotic communities. We show that simple categorical comparisons of burned and unburned areas were insufficient to reveal the underlying community response patterns. The patchy nature of the fire (indicated by soil chemistry data) led to significant changes in viral and prokaryotic community composition in areas of high burn severity, while communities that experienced lower burn severity were indistinguishable from those in unburned controls. Our results highlight the importance of considering highly resolved burn severity and biogeochemical measurements, even in nearby soils after the same fire, in order to understand soil microbial responses to prescribed burns.

Microbiology↗

DOE OSTI2021

Transformer eXplainability and eXploration

The Transformer eXplainability and eXploration library is intended to aid in the explorability and explainability of transformer classification networks, or transformer language models with sequence classification heads. The basic function of this library is to take a trained transformer and test/train dataset and produce an ipywidget dashboard which can be displayed in a jupyter notebook or in jupyter lab.

Martindale, Nathan [Oak Ridge National Lab. (ORNL)↗

DOE OSTI2024

An explainable variational autoencoder model for three-dimensional acoustic emission source localization in hollow cylindrical structures

We introduce an explainable variational autoencoder for three-dimensional (3D) localization of acoustic emission sources in hollow cylindrical structures, with an unsupervised approach. This research capitalizes on multi-arrival waveforms generated by helical path propagation in cylindrical geometries to enable efficient two-receiver localization. By integrating the modal characteristics of Lamb modes under multi-path conditions, we demonstrate that two sets of time-of-arrival differences and peak amplitudes extracted from one receiver can serve as effective localization features. This initial approach identifies four potential source locations, highlighting the feasibility of two-receiver source localization using traditional feature extraction methods. However, direct extraction can be challenging when mode overlaps occur, complicating the localization process. To address this, our work proposes a novel waveform-based method. This method leverages the consistent dispersion characteristics within isotropic materials, where each unique combination of mode arrival times and peak amplitudes constructs a distinct waveform. This distinctiveness overcomes the ambiguities associated with mode overlaps, significantly enhancing the method’s precision and robustness. Our approach adopts a data-driven strategy for waveform-based localization using variational autoencoder (VAE). VAE discerns waveform patterns for localization, while also addressing data uncertainties. The VAE’s encoder and decoder networks capture the localization process and the source’s influence on waveform generation, respectively, guiding latent variables to segregate waveforms by source in the latent space. The design of the learning process focuses on specific localization characteristics to enhance result explainability. Localization predictions are generated by projecting test waveforms, not included in the training set, onto a trained latent space. The prediction is determined using a nearest-neighbor approach based on the closest latent representation of a source. Validation with pencil-lead-break tests on a metallic pipe confirmed our method’s effectiveness, achieving an averaged 3D localization accuracy of 0.84.

Lee, Guan-Wei↗

DOE OSTI2021

TX$^2$: Transformer eXplainability and eXploration

The Transformer eXplainability and eXploration (Martindale & Stewart, 2021), or TX 2 software package, is a library designed for artificial intelligence researchers to better understand the performance of transformer models (Vaswani et al., 2017) used for sequence classification. The tool is capable of integrating with a trained transformer model and a dataset split into training and testing populations to produce an ipywidget (Project Jupyter Contributors, 2021) dashboard with a number of visualizations to understand model performance with an emphasis on explainability and interpretability. The TX 2 package is primarily intended to integrate into a workflow centered around Jupyter Notebooks (Kluyver et al., 2016), and currently assumes the use of PyTorch (Paszke et al., 2019) and Hugging Face transformers library (Wolf et al., 2020). The dashboard includes visualization and data exploration features to aid researchers, including an interactive UMAP embedding graph (McInnes et al., 2018) to understand classification clusters, a word salience map that can be updated as researchers alter textual entries in near real time, a set of tools to understand word frequency and importance based on the clusters in the UMAP embedding graph, and a set of traditional confusion matrix analysis tools.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2024

A semi-supervised learning method to produce explainable radioisotope proportion estimates for NaI-based synthetic and measured gamma spectra

Quantifying the radioactive sources present in gamma spectra is an ever-present and growing national security mission and a time-consuming process for human analysts. While machine learning models exist that are trained to estimate radioisotope proportions in gamma spectra, few address the eventual need to provide explanatory outputs beyond the estimation task. In this work, we develop two machine learning models for a NaI detector measurements: one to perform the estimation task, and the other to characterize the first model’s ability to provide reasonable estimates. To ensure the first model exhibits a behavior that can be characterized by the second model, the first model is trained using a custom, semi-supervised loss function which constrains proportion estimates to be explainable in terms of a spectral reconstruction. The second auxiliary model is an out-of-distribution detection function (a type of meta-model) leveraging the proportion estimates of the first model to identify when a spectrum is sufficiently unique from the training domain and thus is out-of-scope for the model. In demonstrating the efficacy of this approach, we encourage the use of meta-models to better explain ML outputs used in radiation detection and increase trust.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗

DOE OSTI2025

Explaining Nonmerger Gamma-Ray Bursts and Broad-lined Supernovae with Close Binary Progenitors with Black Hole Central Engines

For over 25 yr, the origin of long-duration gamma-ray bursts (lGRBs) has been linked to the collapse of rotating massive stars. However, we have yet to pinpoint the stellar progenitor powering these transients. Moreover, the dominant engine powering the explosions remains open to debate. Observations of both lGRBs, supernovae associated with these GRBs, such as broad-line (BL) stripped-envelope (type Ic) supernovae (hereafter, Ic-BL), supernovae (SNe), and perhaps superluminous SNe, fast blue optical transients, and fast x-ray transients, may provide clues to both engines and progenitors. In this paper, we conduct a detailed study of the tight-binary formation scenario for lGRBs, comparing this scenario to other leading progenitor models. Combining this progenitor scenario with different lGRB engines, we can compare to existing data and make predictions for future observational tests. We find that the combination of the tight-binary progenitor scenario with the black hole accretion disk engine can explain lGRBs, low-luminosity GRBs, ultra-long GRBs, and Ic-BL. We discuss the various progenitor properties required for these different subclasses and note such systems would be future gravitational-wave merger sources. We show that the current literature on other progenitor-engine scenarios cannot explain all of these transient classes with a single origin, motivating additional work. We find that the tight-binary progenitor with a magnetar engine is excluded by existing observations. The observations can be used to constrain the properties of stellar evolution, the nature of the GRB, and the associated SN engines in lGRBs and Ic-BL. We discuss the future observations needed to constrain our understanding of these rare, but powerful, explosions.

79 ASTRONOMY AND ASTROPHYSICS↗

DOE OSTI2020

Hysteretic temperature sensitivity of wetland CH4 fluxes explained by substrate availability and microbial activity: Model Archive

This Modeling Archive is in support of an NGEE Arctic publication "Hysteretic temperature sensitivity of wetland CH4 fluxes explained by substrate availability and microbial activity" in the Journal Biogeosciences (https://doi.org/10.5194/bg-17-5849-2020), which includes the model data used in the publication. CH4 emissions from terrestrial systems are posited to increase, which can offset mitigation efforts and accelerate climate change. Yet, the accuracy of modeled CH4 emissions is sensitive to the prescribed CH4 production (or emission) temperature dependencies that are currently uncertain. Here, we use a comprehensive biogeochemistry model (ecosys) to investigate factors modulating CH4 production and emission rates across a permafrost thaw gradient encompassing a partly thawed bog and a fully thawed fen. We find that seasonally varying substrate availability drives lower and higher modeled methanogen biomass and activity, and thereby CH4 production, during the earlier and later periods of the thawed season, respectively. Package follows the Model data archiving guidelines with data in a *.zip file with three subfolders containing *.csv files and raw model output files; a data dictionary table (data_dictionary.csv) and two model output description tables (ecosys_plantspecies_output_notes.csv and ecosys_soil_ouput_notes.csv) to explain the format and meaning of individual output variables; and a user guide as a *.pdf. A detailed model description can be found in the supplement of (Grant, 2013). The ecosys source code is available at https://doi:10.5281/zenodo.3906642. The Next-Generation Ecosystem Experiments: Arctic (NGEE Arctic), was a research effort to reduce uncertainty in Earth System Models by developing a predictive understanding of carbon-rich Arctic ecosystems and feedbacks to climate. NGEE Arctic was supported by the Department of Energy's Office of Biological and Environmental Research. The NGEE Arctic project had two field research sites: 1) located within the Arctic polygonal tundra coastal region on the Barrow Environmental Observatory (BEO) and the North Slope near Utqiagvik (Barrow), Alaska and 2) multiple areas on the discontinuous permafrost region of the Seward Peninsula north of Nome, Alaska. Through observations, experiments, and synthesis with existing datasets, NGEE Arctic provided an enhanced knowledge base for multi-scale modeling and contributed to improved process representation at global pan-Arctic scales within the Department of Energy's Earth system Model (the Energy Exascale Earth System Model, or E3SM), and specifically within the E3SM Land Model component (ELM).

54 ENVIRONMENTAL SCIENCES↗

DOE OSTI2023

Explainable AI (XAI)-driven vibration sensing scheme for surface quality monitoring in a smart surface grinding process

Local Interpretable and Model-agnostic Explanation (LIME), an explainable artificial intelligence (XAI) approach is adapted to identify the globally important time-frequency bands for predicting average surface roughness (Ra) in a smart grinding process. The smart grinding setup consisted of a Supertech CNC precision surface grinding machine, instrumented with a Dytran piezoelectric accelerometer attached to the tailstock quill along the tangential direction (Y-axis). For every grinding pass, vibration signatures were captured, and the ground truth surface roughness values were recorded using a Mahr Marsurf M300C portable surface roughness profilometer. The roughness values ranged from 0.06 to 0.14 microns over the complete set of experiments. Time-frequency domain spectrogram frames were extracted for each of the vibration signals collected during the grinding process. Convolutional Neural Networks (CNNs) were modeled to predict the surface roughness based on these spectrogram frames and their image augmentations. The best CNN model was able to predict the roughness values with an overall R2-score of 0.95, training R2-score of 0.99, and testing R2-score of 0.81 with only 80 sets of vibration signals corresponding to 4 experiments with 20 trials each. Although the data size is not large enough to guarantee such performance metrics in real-world scenarios, one can extract statistically consistent explanations underlying the relationships these complex deep learning models capture. Further, the LIME methodology was implemented on the developed surface roughness CNN model to identify the important time-frequency bands (i.e., the superpixels of a spectrogram) influencing the predictions. Based on the identified important regions on the spectrogram frames, the corresponding frequency characteristics were determined that influence the surface roughness predictions. The important frequency range based on LIME results was approximately 11.7 to 19.1 kHz. The power of XAI was demonstrated by cutting down the sampling rate from 160 kHz to 30, 20, 10, and 5 kHz based on the important frequency range and considering Nyquist criteria. Separate CNN models were developed for these ranges by only extracting time-frequency contents below their corresponding Nyquist cut-offs. A proper data acquisition strategy is proposed by comparing the model performances to argue the selection of a sufficient sampling rate to capture the grinding process successfully and robustly.

42 ENGINEERING↗

DOE OSTI2023

Reprint of: Explainable AI (XAI)-driven vibration sensing scheme for surface quality monitoring in a smart surface grinding process

47 OTHER INSTRUMENTATION↗