Results for “Explainable machine learning” — Page 7

DOE OSTI2025

XSub: Explanation-Driven Adversarial Attack against Blackbox Classifiers via Feature Substitution

Despite its significant benefits in enhancing the transparency and trustworthiness of artificial intelligence (AI) systems, explainable AI (XAI) can unintentionally provide adversaries with insights into blackbox models, increasing their vulnerability to various attacks. In this paper, we develop a novel explanation-driven adversarial attack against blackbox classifiers based on feature substitution, called XSub. The key idea of XSub is to strategically replace important features (identified via XAI) in the original sample with corresponding important features of a different label, thereby increasing the likelihood of the model misclassifying the perturbed sample. XSub only requires a minimal number of queries and can be easily extended to launch backdoor attacks in case the attacker has access to the model's training data. Our evaluation shows that XSub is not only effective and stealthy but also low-cost, showcasing its feasibility across a wide range of AI applications.

adversarial attack↗

DOE OSTI2025

Physical Interpretation of Early Battery Life Prediction Models

Early battery life prediction models are most useful for R&D if they help us understand the early changes in battery electrochemical response that correspond with long-term degradation and failure. Linear regression models such as Fused lasso and Partial Least Squares can fit coefficients directly to high-dimensional electrochemical data like capacity-voltage and ΔV–state-of-charge, i.e., Q(V) and ΔV(SOC) curves, learning coefficients that can be physically interpreted. We leverage the ISU-ILCC battery aging data set to learn high-dimensional coefficients for early battery life prediction from traditional slow-rate capacity check data, demonstrating learning on Q(V), d Q· d V −1 , and ΔV(SOC) curves. A thorough study on the dependence of coefficient values on train/test size and data preprocessing methods is made, demonstrating the reliability of high-dimensional regression approaches unless very small amounts of data are used for model training. For this data set, coefficients from Q(V) and d Q· d V −1 models highlight changes in electrode stoichiometry due to lithium loss, while ΔV(SOC) coefficients highlight changes in positive electrode diffusivity due to particle cracking as well as electrode stoichiometry shifts. By directly interpreting the coefficients of a regression model, we make physical insights into battery degradation mechanisms without requiring the assumptions of traditional battery data analysis methods.

25 ENERGY STORAGE↗

DOE OSTI2023

Performance Comparison of Object Detection Networks for Shrapnel Identification in Ultrasound Images

Ultrasound imaging is a critical tool for triaging and diagnosing subjects but only if images can be properly interpreted. Unfortunately, in remote or military medicine situations, the expertise to interpret images can be lacking. Machine-learning image interpretation models that are explainable to the end user and deployable in real time with ultrasound equipment have the potential to solve this problem. We have previously shown how a YOLOv3 (You Only Look Once) object detection algorithm can be used for tracking shrapnel, artery, vein, and nerve fiber bundle features in a tissue phantom. However, real-time implementation of an object detection model requires optimizing model inference time. Here, we compare the performance of five different object detection deep-learning models with varying architectures and trainable parameters to determine which model is most suitable for this shrapnel-tracking ultrasound image application. We used a dataset of more than 16,000 ultrasound images from gelatin tissue phantoms containing artery, vein, nerve fiber, and shrapnel features for training and evaluating each model. Every object detection model surpassed 0.85 mean average precision except for the detection transformer model. Overall, the YOLOv7tiny model had the higher mean average precision and quickest inference time, making it the obvious model choice for this ultrasound imaging application. Other object detection models were overfitting the data as was determined by lower testing performance compared with higher training performance. In summary, the YOLOv7tiny object detection model had the best mean average precision and inference time and was selected as optimal for this application. Next steps will implement this object detection algorithm for real-time applications, an important next step in translating AI models for emergency and military medicine.

60 APPLIED LIFE SCIENCES↗

DOE OSTI2021

A systematic comparison of machine learning methods for modeling of dynamic processes applied to combustion emission rate modeling

Ten established, data-driven dynamic algorithms are surveyed and a practical guide for understanding these methods generated. Existing Python programming packages for implementing each algorithm are acknowledged, and the model equations necessary for prediction are presented. A case study on a coal-fired power plant’s NO x emission rates is performed, directly comparing each modeling method’s performance on a mutual system. Each model is evaluated by its root mean squared error (RMSE) on out-of-sample future horizon predictions. Optimal hyperparameters are identified using either an exhaustive search or genetic algorithm. The top five model structures of each method are used to recursively predict future NO x emission rates over a 60-step time horizon. The RMSE at each future timestep is determined, and the recursive output prediction trends compared against measurements in time. The GRU neural network is identified as the best candidate for representing the system, demonstrating accurate and stable predictions across the future horizon by all considered models, while satisfactory performance was observed in several of the ARX/NARX formulations. Finally, these efforts have contributed 1) a concise resource of multiple proven dynamic machine learning methods, 2) a practical guide explaining the use of these methods, effectively lowering the “barrier-to-entry” of deploying such models in control systems, 3) a comparison study evaluating each method’s performance on a mutual system, 4) demonstration of accurate multi-timestep emissions modeling suitable for systems-level control, and 5) generalizable results demonstrating the suitability of each method for prediction over a multi-step future horizon to other complex dynamic systems.

42 ENGINEERING↗

DOE OSTI2026

Continental-Scale Controls on Hyporheic Respiration Revealed by Knowledge-Guided Machine Learning

Hyporheic zone sediments regulate organic matter turnover and in-stream respiration, yet controls on sediment respiration remain poorly constrained across heterogeneous river networks, limiting prediction of stream metabolism and carbon processing at continental scales. Here, we integrate observations from ~90 river corridors across the United States in the WHONDRS consortium with a knowledge-guided machine learning (KGML) framework that couples thermodynamic rate theory with machine learning to identify dominant controls on hyporheic respiration. Diagnostic analyses show that organic matter concentration and thermodynamic favorability define an upper bound on respiration potential, whereas biological catalytic capacity and physical accessibility jointly govern realized respiration rates through interaction effects. To represent unmeasurable accessibility constraints, we use the mechanistic model as a scaffold for KGML, allowing machine learning to target residual structure not explained by process theory. This hybrid framework improves predictive skill relative to both the mechanistic model alone and fully data-driven models while preserving interpretability. These results indicate that variability in hyporheic respiration is largely mechanistically structured and demonstrate how integrating process theory with explainable AI enhances predictive performance while enabling scalable synthesis of river corridor observations.

Zheng, Jianqiu↗

DOE OSTI2022

Automated Infrastructure & Dependency Detection via Satellite Imagery and Dependency Profiles Project Summary & Profession Background

This presentation was presented to N&HS SAC community on July 21 2022.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2026

Estimators and Fusers for Fiber Delay Estimation Using Environmental Measurements

The properties of deployed network fiber are affected by environmental factors due to their exposure to the elements. Particularly for quantum networks, the resultant delay variations may have significant impacts due to the extreme sensitivity of synchronization, coincidence counting, and other critical operations. In this paper, the delays of 15 km aerial-inground fiber connections are measured, and effects due to temperature, humidity and wind speed are analyzed over multiple periods spanning four seasons of a year. Machine learning methods are first utilized to reveal surprisingly pronounced effects of humidity on the delay, in addition to the expected temperature and its seasonal variations. Estimator and fusion methods are developed to estimate the delay using temperature, humidity and wind speed measurements, by utilizing smooth Gaussian Process Regression (GPR) and nonsmooth Ensemble of Trees (EOT) methods. Measurements from winter and summer periods are temporally fused using twelve different methods, and eight methods provide estimates for the delay throughout the year with median test errors under 1.28%. The results reveal distinct temperature-humidity trends across the seasons, and the ability of estimator and temporal fusion methods to exploit them for estimating the delay. These results constitute a case study of machine learning analytical results, wherein generalization equations explain the performance of various estimator and fuser methods.

Rao, Nageswara [ORNL] (ORCID:0000000234085941)↗

DOE OSTI2025

Explainable Synthesizability Prediction of Inorganic Crystal Polymorphs Using Large Language Models

Abstract We evaluate the ability of machine learning to predict whether a hypothetical crystal structure can be synthesized and explain those predictions to scientists. Fine‐tuned large language models (LLMs) trained on a human‐readable text description of the target crystal structure perform comparably to previous bespoke convolutional graph neural network methods, but better prediction quality can be achieved by training a positive‐unlabeled learning model on a text‐embedding representation of the structure. An LLM‐based workflow can then be used to generate human‐readable explanations for the types of factors governing synthesizability, extract the underlying physical rules, and assess the veracity of those rules. These explanations can guide chemists in modifying or optimizing non‐synthesizable hypothetical structures to make them more feasible for materials design.

Kim, Seongmin [Department of Chemical and Biologic↗

DOE OSTI2025

Explainable Synthesizability Prediction of Inorganic Crystal Polymorphs Using Large Language Models

Abstract We evaluate the ability of machine learning to predict whether a hypothetical crystal structure can be synthesized and explain those predictions to scientists. Fine‐tuned large language models (LLMs) trained on a human‐readable text description of the target crystal structure perform comparably to previous bespoke convolutional graph neural network methods, but better prediction quality can be achieved by training a positive‐unlabeled learning model on a text‐embedding representation of the structure. An LLM‐based workflow can then be used to generate human‐readable explanations for the types of factors governing synthesizability, extract the underlying physical rules, and assess the veracity of those rules. These explanations can guide chemists in modifying or optimizing non‐synthesizable hypothetical structures to make them more feasible for materials design.

Kim, Seongmin [Department of Chemical and Biologic↗

DOE OSTI2023

Machine learning model explanation apparatus and methods

Explanation apparatus and methods are described. In one aspect, an explanation apparatus includes processing circuity configured to access a source instance which has been classified by a machine learning model; create associations of the source instance with a plurality of training instances; and process the associations of the source instance and the training instances to identify a first subset of the training instances which have less relevance to the classification decision of the source instance by the machine learning model compared with a second subset of the training instances; and an interface configured to communicate information to a user, and wherein the processing circuitry is configured to control the user interface to communicate the second subset of the training instances to the user as evidence to explain the classification of the source instance by the machine learning model.

Arendt, Dustin↗

DOE OSTI2026

Toward a microscopic picture of hadronization and multi-parton processes

This project advanced the understanding of how quarks and gluons produced in high-energy collisions transform into the hadrons observed in particle detectors, a fundamental process known as quantum chromodynamics (QCD) hadronization. By combining theoretical calculations, quantum simulation methods, and modern AI techniques, the research developed new tools to study multi-parton dynamics and nonperturbative effects that are essential for interpreting data from current and future nuclear physics experiments. Key outcomes include new theoretical frameworks for jet and hadron measurements, pioneering quantum simulation algorithms for real-time dynamics in field theories, and the development of advanced machine-learning models, such as diffusion models and explainable classifiers, to simulate and analyze collider events. These results are directly relevant to experiments at Jefferson Lab, Brookhaven National Laboratory, and the future Electron-Ion Collider, and they also have a broader impact in areas such as quantum information science and data-driven modeling of complex systems. The project supported the training of graduate students and postdoctoral fellows and contributed to the broader scientific community through publications, workshops, and collaborative activities. Overall, this work provides new insights into the microscopic mechanisms of hadron formation and establishes a foundation for future studies at the intersection of nuclear physics, artificial intelligence, and quantum computing.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗

DOE OSTI2023

Explainable AI (XAI)-driven vibration sensing scheme for surface quality monitoring in a smart surface grinding process

Local Interpretable and Model-agnostic Explanation (LIME), an explainable artificial intelligence (XAI) approach is adapted to identify the globally important time-frequency bands for predicting average surface roughness (Ra) in a smart grinding process. The smart grinding setup consisted of a Supertech CNC precision surface grinding machine, instrumented with a Dytran piezoelectric accelerometer attached to the tailstock quill along the tangential direction (Y-axis). For every grinding pass, vibration signatures were captured, and the ground truth surface roughness values were recorded using a Mahr Marsurf M300C portable surface roughness profilometer. The roughness values ranged from 0.06 to 0.14 microns over the complete set of experiments. Time-frequency domain spectrogram frames were extracted for each of the vibration signals collected during the grinding process. Convolutional Neural Networks (CNNs) were modeled to predict the surface roughness based on these spectrogram frames and their image augmentations. The best CNN model was able to predict the roughness values with an overall R2-score of 0.95, training R2-score of 0.99, and testing R2-score of 0.81 with only 80 sets of vibration signals corresponding to 4 experiments with 20 trials each. Although the data size is not large enough to guarantee such performance metrics in real-world scenarios, one can extract statistically consistent explanations underlying the relationships these complex deep learning models capture. Further, the LIME methodology was implemented on the developed surface roughness CNN model to identify the important time-frequency bands (i.e., the superpixels of a spectrogram) influencing the predictions. Based on the identified important regions on the spectrogram frames, the corresponding frequency characteristics were determined that influence the surface roughness predictions. The important frequency range based on LIME results was approximately 11.7 to 19.1 kHz. The power of XAI was demonstrated by cutting down the sampling rate from 160 kHz to 30, 20, 10, and 5 kHz based on the important frequency range and considering Nyquist criteria. Separate CNN models were developed for these ranges by only extracting time-frequency contents below their corresponding Nyquist cut-offs. A proper data acquisition strategy is proposed by comparing the model performances to argue the selection of a sufficient sampling rate to capture the grinding process successfully and robustly.

42 ENGINEERING↗

DOE OSTI2023

Reprint of: Explainable AI (XAI)-driven vibration sensing scheme for surface quality monitoring in a smart surface grinding process

Local Interpretable and Model-agnostic Explanation (LIME), an explainable artificial intelligence (XAI) approach is adapted to identify the globally important time-frequency bands for predicting average surface roughness (Ra) in a smart grinding process. The smart grinding setup consisted of a Supertech CNC precision surface grinding machine, instrumented with a Dytran piezoelectric accelerometer attached to the tailstock quill along the tangential direction (Y-axis). For every grinding pass, vibration signatures were captured, and the ground truth surface roughness values were recorded using a Mahr Marsurf M300C portable surface roughness profilometer. The roughness values ranged from 0.06 to 0.14 microns over the complete set of experiments. Time-frequency domain spectrogram frames were extracted for each of the vibration signals collected during the grinding process. Convolutional Neural Networks (CNNs) were modeled to predict the surface roughness based on these spectrogram frames and their image augmentations. The best CNN model was able to predict the roughness values with an overall R2-score of 0.95, training R2-score of 0.99, and testing R2-score of 0.81 with only 80 sets of vibration signals corresponding to 4 experiments with 20 trials each. Although the data size is not large enough to guarantee such performance metrics in real-world scenarios, one can extract statistically consistent explanations underlying the relationships these complex deep learning models capture. Further, the LIME methodology was implemented on the developed surface roughness CNN model to identify the important time-frequency bands (i.e., the superpixels of a spectrogram) influencing the predictions. Based on the identified important regions on the spectrogram frames, the corresponding frequency characteristics were determined that influence the surface roughness predictions. The important frequency range based on LIME results was approximately 11.7 to 19.1 kHz. The power of XAI was demonstrated by cutting down the sampling rate from 160 kHz to 30, 20, 10, and 5 kHz based on the important frequency range and considering Nyquist criteria. Separate CNN models were developed for these ranges by only extracting time-frequency contents below their corresponding Nyquist cut-offs. A proper data acquisition strategy is proposed by comparing the model performances to argue the selection of a sufficient sampling rate to capture the grinding process successfully and robustly.

47 OTHER INSTRUMENTATION↗

NASA NTRSDate not supplied

Multi-Class Anomaly Detection in Flight Data using Semi-Supervised Explainable Deep Learning Model

Identifying precursor for safety incidents in aviation data is a crucial task, yet extremely challenging. The main approach, in practice, leverages domain expertise to define expected tolerances in system’s behavior and alarm exceedance from such safety margins. However, this approach is incapable of identifying unknown risk and vulnerabilities. Machine learning has been long studied and deployed to identify precursors for such anomalies, with the great challenge of the need for sufficient labelled set of data to achieve a reliable and accurate performance. In this article, we develop an explainable deep semi-supervised model for anomaly detection in aviation, building upon recent advancements in the machine learning literature. The proposed model combines feature engineering and classification in the feature space, while leveraging all available data (labelled and unlabeled). Validating on two case studies of anomaly detection in take-off and landing phases of commercial aircraft, we show that our model is able to outperform state-of-the-art supervised anomaly detection model and reach significantly high accuracy and low false alarm with minimum amount of available labelled data.

Anomaly Detection↗

DOE OSTI2024

Machine learning-based discovery of molecular descriptors that control polymer gas permeation

While machine learning has found increasing use in predicting the properties of polymeric materials with only a knowledge of chain architecture, determining the molecular factors underpinning properties (“interpretable AI”) has remained less well explored. We show that encoding chain chemistry in commonly employed formats, e.g., binary-valued fingerprints, leads to uniqueness issues during the hashing process to save storage space. This is because the hashing algorithm can map several chemical moieties into the same bit. These issues carry over into the ML algorithms, especially for “inverse” design and interpretable AI, and cannot be avoided by changing the length of the fingerprint. Using MACCS key featurizations of monomer repeats resolves some of these issues, and we show that a few substructures consistently appear in top features for maximizing permeability across several gases and ML models. These are carbon–carbon double bonds (as in polyacetylenes) especially when they are associated with methyl groups (found in branching architectures). Here these results, derived from the limited data set of ~ 500 polymers with experimental gas permeation data, are in agreement with physical insight and thus provide a robust foundation which could further enable study of these material classes through detailed experiments and simulations.

36 MATERIALS SCIENCE↗

DOE OSTI2023

Quantum Neural Networks: Issues, Training, and Applications

Our work in the field aims at explaining the limitations and expressive power of Quantum Machine Learning models, as well as finding feasible training algorithms that could be implemented in near-term Quantum Computers. The promise of Quantum Machine Learning is that by incorporating quantum effects, such as entanglement, into machine learning models researchers can improve model performance and understand more complex datasets. This pledge is particularly pronounced in the design of Quantum neural networks (QNNs), a promising framework for creating quantum algorithms, that promise to outperform classical models by combining the speedups of quantum computation with the widespread successes of deep learning. We show that applying this approach alone to quantum deep learning is problematic given that an excess of entanglement between the hidden and visible layers can destroy the predictive power of our QNN models. We address the barren plateau problem by suggesting the use of a generative, unbounded, nonlinear loss function with simple gradients. The loss function quantifies how much the quantum states generated by the QNNs differ from the data and the goal during training is to minimize it. Finally, we showcase how to use generative training to construct a "classical-quantum" neural network to accurately interpolate between the ground states of a Molecular Hamiltonian, a central question in Quantum Chemistry.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2023

Evasion Attacks

Evasion Attacks are cases when an adversary is trying to hide something, or fool, a machine learning system. See reference here: Goodfellow, Shlens, Szegedy, Explaining and Harnessing Adversarial Examples. You can think of evasion attack as similar the little boy from the story, “A Wolf in Sheep’s Clothing”. The little boy appears as one thing (a wolf), but is actually just a little boy. Below are a few more examples that have appeared in literature and the media.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2022

Edaphic controls on genome size and GC content of bacteria in soil microbial communities

Nutrient limitation has been shown to reduce bacterial genome size and influence nucleotide composition; however, much of this work has been conducted in marine systems and the factors which shape soil bacterial genomic traits remain largely unknown. Here, for this work, we determined average genome size, GC content, codon usage, and amino acid content from 398 soil metagenomes across a broad geographic range and used machine-learning to determine the environmental parameters that most strongly explain the distribution of these traits. We found that genomic trait averages were most related to pH, which we suggest is primarily due to the correlation of pH with several environmental parameters, particularly soil carbon content. Low pH soils had higher carbon to nitrogen ratios (C:N) and tended to have communities with lower GC content and larger genomes, potentially a response to increased physiological stress and a requirement for metabolic diversity. Conversely, communities in high pH and low soil C:N had smaller genomes and higher GC content—indicating potential resource driven selection against AT base pairs, which have a higher C:N than GC base pairs. Similarly, we found that nutrient conservation also applied to amino acid stoichiometry, where bacteria in soils with low C:N ratios tended to code for amino acids with lower C:N. Together, these relationships point towards fundamental mechanisms that underpin genome size, and nucleotide and amino acid selection in soil bacteria.

54 ENVIRONMENTAL SCIENCES↗