Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “interpretable machine learning”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 163 records · Page 9
All sourcesNASA NTRSDOE OSTI
DOE OSTI2026

Generation of random geological models using multi-randomization for machine learning

Generating high-fidelity geological models is essential for advancing machine learning (ML) methods in automated seismic interpretation. For instance, seismic images paired with corresponding fault labels are foundational for ML-based fault detection from seismic migration sections. While several open-access datasets of random geological models exist, open-source tools specifically designed to produce large volumes of such models for ML applications remain scarce. To address this gap, we present RGM (Random Geological Model), an open-source software package for efficiently generating 2D and 3D synthetic geological models tailored for ML workflows. RGM supports the creation of diverse model components, including medium property distributions (P-/S-wave velocities and density), seismic reflectivity images (i.e., synthetic migration sections), relative geological time, and discrete fault attributes such as probability, dip, strike, rake, and displacement. It also accommodates the creation of complex geological features such as salt bodies and unconformities. The model generation algorithm employs a multi-randomization strategy, yielding an effectively infinite-dimensional model space that encompasses a wide range of geological scenarios and associated seismic features. Furthermore, RGM incorporates a method to generate synthetic elastic migration images using analytical elastic reflection coefficients combined with frequency-dependent scaling. This functionality enables the creation of training datasets for ML models that leverage elastic seismic images. RGM is implemented in modern object-oriented Fortran, allowing users to flexibly control statistical parameters governing model variability. We demonstrate the capability, performance, and geological realism of the package through comprehensive 2D and 3D examples.

58 GEOSCIENCES↗
DOE OSTI2025

Machine learning inversion from small-angle scattering for charged polymers

We develop Monte Carlo simulations for uniformly charged polymers and a machine learning algorithm to interpret the intra-polymer structure factor of the charged polymer system, which can be obtained from small-angle scattering experiments. The polymer is modeled as a chain of fixed-length bonds, where the connected bonds are subject to bending energy, and there is also a screened Coulomb potential for charge interaction between all joints. The bending energy is determined by the intrinsic bending stiffness, and the charge interaction depends on the interaction strength and screening length. All three contribute to the stiffness of the polymer chain and lead to longer and larger polymer conformations. The screening length also introduces a second length scale for the polymer besides the bending persistence length. To obtain the inverse mapping from the structure factor to these polymer conformation and energy-related parameters, we generate a large data set of structure factors by running simulations for a wide range of polymer energy parameters. We use principal component analysis to investigate the intra-polymer structure factors and determine the feasibility of the inversion using the nearest neighbor distance. We employ Gaussian process regression to achieve the inverse mapping and extract the characteristic parameters of polymers from the structure factor with low relative error.

36 MATERIALS SCIENCE↗
DOE OSTI2021

Machine learning to extend and understand the sources and limits of water cycle predictability on subseasonal-to-decadal timescales in the Earth system

This white paper provides initial insight on how artificial intelligence (AI) and machine learning (ML), including interpretability and explainable AI (XAI) methods, can be leveraged to glean insight from complex data for a paradigm-changing improvement in Earth system predictability on subseasonal-to-seasonal (S2S) and seasonal-to-decadal (S2D) timescales. The application of AI to extend and improve predictability, in combination with causal inference and uncertainty quantification, could lead to a transformative understanding of the integrative water cycle and associated extremes.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2024

Time Dilated Bundt Cake Analysis of PV Output [Poster]

We present a novel method for modeling time-dependent statistics in the power signal generated by a photovoltaic (PV) system. Our white-box machine learning method is interpretable and auditable, based on principles of multiperiodic basis functions and convex optimization. Our proposed method of time dilating the daily signal to remove night time values results in a novel representation of PV power signals, evocative of a ‘Bundt cake’. The proposed model describes the marginal distribution of power output as a function of date and time. The resulting probabilistic model of a PV system can be used to perform a variety of tasks, and here, we demonstrate the application of clear sky detection.

14 SOLAR ENERGY↗
DOE OSTI2020

On-the-fly active learning of interpretable Bayesian force fields for atomistic rare events

Abstract Machine learned force fields typically require manual construction of training sets consisting of thousands of first principles calculations, which can result in low training efficiency and unpredictable errors when applied to structures not represented in the training set of the model. This severely limits the practical application of these models in systems with dynamics governed by important rare events, such as chemical reactions and diffusion. We present an adaptive Bayesian inference method for automating the training of interpretable, low-dimensional, and multi-element interatomic force fields using structures drawn on the fly from molecular dynamics simulations. Within an active learning framework, the internal uncertainty of a Gaussian process regression model is used to decide whether to accept the model prediction or to perform a first principles calculation to augment the training set of the model. The method is applied to a range of single- and multi-element systems and shown to achieve a favorable balance of accuracy and computational efficiency, while requiring a minimal amount of ab initio training data. We provide a fully open-source implementation of our method, as well as a procedure to map trained models to computationally efficient tabulated force fields.

36 MATERIALS SCIENCE↗
DOE OSTI2019

Quantifying Climate and Catchment Control on Hydrological Drought in the Continental United States

The evolution of hydrological drought events is a result of complex (nonlinear) interactions between climate and catchment processes. To investigate such nonlinear relationship, we integrated a machine learning modeling framework based on the random forest (RF) algorithms with an interpretation framework to quantify the role of climate and catchment controls on hydrological drought. More particularly, our framework interprets a built RF machine-learning model to identify dominant variables and visualize their functional dependence and interaction effects on hydrological drought characteristics utilizing concepts of minimal depth, interactive depth, and partial dependence. We test our proposed modeling framework based on a set of 652 continental United States catchments with minimal human interference for a period of 1979–2010. Application of this framework indicated presence of three distinct drought regimes, which includes, Regime 1: droughts with longer duration, less frequent and lesser intensity; Regime 2: droughts with moderate duration, moderate frequency, and moderate intensity; and Regime 3: droughts with shorter duration, more frequent, and more intense. RF algorithm was able to accurately model the drought characteristics (intensity, duration, and number of events) for all the three drought regimes as a function of selected variables. It was observed that the type of dominant variables as well as their nonlinear functional relationship with hydrological droughts characteristics can vary between three selected regimes. Finally, our interpretation framework indicated that catchment characteristics have a significant role in controlling the hydrologic drought for catchments (regime 1), whereas both climate and catchment characteristics control hydrological drought in regimes 2 and 3.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2023

A detailed study of interpretability of deep neural network based top taggers

Abstract Recent developments in the methods of explainable artificial intelligence (XAI) allow researchers to explore the inner workings of deep neural networks (DNNs), revealing crucial information about input–output relationships and realizing how data connects with machine learning models. In this paper we explore interpretability of DNN models designed to identify jets coming from top quark decay in high energy proton–proton collisions at the Large Hadron Collider. We review a subset of existing top tagger models and explore different quantitative methods to identify which features play the most important roles in identifying the top jets. We also investigate how and why feature importance varies across different XAI metrics, how correlations among features impact their explainability, and how latent space representations encode information as well as correlate with physically meaningful quantities. Our studies uncover some major pitfalls of existing XAI methods and illustrate how they can be overcome to obtain consistent and meaningful interpretation of these models. We additionally illustrate the activity of hidden layers as neural activation pattern diagrams and demonstrate how they can be used to understand how DNNs relay information across the layers and how this understanding can help to make such models significantly simpler by allowing effective model reoptimization and hyperparameter tuning. These studies not only facilitate a methodological approach to interpreting models but also unveil new insights about what these models learn. Incorporating these observations into augmented model design, we propose the particle flow interaction network model and demonstrate how interpretability-inspired model augmentation can improve top tagging performance.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2026

Data from: "Towards CONUS-Wide ML-Augmented Conceptually-Interpretable Modeling of Catchment-Scale Precipitation-Storage-Runoff Dynamics"

This data package was generated to support the manuscript “Towards CONUS-Wide Machine Learning-Augmented Conceptually Interpretable Modeling of Catchment-Scale Precipitation-Storage-Runoff Dynamics.” It provides input files, model outputs, plotting data, scripts, notebooks, and documentation used to develop, evaluate, and reproduce Mass-Conserving Perceptron (MCP)-based hydrologic modeling experiments across 513 selected Catchment Attributes and Meteorology for Large-sample Studies in the United States (CAMELS-US) basins. The files are organized by modeling component and analysis purpose, including rainfall–runoff experiments, snow module experiments, coupled hydrologic-snow experiments, Long Short-Term Memory (LSTM) benchmark results, model skill metrics, initialization and epoch records, cell-state normalization files, Akaike Information Criterion (AIC)-based model comparison files, and data used to generate manuscript figures. Tabular files can be opened using standard spreadsheet software or Python/R data-analysis tools. Python scripts, Jupyter notebooks, and selected MATLAB scripts are included for model execution, postprocessing, plotting, and statistical analysis. Quality assurance and quality control were conducted through the source-data selection and modeling workflow. Meteorological forcing, streamflow, and static catchment attributes were derived from the CAMELS-US dataset, and snow water equivalent data were derived from the University of Arizona (UA) Snow Water Equivalent dataset. Selected basins and time periods were screened during the associated research workflow to avoid missing observations or poor-quality cases. Static geospatial features were processed primarily using Quantum Geographic Information System (QGIS) and Geospatial Data Abstraction Library (GDAL) workflows. Additional details are provided in the associated manuscript and documentation.

ESS-DIVE CSV File Formatting Guidelines Reporting ↗
DOE OSTI2021

Machine learning assisted modeling of mixing timescale for LES/PDF of high-Karlovitz turbulent premixed combustion

Accurate modeling of mixing in the transported probability density function (PDF) method remains a great challenge, especially for turbulent premixed combustion under extreme conditions such as high Karlovitz number Ka. Recently, a power-law based mixing timescale model was developed for the large-eddy simulations (LES)/PDF modeling of high-Ka number turbulent premixed flames. It is found in this work that the power-law mixing timescale model is highly sensitive to the model parameters. It is thus critically needed to develop accurate calibration of these model parameters. The empirical specification of the model parameters developed in Zhang et. al. is found to be inadequate for accurate modeling of the mixing timescale. Here, machine learning is introduced as an attractive alternative in this work for the specification of the model parameters. A high-Ka number DNS jet flame is used as the training and validation of the machine learning models. The choices of the input parameters are discussed and compared for the machine learning models. The effect of differential molecular diffusion on mixing is examined by including the effect of the Lewis number in the training of the machine learning models. The performance of different machine learning algorithms is compared for the specification of the mixing model parameters. Overall, excellent performance of the machine learning models is observed for assisting the mixing modeling. The feasibility, interpretability, applicability, generality, and portability of using machine learning are discussed in general to provide a perspective on applying data-driven machine learning for turbulent combustion modeling studies.

42 ENGINEERING↗
DOE OSTI2026

Multi‐Scale Model‐Informed Deep Learning for Plasma‐Nanoparticle Interaction

The Overarching Goal of this proposed research is to understand and quantitively determine the interactions between non-thermal plasma (hot electrons, reactive radicals, vibrationally excited species) and surface reactions on influencing the activity and selectivity of the desired reactions via developing multi-scale model informed deep learning algorithm. Investigating non-thermal plasma-surface interaction is feasible due to the low bulk temperature in the discharge region. To investigate the role of plasma-nanoparticle interaction on enhancing the reaction kinetics, we will focus on ammonia cracking to generate clean hydrogen over earth-abundant, non-critical metallic nanoparticles, which is of great significance for decarbonization. We hypothesize that (1) reactive radicals interacting with surface reaction species via Eley–Rideal mechanism will significantly lower the energetics of the potential rate-limiting step of nitrogen formation; (2) the surface will be charged heterogeneously under non-thermal plasma conditions and the charged site will lower the energetics of ammonia cracking through Langmuir– Hinshelwood mechanism; (3) vibrationally excited ammonia will further promote the initial N-H bond cleavage. To access the hypothesis, we will (1) reveal the surface charge effects on tunning the reaction energetics via interpretable, physics-informed deep learning accelerated density functional theory (DFT) calculations; (2) determine the reactive radicals interacting with surface reaction species on tuning the reaction energetics via DFT; (3) reveal the surface charge effects on tunning the reaction energetics via DFT and deep learning models, (4) quantify how vibrationally excited species, reactive radicals, and surface charging effects on enhancing the catalysis via developing DFT-based microkinetic modeling (MKM) and active learning. Deep and active learning of plasma-nanoparticle interactions effects on enhancing ammonia cracking to generate hydrogen represents a new paradigm for designing high performance plasma materials. The fundamental science of how plasma-nanoparticle interactions will change the plasma kinetics and will improve the energy efficiency for decarbonization and sustainability. The interpretable and physics-informed machine learning model will accelerate low temperature plasma chemistry and material discovery with physics rules and model interpretation.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗
DOE OSTI2021

Tensor Text-Mining Methods for Malware Identification and Detection, Malware Dynamics Characterization, and Hosts Ranking

Malware is one of the most persistent and costly cyber threats endangering reputation, confidentiality, integrity, and availability for organizations and national security. Consequently, many of the incident detection and prevention systems, and incident responders have begun to utilize machine learning as a helper in the fight against malware and other cyber threats. However, cyber defenders rely on interpretability and generalizability, yet the popular machine learning methods are black-box and often use traditional supervised solutions that do not generalize to novel malware. Therefore, there is a need to improve the existing solutions. At the same time, the majority of the prior research ignored essential evaluation criteria when reporting the results of their methods, which disables the safe reproducibility of the methods in a production environment. Tensor decomposition, on the other hand, enables interpretable unsupervised analysis of the large-scale data for the discovery of hidden patterns. Our findings, performed on real-world and large-scale experiments, show that tensor factorization-based methods yield performance results that surpasses or competes with existing supervised solutions with the added benefit of interpretability and generalizability. With the ability to analyze complex and large-scale data using tensors, we report results that reflect real-world production environments. We propose to develop new game- changing tools for malware identification and characterization that can trace malware evolution, rank the infected or malicious hosts, and streamline the work of incident response teams, malware analysts, and incident detection and prevention systems.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2026

Graph Identification of Proteins in Tomograms (GRIP-Tomo) 2.0: Topologically aware classification for proteins

Cryo-electron tomography (cryo-ET) enables structural characterization of biomolecules under near-native conditions. Existing approaches for interpreting the resulting three-dimensional volumes are computationally expensive and have difficulty interpreting density associated with small proteins/complexes. To explore alternate approaches for identifying proteins in cryo-ET data we pursued a Graph Network and topologically invariant approach. Here, we report on a fast algorithm that classifies particles by searching for nuances of evolutionarily conversed motifs and the geometrical characteristics of protein structure. GRIP-Tomo 2.0 is a machine-learning pipeline that extracts interpretable topological features of protein structures within noisy experimental backgrounds. Compared to version 1.0, the new pipeline includes three upgrades that significantly improve performance including synthetic tomogram generation simulating realistic noise, graph-based persistent feature extraction as protein fingerprints, and high-performance computing acceleration. GRIP-Tomo 2.0 achieves over 90% accuracy in classifying between proteins and noise using both real and synthetic datasets which represents a foundational step toward advancing cryo-ET workflows and empowering automated visual proteomics.

Li, Chengxuan↗
DOE OSTI2024

Interpreting Transformers for Jet Tagging

Machine learning (ML) algorithms, particularly attention-based transformer models, have become indispensable for analyzing the vast data generated by particle physics experiments like ATLAS and CMS at the CERN LHC. Particle Transformer (ParT), a state-of-the-art model, leverages particle-level attention to improve jet-tagging tasks, which are critical for identifying particles resulting from proton collisions. This study focuses on interpreting ParT by analyzing attention heat maps and particle-pair correlations on the $\eta$-$\phi$ plane, revealing a binary attention pattern where each particle attends to at most one other particle. At the same time, we observe that ParT shows varying focus on important particles and subjets depending on decay, indicating that the model learns traditional jet substructure observables. These insights enhance our understanding of the model's internal workings and learning process, offering potential avenues for improving the efficiency of transformer architectures in future high-energy physics applications.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗
DOE OSTI2021

Verification of Triso Fuel Burnup Using Machine Learning Algorithms

Pebble Bed Reactors are fueled with fuel pebbles that are circulated multiple times through the reactor vessel before discharge. During the normal operation of a PBR, ejected pebbles are returned to the reactor or discharged depending on the fuel burnup and physical condition of the pebbles. The burnup measurement is usually based on detected radiation signatures of fission products accumulated in the pebble fuel over burnup. Previous research has shown that height of photopeaks of fission products, such as 134 Cs, 137 Cs, 154 Eu, etc., can be used independently or in combination to infer or predict the level of burnup in the fuel. However, it remains challenging to measure such complex sources due to self-shielding effects, strong radiation background and intervening materials. Another operational challenge is the required high throughput of burnup measurement, which necessitates limited measurement time and thus impacts quality of measured gamma-ray spectra. Hence, advanced spectral analysis methods are needed to analyze the noisy gamma spectra and predict the burnup values. We propose to use machine learning (ML) method to interpret gamma-ray spectra and predict the burnup values of the pebbles. ML has achieved widespread success and adoption across a few domains that require pattern recognition and analysis in varied data types. In this work, we apply three proven ML approaches - multilayer perceptrons, convolutional neural networks, and transformers - to the task of predicting fuel burnup from measured gamma spectra, and compile a dataset of simulated spectra for training and validation of the ML models. In this paper, we will discuss the network architecture of these three ML approaches and compare the performance of the simplest of these (MLP) to a standard linear regression.

11 NUCLEAR FUEL CYCLE AND FUEL MATERIALS↗
DOE OSTI2022

GFINNs: GENERIC formalism informed neural networks for deterministic and stochastic dynamical systems

Here we propose the GENERIC formalism informed neural networks (GFINNs) that obey the symmetric degeneracy conditions of the GENERIC formalism. GFINNs comprise two modules, each of which contains two components. We model each component using a neural network whose architecture is designed to satisfy the required conditions. The component-wise architecture design provides flexible ways of leveraging available physics information into neural networks. We prove theoretically that GFINNs are sufficiently expressive to learn the underlying equations, hence establishing the universal approximation theorem. We demonstrate the performance of GFINNs in three simulation problems: gas containers exchanging heat and volume, thermoelastic double pendulum and the Langevin dynamics. In all the examples, GFINNs outperform existing methods, hence demonstrating good accuracy in predictions for both deterministic and stochastic systems.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2023

MACHINE LEARNING-ENABLED PREDICTION OF TRANSIENT INJECTION MAP IN AUTOMOTIVE INJECTORS WITH UNCERTAINTY QUANTIFICATION

Accurate prediction of injection profiles is a critical aspect of linking injector operation with engine performance and emissions. However, highly resolved injector simulations can take one to two weeks of wall-clock time, which is incompatible with engine design cycles with desired turnaround times of less than a day. Hence, it is important to reduce the time-to-solution of the internal flow simulations by several orders of magnitude to make it compatible with engine simulations. This work demonstrates a data-driven approach for tackling the computational overhead of injector simulations, whereby the transient injection profiles are emulated for a side-oriented, single-hole diesel injector using a Bayesian machine-learning framework. First, an interpretable Bayesian learning strategy was employed to understand the effect of design parameters on the total void fraction field. Then, autoencoders are utilized for efficient dimensionality reduction of the flowfields. Gaussian process models are finally used to predict the spatiotemporal void fraction field at the injector exit for unknown operating conditions. The Gaussian process models produce principled uncertainty estimates associated with the emulated flowfields, which provide the engine designer with valuable information of where the data-driven predictions can be trusted in the design space. The Bayesian flowfield predictions are compared with the corresponding predictions from a deep neural network, which has been transfer-learned from static needle simulations from a previous work by the authors. The emulation framework can predict the void fraction field at the exit of the orifice within a few seconds, thus achieving a speed-up factor of up to 38 x 10(6) over the traditional simulation-based approach of generating transient injection maps.

machine learning↗
DOE OSTI2021

VERIFICATION OF TRISO FUEL BURNUP USING MACHINE LEARNING ALGORITHMS

Pebble Bed Reactors are fueled with fuel pebbles that are circulated multiple times through the reactor vessel before discharge. During the normal operation of a PBR, ejected pebbles are returned to the reactor or discharged depending on the fuel burnup and physical condition of the pebbles. The burnup measurement is usually based on detected radiation signatures of fission products accumulated in the pebble fuel over burnup. Previous research has shown that height of photopeaks of fission products, such as 134Cs, 137Cs, 154Eu, etc., can be used independently or in combination to infer or predict the level of burnup in the fuel. However, it remains challenging to measure such complex sources due to self-shielding effects, strong radiation background and intervening materials. Another operational challenge is the required high throughput of burnup measurement, which necessitates limited measurement time and thus impacts quality of measured gamma-ray spectra. Hence, advanced spectral analysis methods are needed to analyze the noisy gamma spectra and predict the burnup values. We propose to use machine learning (ML) method to interpret gamma-ray spectra and predict the burnup values of the pebbles. ML has achieved widespread success and adoption across a few domains that require pattern recognition and analysis in varied data types. In this work, we apply three proven ML approaches - multilayer perceptrons, convolutional neural networks, and transformers - to the task of predicting fuel burnup from measured gamma spectra, and compile a dataset of simulated spectra for training and validation of the ML models. In this paper, we will discuss the network architecture of these three ML approaches and compare the performance of the simplest of these (MLP) to a standard linear regression.

98 NUCLEAR DISARMAMENT, SAFEGUARDS, AND PHYSICAL P↗