Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “interpretable machine learning”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 163 records · Page 9
All sourcesNASA NTRSDOE OSTI
DOE OSTI2022

Introduction to Special Section: Machine Learning for Image-based Geologic Interpretation

Image-based geological interpretation has been a labor-intensive and time-consuming process because it requires well-trained geoscientists to identify geological structures, features, and textures from various types of images. These images include scanning electron microscopic images, optical microscopic images, optical photos, resistivity images, seismic volumes, remote-sensing images, etc. With fast-evolving machine learning (ML) technology and computing power in recent decades, computers can achieve nearhuman-level to super-human-level performance with scalable high efficiency in the computer vision field. These technological revolutions facilitated image-based geological interpretation in petroleum exploration and production. For example, a fault picking method applied to 3-D seismic volume data using deep learning can achieve superior performance in comparison to conventional auto-picking methods. In addition, under the new normal of low oil prices, the petroleum industry seeks cost-effective strategies such as automating traditionally labor-intensive processes. Nevertheless, the potential of applying ML to geological image interpretation is still facing a few key challenges including data scarcity, data distribution, poor data and/or label quality, data leakage, learning algorithms, model architecture, training methodologies, testing and evaluation metrics, hyper-parameters optimization, model drift, production deployment, and the like.

58 GEOSCIENCES↗
DOE OSTI2022

Hypothesis testing via AI: Generating physically interpretable models of scientific data with machine learning (Full Technical Report)

Deep learning has demonstrated an exceptional ability to solve complex tasks (an engineering success); however, it has done so at the expense of the ability to generate new knowledge (a scientific failure). We propose an alternative framework—entitled Deep Symbolic Regression (DSR)—in which artificial neural networks (NNs) rapidly generate hypotheses about physical relationships among inputs. This framework bypasses the need to interpret an NN altogether, while still leveraging the representational power of deep learning. The resulting models are tractable mathematical expressions, which are inherently and readily human interpretable and can provide insights into underlying physical phenomena. Further, we fold this methodology into the scientific process by allowing the scientist to directly integrate a priori knowledge and beliefs to accelerate learning. We demonstrate this methodology on symbolic regression—the problem of rediscovering underlying expressions describing a dataset—and achieve state-of-the-art performance across a wide variety of symbolic regression problems. Further, we generalize our DSR framework to apply to the more general class of symbolic optimization problems, in which one seeks to optimize a sequence of symbols or “tokens” under a black-box reward function. Examples of other symbolic optimization problems include neural architecture search and computational antibody design. Our generalized tool, Deep Symbolic Optimization (DSO), has been demonstrated on the task of learning symbolic control policies for reinforcement learning environments, and has been adopted as an enabling capability for computational antibody design.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2021

Verification of Data-Driven Models of Physical Phenomena using Interpretable Approximation

Machine-learned models, specifically neural networks, are increasingly used as “closures” or “constitutive models” in engineering simulators to represent fine-scale physical phenomena that are too computationally expensive to resolve explicitly. However, these neural net models of unresolved physical phenomena tend to fail unpredictably and are therefore not used in mission-critical simulations. In this report, we describe new methods to authenticate them, i.e., to determine the (physical) information content of their training datasets, qualify the scenarios where they may be used and to verify that the neural net, as trained, adhere to physics theory. We demonstrate these methods with neural net closure of turbulent phenomena used in Reynolds Averaged Navier-Stokes equations. We show the types of turbulent physics extant in our training datasets, and, using a test flow of an impinging jet, identify the exact locations where the neural network would be extrapolating i.e., where it would be used outside the feature-space where it was trained. Using Generalized Linear Mixed Models, we also generate explanations of the neural net (à la Local Interpretable Model agnostic Explanations) at prototypes placed in the training data and compare them with approximate analytical models from turbulence theory. Finally, we verify our findings by reproducing them using two different methods.

42 ENGINEERING↗
DOE OSTI2023

Interpretable boosted-decision-tree analysis for the Majorana Demonstrator

The Majorana Demonstrator is a leading experiment searching for neutrinoless double-beta decay with high purity germanium detectors (HPGe). Machine learning provides a new way to maximize the amount of information provided by these detectors, but the data-driven nature makes it less interpretable compared to traditional analysis. An interpretability study reveals the machine's decision-making logic, allowing us to learn from the machine to feedback to the traditional analysis. In this work, we have presented the first machine learning analysis of the data from the Majorana Demonstrator; this is also the first interpretable machine learning analysis of any germanium detector experiment. Two gradient boosted decision tree models are trained to learn from the data, and a game-theory-based model interpretability study is conducted to understand the origin of the classification power. By learning from data, this analysis recognizes the correlations among reconstruction parameters to further enhance the background rejection performance. By learning from the machine, this analysis reveals the importance of new background categories to reciprocally benefit the standard Majorana analysis. This model is highly compatible with next-generation germanium detector experiments like LEGEND since it can be simultaneously trained on a large number of detectors.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗
DOE OSTI2026

Deciphering the Solvation Structure of Aqueous ZnCl 2 Solutions from X-ray Absorption Spectra Using the Interpretable Graph Neural Network

Machine learning (ML) provides powerful pathways for predicting spectroscopic observables from atomic structures, but its broader impact depends on making model predictions interpretable in terms of physical and chemical principles. Here, we introduce a physics-guided graph neural network (GNN) model that predicts Zn K-edge X-ray spectroscopy (XAS) spectra of aqueous ZnCl 2 solutions. Training data are generated from ab initio XAS calculations on molecular dynamics snapshots obtained using a machine learning interatomic potential. The GNN reproduces experimental spectra across concentrations from dilute (<0.1 m) to highly concentrated (30 m, “water-in-salt”) regimes and scales efficiently to large, disordered liquid systems beyond the reach of conventional ab initio approaches. Gradient-based attribution analysis reveals that the model learns physically meaningful structure-spectrum relationships. Ligand-specific attributions reflect orbital hybridization patterns and the origin of the excitations derived from the density functional theory. Bond-length attributions recover spectral shifts consistent with multiple-scattering theory. Finally, this work bridges data-driven prediction with electronic-structure theory, establishing a general paradigm for interpretable ML that links atomic structure, electronic structure, and spectroscopic observables.

25 ENERGY STORAGE↗
DOE OSTI2023

Using machine learning with optical profilometry for GaN wafer screening

Abstract To improve the manufacturing process of GaN wafers, inexpensive wafer screening techniques are required to both provide feedback to the manufacturing process and prevent fabrication on low quality or defective wafers, thus reducing costs resulting from wasted processing effort. Many of the wafer scale characterization techniques—including optical profilometry—produce difficult to interpret results, while models using classical programming techniques require laborious translation of the human-generated data interpretation methodology. Alternatively, machine learning techniques are effective at producing such models if sufficient data is available. For this research project, we fabricated over 6000 vertical PiN GaN diodes across 10 wafers. Using low resolution wafer scale optical profilometry data taken before fabrication, we successfully trained four different machine learning models. All models predict device pass and fail with 70–75% accuracy, and the wafer yield can be predicted within 15% error on the majority of wafers.

36 MATERIALS SCIENCE↗
DOE OSTI2024

Automatic Loss Factor Modeling and Attribution on Unlabeled PV Energy Data

We present a novel approach for modeling the loss factors of photovoltaic power generation systems (PV systems). This method is a white-box machine learning model built on convex optimization that is fast, interpretable, and auditable. It takes as an input the measured daily energy produced by the system, over a multi-year period, and returns a multiplicative decomposition model of the daily energy signal and full attribution of the total energy loss to each feature. The methods section of this paper has two major components: (1) the description of the signal decomposition (SD) model, expressed in the SD framework, and (2) the attribution of total energy losses via Shapley values. We validate the method on synthetic and open-source data sets and compare to similar methods from the literature.

artificial intelligence↗
DOE OSTI2021

hls4ml: An Open-Source Codesign Workflow to Empower Scientific Low-Power Machine Learning Devices

Accessible machine learning algorithms, software, and diagnostic tools for energy-efficient devices and systems are extremely valuable across a broad range of application domains. In scientific domains, real-time near-sensor processing can drastically improve experimental design and accelerate scientific discoveries. To support domain scientists, we have developed hls4ml, an open-source software-hardware codesign workflow to interpret and translate machine learning algorithms for implementation with both FPGA and ASIC technologies. We expand on previous hls4ml work by extending capabilities and techniques towards low-power implementations and increased usability: new Python APIs, quantization-aware pruning, end-to-end FPGA workflows, long pipeline kernels for low power, and new device backends include an ASIC workflow. Taken together, these and continued efforts in hls4ml will arm a new generation of domain scientists with accessible, efficient, and powerful tools for machine-learning-accelerated discovery.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗
DOE OSTI2023

Differentiable modelling to unify machine learning and physical models for geosciences

Process-based modelling offers interpretability and physical consistency in many domains of geosciences but struggles to leverage large datasets efficiently. Machine-learning methods, especially deep networks, have strong predictive skills yet are unable to answer specific scientific questions. Here, in this Perspective, we explore differentiable modelling as a pathway to dissolve the perceived barrier between process-based modelling and machine learning in the geosciences and demonstrate its potential with examples from hydrological modelling. ‘Differentiable’ refers to accurately and efficiently calculating gradients with respect to model variables or parameters, enabling the discovery of high-dimensional unknown relationships. Differentiable modelling involves connecting (flexible amounts of) prior physical knowledge to neural networks, pushing the boundary of physics-informed machine learning. It offers better interpretability, generalizability, and extrapolation capabilities than purely data-driven machine learning, achieving a similar level of accuracy while requiring less training data. Additionally, the performance and efficiency of differentiable models scale well with increasing data volumes. Under data-scarce scenarios, differentiable models have outperformed machine-learning models in producing short-term dynamics and decadal-scale trends owing to the imposed physical constraints. Differentiable modelling approaches are primed to enable geoscientists to ask questions, test hypotheses, and discover unrecognized physical relationships. Future work should address computational challenges, reduce uncertainty, and verify the physical significance of outputs.

58 GEOSCIENCES↗
DOE OSTI2024

Extreme sparsification of physics-augmented neural networks for interpretable model discovery in mechanics

Data-driven constitutive modeling with neural networks has received increased interest in recent years due to its ability to easily incorporate physical and mechanistic constraints and to overcome the challenging and time-consuming task of formulating phenomenological constitutive laws that can accurately capture the observed material response. However, even though neural network-based constitutive laws have been shown to generalize proficiently, the generated representations are not easily interpretable due to their high number of trainable parameters. Sparse regression approaches exist that allow for obtaining interpretable expressions, but the user is tasked with creating a library of model forms which by construction limits their expressiveness to the functional forms provided in the libraries. Here, in this work, we propose to train regularized physics-augmented neural network-based constitutive models utilizing a smoothed version of $L^0$-regularization. This aims to maintain the trustworthiness inherited by the physical constraints, but also enables interpretability which has not been possible thus far on any type of machine learning-based constitutive model where model forms were not assumed a priori but were actually discovered. During the training process, the network simultaneously fits the training data and penalizes the number of active parameters, while also ensuring constitutive constraints such as thermodynamic consistency. We show that the method can reliably obtain interpretable and trustworthy constitutive models for compressible and incompressible hyperelasticity, yield functions, and hardening models for elastoplasticity, using synthetic and experimental data. This work aims to set a new paradigm for interpretable machine learning models in the broad area of solid mechanics where low and limited data is available along with prior knowledge of physical constraints that the learned maps need to obey. This paradigm can potentially be extended to a broader spectrum of scientific exploration.

Data-driven constitutive models↗
DOE OSTI2025

Self-Supervised and Interpretable Anomaly Detection Using Network Transformers

Machine learning and deep neural networks (DNNs) have been proposed as a tool to identify anomalies in computer network communications. However, due the obfuscated nature of off-the-shelf machine learning models, their output often does not provide enough information to isolate the source of the anomaly to take corrective measures. In this article, we introduce the network transformer (NeT), a DNN model for anomaly detection that incorporates the graph structure of the communication network in order to improve interpretability. Further, the presented approach has the following advantages: first, enhanced interpretability by incorporating the graph structure of computer networks; second, provides a hierarchical set of features that enables analysis at different levels of granularity; second, self-supervised training that does not require labeled data. The NeT model was evaluated on a set of anomalous scenarios executed in a real industrial control system. The presented approach successfully identified the anomalies, the devices affected, and the specific connections causing the anomalies, providing a data-driven hierarchical approach to analyze the behavior of a cyber network.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2022

Development, characterization, and modeling of a high-performance Ru/B2CA catalyst for ammonia synthesis

This paper documents the development and performance of a nano-phase Ru catalyst on a (BaO) x (CaO) y (Al 2 O 3 ) support. Extensive screening of the support’s ternary composition shows the best stoichiometry is (BaO) 2 (CaO)(Al 2 O 3 ), denoted B2CA. The paper first describes catalyst preparation and characterization. The paper reports a detailed 12-step reaction mechanism that represents ammonia synthesis over wide ranges of temperature, pressure, space velocity, and feed composition. Additionally, the mechanism is developed and validated using results of packed-bed experiments. The elementary reaction pathways consider surface adsorbates, including catalyst-poisoning behaviors. The rate expressions include important coverage-dependent activation barriers. Machine learning models assist interpretation of the catalyst-support interactions. The detailed chemistry is much more predictive than is possible with global representations (N 2 + 3H 2 ⇌ 2NH 3 ). The validated models can be applied to assist optimizing reactor design and operating conditions.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2026

Generation of random geological models using multi-randomization for machine learning

Generating high-fidelity geological models is essential for advancing machine learning (ML) methods in automated seismic interpretation. For instance, seismic images paired with corresponding fault labels are foundational for ML-based fault detection from seismic migration sections. While several open-access datasets of random geological models exist, open-source tools specifically designed to produce large volumes of such models for ML applications remain scarce. To address this gap, we present RGM (Random Geological Model), an open-source software package for efficiently generating 2D and 3D synthetic geological models tailored for ML workflows. RGM supports the creation of diverse model components, including medium property distributions (P-/S-wave velocities and density), seismic reflectivity images (i.e., synthetic migration sections), relative geological time, and discrete fault attributes such as probability, dip, strike, rake, and displacement. It also accommodates the creation of complex geological features such as salt bodies and unconformities. The model generation algorithm employs a multi-randomization strategy, yielding an effectively infinite-dimensional model space that encompasses a wide range of geological scenarios and associated seismic features. Furthermore, RGM incorporates a method to generate synthetic elastic migration images using analytical elastic reflection coefficients combined with frequency-dependent scaling. This functionality enables the creation of training datasets for ML models that leverage elastic seismic images. RGM is implemented in modern object-oriented Fortran, allowing users to flexibly control statistical parameters governing model variability. We demonstrate the capability, performance, and geological realism of the package through comprehensive 2D and 3D examples.

58 GEOSCIENCES↗
DOE OSTI2025

Machine learning inversion from small-angle scattering for charged polymers

We develop Monte Carlo simulations for uniformly charged polymers and a machine learning algorithm to interpret the intra-polymer structure factor of the charged polymer system, which can be obtained from small-angle scattering experiments. The polymer is modeled as a chain of fixed-length bonds, where the connected bonds are subject to bending energy, and there is also a screened Coulomb potential for charge interaction between all joints. The bending energy is determined by the intrinsic bending stiffness, and the charge interaction depends on the interaction strength and screening length. All three contribute to the stiffness of the polymer chain and lead to longer and larger polymer conformations. The screening length also introduces a second length scale for the polymer besides the bending persistence length. To obtain the inverse mapping from the structure factor to these polymer conformation and energy-related parameters, we generate a large data set of structure factors by running simulations for a wide range of polymer energy parameters. We use principal component analysis to investigate the intra-polymer structure factors and determine the feasibility of the inversion using the nearest neighbor distance. We employ Gaussian process regression to achieve the inverse mapping and extract the characteristic parameters of polymers from the structure factor with low relative error.

36 MATERIALS SCIENCE↗
DOE OSTI2021

Machine learning to extend and understand the sources and limits of water cycle predictability on subseasonal-to-decadal timescales in the Earth system

This white paper provides initial insight on how artificial intelligence (AI) and machine learning (ML), including interpretability and explainable AI (XAI) methods, can be leveraged to glean insight from complex data for a paradigm-changing improvement in Earth system predictability on subseasonal-to-seasonal (S2S) and seasonal-to-decadal (S2D) timescales. The application of AI to extend and improve predictability, in combination with causal inference and uncertainty quantification, could lead to a transformative understanding of the integrative water cycle and associated extremes.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2024

Time Dilated Bundt Cake Analysis of PV Output [Poster]

We present a novel method for modeling time-dependent statistics in the power signal generated by a photovoltaic (PV) system. Our white-box machine learning method is interpretable and auditable, based on principles of multiperiodic basis functions and convex optimization. Our proposed method of time dilating the daily signal to remove night time values results in a novel representation of PV power signals, evocative of a ‘Bundt cake’. The proposed model describes the marginal distribution of power output as a function of date and time. The resulting probabilistic model of a PV system can be used to perform a variety of tasks, and here, we demonstrate the application of clear sky detection.

14 SOLAR ENERGY↗
DOE OSTI2020

On-the-fly active learning of interpretable Bayesian force fields for atomistic rare events

Abstract Machine learned force fields typically require manual construction of training sets consisting of thousands of first principles calculations, which can result in low training efficiency and unpredictable errors when applied to structures not represented in the training set of the model. This severely limits the practical application of these models in systems with dynamics governed by important rare events, such as chemical reactions and diffusion. We present an adaptive Bayesian inference method for automating the training of interpretable, low-dimensional, and multi-element interatomic force fields using structures drawn on the fly from molecular dynamics simulations. Within an active learning framework, the internal uncertainty of a Gaussian process regression model is used to decide whether to accept the model prediction or to perform a first principles calculation to augment the training set of the model. The method is applied to a range of single- and multi-element systems and shown to achieve a favorable balance of accuracy and computational efficiency, while requiring a minimal amount of ab initio training data. We provide a fully open-source implementation of our method, as well as a procedure to map trained models to computationally efficient tabulated force fields.

36 MATERIALS SCIENCE↗