Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Kernel learning”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 19 records
All sourcesNASA NTRSDOE OSTI
DOE OSTI2022

Kernel learning backward SDE filter for data assimilation

In this paper, we develop a kernel learning backward SDE filter method to estimate the state of a stochastic dynamical system based on its partial noisy observations. A system of forward backward stochastic differential equations is used to propagate the state of the target dynamical model, and Bayesian inference is applied to incorporate the observational information. Further, to characterize the dynamical model in the entire state space, we introduce a kernel learning method to learn a continuous global approximation for the conditional probability density function of the target state by using discrete approximated density values as training data. Numerical experiments demonstrate that the kernel learning backward SDE is highly effective.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2025

Active deep kernel learning of molecular properties from structural embeddings

As vast databases of chemical identities become increasingly available, the challenge shifts to how we effectively explore and leverage these resources to study molecular properties. This paper presents an active learning approach for molecular discovery using deep kernel learning (DKL), demonstrated on the QM9 dataset. DKL links structural embeddings directly to properties, creating organized latent spaces that prioritize relevant property information. By iteratively recalculating embedding vectors in alignment with target properties, DKL uncovers concentrated maxima representing key molecular properties and reveals unexplored regions with potential for innovation. This approach underscores DKL’s potential in advancing molecular research and discovery.

Artificial neural networks↗
DOE OSTI2022

Coarse-Grained Density Functional Theory Predictions via Deep Kernel Learning

Scalable electronic predictions are critical for soft materials design. Recently, the Electronic Coarse-Graining (ECG) method was introduced to renormalize all-atom quantum chemical (QC) predictions to coarse-grained (CG) resolutions using deep neural networks (DNNs). While DNNs can learn complex representations that prove challenging for kernel-based methods, they are susceptible to overfitting and the overconfidence of uncertainty estimations. Here, we develop ECG within a GPU-accelerated Deep Kernel Learning (DKL) framework to enable CG QC predictions using range-separated hybrid density functional theory (DFT), obtaining a 107 speedup relative to naive all-atom QC. By treating the predicted electronic properties as random Gaussian Processes, DKL incorporates CG mapping degeneracy by learning the distribution of electronic energies as a function of CG configuration. DKL-ECG accurately reproduces molecular orbital energies from range-separated DFT while facilitating efficient training via active learning using the uncertainties provided by DKL. Further, we show that while active learning algorithms enable efficient sampling of a more diverse configurational space relative to random sampling, all explored query methods exhibit comparable performance for the examined system. We attribute this result to the significant overlap of the feature space and output property distributions across multiple temperatures.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2021

Exploring Causal Physical Mechanisms via Non-Gaussian Linear Models and Deep Kernel Learning: Applications for Ferroelectric Domain Structures

Rapid emergence of multimodal imaging in scanning probe, electron, and optical microscopies has brought forth the challenge of understanding the information contained in these complex data sets, targeting the intrinsic correlations between different channels, and further exploring the underpinning causal physical mechanisms. Here, we develop such an analysis framework for Piezoresponse Force Microscopy. We argue that under certain conditions, we can bootstrap experimental observations with the prior knowledge of materials structure to get information on certain nonobserved properties, and demonstrate linear causal analysis for PFM observables. We further demonstrate that the strength of individual causal links between complex descriptors can be ascertained using the deep kernel learning (DKL) model. In this DKL analysis, we use the prior information on domain structure within the image to predict the physical properties. This analysis demonstrates the correlative relationships between morphology, piezoresponse, elastic property, etc., at nanoscale. The prediction of morphology and other physical parameters illustrates a mutual interaction between surface condition and physical properties in ferroelectric materials. Overall, this analysis is universal and can be extended to explore the correlative relationships of other multichannel data sets, and allow for high-fidelity reconstruction of underpinning functionalities and physical mechanisms.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2021

Simple, low-cost and accurate data-driven geophysical forecasting with learned kernels

Modelling geophysical processes as low-dimensional dynamical systems and regressing their vector field from data is a promising approach for learning emulators of such systems. We show that when the kernel of these emulators is also learned from data (using kernel flows, a variant of cross-validation), then the resulting data-driven models are not only faster than equation-based models but are easier to train than neural networks such as the long short-term memory neural network. In addition, they are also more accurate and predictive than the latter. When trained on geophysical observational data, for example the weekly averaged global sea-surface temperature, considerable gains are also observed by the proposed technique in comparison with classical partial differential equation-based models in terms of forecast computational cost and accuracy. When trained on publicly available re-analysis data for the daily temperature of the North American continent, we observe significant improvements over classical baselines such as climatology and persistence-based forecast techniques. Although our experiments concern specific examples, the proposed approach is general, and our results support the viability of kernel methods (with learned kernels) for interpretable and computationally efficient geophysical forecasting for a large diversity of processes.

58 GEOSCIENCES↗
DOE OSTI2024

Parallel hybrid quantum-classical machine learning for kernelized time-series classification

Supervised time-series classification garners widespread interest because of its applicability throughout a broad application domain including finance, astronomy, biosensors, and many others. Here, in this work, we tackle this problem with hybrid quantum-classical machine learning, deducing pairwise temporal relationships between time-series instances using a timeseries Hamiltonian kernel (TSHK). A TSHK is constructed with a sum of inner products generated by quantum states evolved using a parameterized time evolution operator. This sum is then optimally weighted using techniques derived from multiple kernel learning. Because we treat the kernel weighting step as a differentiable convex optimization problem, our method can be regarded as an end-to-end learnable hybrid quantum-classical-convex neural network, or QCC-net, whose output is a data set-generalized kernel function suitable for use in any kernelized machine learning technique such as the support vector machine (SVM). Using our TSHK as input to a SVM, we classify univariate and multivariate time-series using quantum circuit simulators and demonstrate the efficient parallel deployment of the algorithm to 127-qubit superconducting quantum processors using quantum multi-programming.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2024

Deep kernel methods learn better: from cards to process optimization

Abstract The ability of deep learning methods to perform classification and regression tasks relies heavily on their capacity to uncover manifolds in high-dimensional data spaces and project them into low-dimensional representation spaces. In this study, we investigate the structure and character of the manifolds generated by classical variational autoencoder (VAE) approaches and deep kernel learning (DKL). In the former case, the structure of the latent space is determined by the properties of the input data alone, while in the latter, the latent manifold forms as a result of an active learning process that balances the data distribution and target functionalities. We show that DKL with active learning can produce a more compact and smooth latent space which is more conducive to optimization compared to previously reported methods, such as the VAE. We demonstrate this behavior using a simple cards dataset and extend it to the optimization of domain-generated trajectories in physical systems. Our findings suggest that latent manifolds constructed through active learning have a more beneficial structure for optimization problems, especially in feature-rich target-poor scenarios that are common in domain sciences, such as materials synthesis, energy storage, and molecular discovery. The Jupyter Notebooks that encapsulate the complete analysis accompany the article.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2023

Interpretable Data-Driven Probabilistic Power System Load Margin Assessment with Uncertain Renewable Energy and Loads

The increasing uncertainties caused by the high-penetration of stochastic renewable generation resources poses a significant threat to the power system voltage stability. To address this issue, this paper proposes a probabilistic deep kernel learning enabled surrogate model to extract the hidden relationship between uncertain sources, i.e., wind power and loads, and load margin for probabilistic load margin assessment (PLMA). Unlike other deep learning approaches, a kernel SHAP provides the sensitivity analysis as well as interpretability of the inputs to outputs influences. This allows identifying the critical factors that affect load margin so that corrective control can be initiated for stability enhancement. Numerical results carried out on the IEEE 118-bus power system demonstrate the accuracy and efficiency of the proposed data-driven PLMA scheme.

deep kernel learning↗
DOE OSTI2025

Quantum adversarial learning for kernel methods

We show that hybrid quantum classifiers based on quantum kernel methods and support vector machines are vulnerable against adversarial attacks, namely small engineered perturbations of the input data can deceive the classifier into predicting the wrong result. Nonetheless, we also show that simple defense strategies based on data augmentation with a few crafted perturbations can make the classifier robust against new attacks. Our results find applications in security-critical learning problems and in mitigating the effect of some forms of quantum noise, since the attacker can also be understood as part of the surrounding environment.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
DOE OSTI2022

Importance of kernel bandwidth in quantum machine learning

Quantum kernel methods are considered a promising avenue for applying quantum computers to machine learning problems. Identifying hyperparameters controlling the inductive bias of quantum machine learning models is expected to be crucial given the central role hyperparameters play in determining the performance of classical machine learning methods. In this work we introduce the hyperparameter controlling the bandwidth of a quantum kernel and show that it controls the expressivity of the resulting model. We use extensive numerical experiments with multiple quantum kernels and classical data sets to show consistent change in the model behavior from underfitting (bandwidth too large) to overfitting (bandwidth too small), with optimal generalization in between. We draw a connection between the bandwidth of classical and quantum kernels and show analogous behavior in both cases. Furthermore, we show that optimizing the bandwidth can help mitigate the exponential decay of kernel values with qubit count, which is the cause behind recent observations that the performance of quantum kernel methods decreases with qubit count. Here, we reproduce these negative results and show that if the kernel bandwidth is optimized, the performance instead improves with growing qubit count and becomes competitive with the best classical methods.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
DOE OSTI2022

Automated Experiment in 4D-STEM: Exploring Emergent Physics and Structural Behaviors

Automated experiments in 4D scanning transmission electron microscopy (STEM) are implemented for rapid discovery of local structures, symmetry-breaking distortions, and internal electric and magnetic fields in complex materials. Deep kernel learning enables active learning of the relationship between local structure and 4D-STEM-based descriptors. With this, efficient and “intelligent” probing of dissimilar structural elements to discover desired physical functionality is made possible. This approach allows effective navigation of the sample in an automated fashion guided by either a predetermined physical phenomenon, such as strongest electric field magnitude, or in an exploratory fashion. Therefore, we verify the approach first on preacquired 4D-STEM data and further implement it experimentally on an operational STEM. The experimental discovery workflow is demonstrated using graphene and subsequently extended toward a lesser-known layered 2D van der Waals material, MnPS 3 . This approach establishes a pathway for physics-driven automated 4D-STEM experiments that enable probing the physics of strongly correlated systems and quantum materials and devices, as well as exploration of beam-sensitive materials.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2024

Unraveling the impact of initial choices and in-loop interventions on learning dynamics in autonomous scanning probe microscopy

The current focus in Autonomous Experimentation (AE) is on developing robust workflows to conduct the AE effectively. This entails the need for well-defined approaches to guide the AE process, including strategies for hyperparameter tuning and high-level human interventions within the workflow loop. This paper presents a comprehensive analysis of the influence of initial experimental conditions and in-loop interventions on the learning dynamics of Deep Kernel Learning (DKL) within the realm of AE in scanning probe microscopy. We explore the concept of the “seed effect,” where the initial experiment setup has a substantial impact on the subsequent learning trajectory. Additionally, we introduce an approach of the seed point interventions in AE allowing the operator to influence the exploration process. Using a dataset from Piezoresponse Force Microscopy on PbTiO 3 thin films, we illustrate the impact of the “seed effect” and in-loop seed interventions on the effectiveness of DKL in predicting material properties. The study highlights the importance of initial choices and adaptive interventions in optimizing learning rates and enhancing the efficiency of automated material characterization. This work offers valuable insights into designing more robust and effective AE workflows in microscopy with potential applications across various characterization techniques.

47 OTHER INSTRUMENTATION↗
DOE OSTI2022

Bayesian Active Learning for Scanning Probe Microscopy: From Gaussian Processes to Hypothesis Learning

Recent progress in machine learning methods and the emerging availability of programmable interfaces for scanning probe microscopes (SPMs) have propelled automated and autonomous microscopies to the forefront of attention of the scientific community. However, enabling automated microscopy requires the development of task-specific machine learning methods, understanding the interplay between physics discovery and machine learning, and fully defined discovery workflows. This, in turn, requires balancing the physical intuition and prior knowledge of the domain scientist with rewards that define experimental goals and machine learning algorithms that can translate these to specific experimental protocols. Here, we discuss the basic principles of Bayesian active learning and illustrate its applications for SPM. We progress from the Gaussian process as a simple data-driven method and Bayesian inference for physical models as an extension of physics-based functional fits to more complex deep kernel learning methods, structured Gaussian processes, and hypothesis learning. These frameworks allow for the use of prior data, the discovery of specific functionalities as encoded in spectral data, and exploration of physical laws manifesting during the experiment. Here, the discussed framework can be universally applied to all techniques combining imaging and spectroscopy, SPM methods, nanoindentation, electron microscopy and spectroscopy, and chemical imaging methods and can be particularly impactful for destructive or irreversible measurements.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2022

Physics Discovery in Nanoplasmonic Systems via Autonomous Experiments in Scanning Transmission Electron Microscopy

Abstract Physics‐driven discovery in an autonomous experiment has emerged as a dream application of machine learning in physical sciences. Here, this work develops and experimentally implements a deep kernel learning (DKL) workflow combining the correlative prediction of the target functional response and its uncertainty from the structure, and physics‐based selection of acquisition function, which autonomously guides the navigation of the image space. Compared to classical Bayesian optimization (BO) methods, this approach allows to capture the complex spatial features present in the images of realistic materials, and dynamically learn structure–property relationships. In combination with the flexible scalarizer function that allows to ascribe the degree of physical interest to predicted spectra, this enables physical discovery in automated experiment. Here, this approach is illustrated for nanoplasmonic studies of nanoparticles and experimentally implemented in a truly autonomous fashion for bulk‐ and edge plasmon discovery in MnPS 3 , a lesser‐known beam‐sensitive layered 2D material. This approach is universal, can be directly used as‐is with any specimen, and is expected to be applicable to any probe‐based microscopic techniques including other STEM modalities, scanning probe microscopies, chemical, and optical imaging.

42 ENGINEERING↗
DOE OSTI2023

Learning the right channel in multimodal imaging: automated experiment in piezoresponse force microscopy

We report the development and experimental implementation of the automated experiment workflows for the identification of the best predictive channel for a phenomenon of interest in spectroscopic measurements. The approach is based on the combination of ensembled deep kernel learning for probabilistic predictions and a basic reinforcement learning policy for channel selection. It allows the identification of which of the available observational channels, sampled sequentially, are most predictive of selected behaviors, and hence have the strongest correlations. We implement this approach for multimodal imaging in piezoresponse force microscopy (PFM), with the behaviors of interest manifesting in piezoresponse spectroscopy. We illustrate the best predictive channel for polarization-voltage hysteresis loop and frequency-voltage hysteresis loop areas is amplitude in the model samples. The same workflow and code are applicable for any multimodal imaging and local characterization methods.

36 MATERIALS SCIENCE↗
DOE OSTI2026

Accelerating Structure–Property Relationship Discovery with Multimodal Machine Learning and Self-Driving Microscopy

Microscopy combined with local spectroscopy is widely used to correlate nanoscale structure with functional properties in materials, but conventional measurements rely heavily on human-selected sampling locations and predefined targets, limiting data set diversity and the potential for discovery. Here, we present a framework that integrates autonomous microscopy with dual-novelty deep kernel learning (DN-DKL) for adaptive data acquisition and a dual variational autoencoder (VAE) for representation learning. DN-DKL actively guides the microscopy toward structurally and spectroscopically novel regions, enabling efficient collection of large spectral data sets. Dual-VAE embeds local structures and spectroscopic responses into a shared latent manifold that serves as a structure–property relationship map. We applied this framework for the investigation of halide perovskite films by using conductive atomic force microscopy. The results reveal distinct hysteresis behaviors that are linked to specific nanoscale structural motifs, including grain boundary junction points that show hysteresis under different bias conditions and asymmetric grain boundaries that suppress the charge transport. This framework establishes a general strategy that leverages the complementary strengths of self-driving microscopy, machine learning, and human expertise to accelerate scientific discovery in functional materials.

atomic force microscopy↗
DOE OSTI2022

Automated Experiments of Local Non–Linear Behavior in Ferroelectric Materials

An automated experiment in multimodal imaging to probe structural, chemical, and functional behaviors in complex materials and elucidate the dominant physical mechanisms that control device function is developed and implemented. Here, the emergence of non-linear electromechanical responses in piezoresponse force microscopy (PFM) is explored. Non-linear responses in PFM can originate from multiple mechanisms, including intrinsic material responses often controlled by domain structure, surface topography that affects the mechanical phenomena at the tip-surface junction, and the presence of surface contaminants. Using an automated experiment to probe the origins of non-linear behavior in ferroelectric lead titanate (PTO) and ferroelectric Al 0.93 B 0.07 N films, it is found that PTO shows asymmetric nonlinear behavior across a/c domain walls and a broadened high nonlinear response region around c/c domain walls. In contrast, for Al 0.93 B 0.07 N, well-poled regions show high linear piezoelectric responses, when paired with low non-linear responses regions that are multidomain show low linear responses and high nonlinear responses. It is shown that formulating dissimilar exploration strategies in deep kernel learning as alternative hypotheses allows for establishing the preponderant physical mechanisms behind the non-linear behaviors, suggesting that automated experiments can potentially discern between competing physical mechanisms. Furthermore, this technique can also be extended to electron, probe, and chemical imaging.

36 MATERIALS SCIENCE↗