Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Bayesian learning”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 325 records · Page 18
All sourcesNASA NTRSDOE OSTI
DOE OSTI2024

Tokamak divertor plasma emulation with machine learning

Abstract Future tokamak devices that aim to create conditions relevant to power plant operations must consider strategies for mitigating damage to plasma facing components in the divertor. One of the goals of MAST-U tokamak operations is to inform these considerations by researching advanced divertor configurations that aid stable plasma detachment. Machine design, scenario planning and detachment control would all greatly benefit from tools that enable rapid calculation of scenario-relevant quantities given some input parameters. This paper presents a method for generating large, simulated scrape-off layer data sets, which was applied to generate a data set of steady-state Hermes-3 simulations of the MAST-U tokamak. A machine learning model was constructed using a Bayesian approach to hyperparameter optimisation to predict diagnosable output quantities given control-relevant input features. The resulting best-performing model, which is based on a feedforward neural network, achieves high accuracy when predicting electron temperature at the divertor target and carbon impurity radiation front position and runs in around 1 ms in inference mode. Techniques for interpreting the predictions made by the model were applied, and a high-resolution parameter scan of upstream conditions was performed to demonstrate the utility of rapidly generating accurate predictions using the emulator. This work represents a step forward in the design of machine learning-driven emulators of tokamak exhaust simulation codes in operational modes relevant to divertor detachment control and plasma scenario design.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗
NASA NTRS1994

Operations for Learning with Graphical Models

This paper is a multidisciplinary review of empirical, statistical learning from a graphical model perspective. Well-known examples of graphical models include Bayesian net- works, directed graphs representing a Markov chain, and undirected networks representing a Markov field. These graphical models are extended to model data analysis and empirical learning using the notation of plates. Graphical operations for simplifying and manipulating a problem are provided including decomposition, differentiation, and the manipulation of probability models from the exponential family. These operations adapt existing techniques from statistics and automatic differentiation to graphs. Two standard algorithm schemes for learning are reviewed in a graphical framework: Gibbs sampling and the expectation maximization algorithm. Some algorithms are developed in this graphical framework including a generalized version of linear regression, techniques for feed-forward networks, and learning Gaussian and discrete Bayesian networks from data. The paper concludes by sketching some implications for data analysis and summarizing some popular algorithms that fall within the framework presented. The main original contributions here are the decomposition techniques and the demonstration that graphical models provide a framework for understanding and developing complex learning algorithms.

Buntine, Wray L.↗
DOE OSTI2022

Machine Learning on a Robotic Platform for the Design of Polymer–Protein Hybrids

Polymer–protein hybrids are intriguing materials that can bolster protein stability in non-native environments, thereby enhancing their utility in diverse medicinal, commercial, and industrial applications. One stabilization strategy involves designing synthetic random copolymers with compositions attuned to the protein surface, but rational design is complicated by the vast chemical and composition space. Here, a strategy is reported to design protein-stabilizing copolymers based on active machine learning, facilitated by automated material synthesis and characterization platforms. The versatility and robustness of the approach is demonstrated by the successful identification of copolymers that preserve, or even enhance, the activity of three chemically distinct enzymes following exposure to thermal denaturing conditions. Although systematic screening results in mixed success, active learning appropriately identifies unique and effective copolymer chemistries for the stabilization of each enzyme. Overall, this work broadens the capabilities to design fit-for-purpose synthetic copolymers that promote or otherwise manipulate protein activity, with extensions toward the design of robust polymer–protein hybrid materials.

36 MATERIALS SCIENCE↗
DOE OSTI2024

Structural mode coupling in perovskite oxides using hypothesis-driven active learning

Abstract Finding the ground-state structure with minimum energy is paramount to designing any material. In ABO 3 -type perovskite oxides with Pnma symmetry, the lowest energy phase is driven by an inherent trilinear coupling between the two primary order parameters such as rotation and tilt with antiferroelectric displacement of the A-site cations as established via hybrid improper ferroelectric mechanism. Conventionally, finding the relevant mode coupling driving phase transition requires performing first-principles calculations which is computationally time-consuming as well as expensive. It involves following an intuitive iterative hit and trial method of (a) adding two or multiple mode vectors, followed by (b) evaluating which combination would lead to the ground-state energy. In this study, we show how a hypothesis-driven active learning framework can identify suitable mode couplings within the Landau free energy expansion with minimal information on amplitudes of modes for a series of double perovskite oxides with A-site layered, columnar and rocksalt ordering. This scheme is expected to be applicable universally for understanding atomistic mechanisms derived from various structural mode couplings behind functionalities, for e.g. polarization, magnetization and metal–insulator transitions.

36 MATERIALS SCIENCE↗
DOE OSTI2024

Neural simulation-based inference of the neutron star equation of state directly from telescope spectra

Neutron stars provide a unique opportunity to study strongly interacting matter under extreme density conditions. The intricacies of matter inside neutron stars and their equation of state are not directly visible, but determine bulk properties, such as mass and radius, which affect the star's thermal X-ray emissions. However, the telescope spectra of these emissions are also affected by the stellar distance, hydrogen column, and effective surface temperature, which are not always well-constrained. Uncertainties on these nuisance parameters must be accounted for when making a robust estimation of the equation of state. In this study, we develop a novel methodology that, for the first time, can infer the full posterior distribution of both the equation of state and nuisance parameters directly from telescope observations. This method relies on the use of neural likelihood estimation, in which normalizing flows use samples of simulated telescope data to learn the likelihood of the neutron star spectra as a function of these parameters, coupled with Hamiltonian Monte Carlo methods to efficiently sample from the corresponding posterior distribution. Our approach surpasses the accuracy of previous methods, improves the interpretability of the results by providing access to the full posterior distribution, and naturally scales to a growing number of neutron star observations expected in the coming years.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2023

Deducing neutron star equation of state from telescope spectra with machine-learning-derived likelihoods

The interiors of neutron stars reach densities and temperatures beyond the limits of terrestrial experiments, providing vital laboratories for probing nuclear physics. While the star's interior is not directly observable, its pressure and density determine the star's macroscopic structure which affects the spectra observed in telescopes. The relationship between the observations and the internal state is complex and partially intractable, presenting difficulties for inference. Previous work has focused on the regression from stellar spectra of parameters describing the internal state. We demonstrate a calculation of the full likelihood of the internal state parameters given observations, accomplished by replacing intractable elements with machine learning models trained on samples of simulated stars. Our machine-learning-derived likelihood allows us to perform maximum a posteriori estimation of the parameters of interest, as well as full scans. We demonstrate the technique by inferring stellar mass and radius from an individual stellar spectrum, as well as equation of state parameters from a set of spectra. Our results are more precise than pure regression models, reducing the width of the parameter residuals by 11.8% in the most realistic scenario. The neural networks will be released as a tool for fast simulation of neutron star properties and observed spectra.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2020

A machine learning Automated Recommendation Tool for synthetic biology

Abstract Synthetic biology allows us to bioengineer cells to synthesize novel valuable molecules such as renewable biofuels or anticancer drugs. However, traditional synthetic biology approaches involve ad-hoc engineering practices, which lead to long development times. Here, we present the Automated Recommendation Tool (ART), a tool that leverages machine learning and probabilistic modeling techniques to guide synthetic biology in a systematic fashion, without the need for a full mechanistic understanding of the biological system. Using sampling-based optimization, ART provides a set of recommended strains to be built in the next engineering cycle, alongside probabilistic predictions of their production levels. We demonstrate the capabilities of ART on simulated data sets, as well as experimental data from real metabolic engineering projects producing renewable biofuels, hoppy flavored beer without hops, fatty acids, and tryptophan. Finally, we discuss the limitations of this approach, and the practical consequences of the underlying assumptions failing.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2021

Particle image velocimetry analysis with simultaneous uncertainty quantification using Bayesian neural networks

Particle image velocimetry (PIV) is an effective tool in experimental fluid mechanics for extracting flow fields from images. Recently, convolutional neural networks (CNNs) have been used to perform PIV analysis with accuracy on par with classical methods. Here we extend the use of CNNs to analyze PIV data while providing simultaneous uncertainty quantification on the inferred flow field. The method we apply in this paper is a Bayesian convolutional neural network (BCNN) which learns distributions of the CNN weights through variational Bayes. In order to demonstrate the utility of BCNNs for the PIV task, we compare the performance of three distinct BCNN models with simple architectures. The first network estimates flow velocity from image interrogation regions only. Our second model learns to infer velocity from both the image interrogation regions and interrogation region cross-correlation maps. Finally, our best performing network infers velocities from interrogation region cross-correlation maps only. We find that BCNNs using interrogation region cross-correlation maps as inputs perform better than those using interrogation windows only as inputs and discuss reasons why this may be the case. Additionally, we test the best performing BCNN on a full synthetic test image pair and a real image pair from the 1st International PIV Challenge. We show that ~98% of true particle displacements from the full synthetic image pair can be captured within the BCNN's 95% confidence intervals, and that the BCNN's performance on the real image pair is quantitatively similar to that of algorithms tested in the 1st International PIV Challenge. Finally, we show that BCNNs can be generalized to be used with multi-pass PIV algorithms with a moderate loss in accuracy, which may be overcome by future work on finetuning and training schemes. So to our knowledge, this is the first use of Bayesian neural networks to perform PIV.

47 OTHER INSTRUMENTATION↗
DOE OSTI2024

Bayesian Analysis of TRISO Fuel: Quantifying Model Inadequacy, Incorporating Lower-Length-Scale Effects, and Developing Parallel Active Learning Capabilities

The U.S. Department of Energy (DOE)’s Nuclear Energy Advanced Modeling and Simulation (NEAMS) program aims to develop predictive capabilities by applying computational methods to the analysis and design of advanced reactor and fuel-cycle systems. This program has been providing engineering-scale support for the continued development of BISON, a high-fidelity, high-resolution fuel performance tool. Fuel behavior in nuclear reactors is governed by a complex network of mechanisms that interact with various other physics aspects in the reactor system. Any model developed to represent fuel behavior will likely be idealized, resulting in uncertainties when comparing their predictions against the observed data. In Fiscal Year (FY)-23, we initiated the Uncertainty Quantification (UQ) work by using Bayesian methods to establish a level of model trustworthiness and further improve it, with a particular emphasis on TRI-Structural isOtropic (TRISO) nuclear fuel. This year, we further expanded on that UQ work by investigating an approach to quantifying model inadequacy and accounting for lower-length scale (LLS) effects in TRISO silver (Ag) release modeling. Furthermore, we are implementing parallel active learning capabilities to reduce the computational cost (i.e., required computational resources and elapsed time) of performing UQ. Specifically, we utilized The Kennedy O’Hagan framework for Bayesian uncertainty quantification (KOH) to account for model inadequacy in TRISO Ag release predictions made by BISON. The KOH framework represents an improvement over the standard Bayesian framework used in FY-23. Explicitly accounting for model inadequacy in the Bayesian framework helps establish the level of experimental noise uncertainty in the Advanced Gas Reactor (AGR) data. We compared the inverse UQ results obtained from both the standard Bayesian and KOH frameworks in light of the AGR-2/3/4 data, and also compared the predictive UQ results obtained from these two frameworks in light of the AGR-1 data. Next, we investigated the impact of considering LLS effects in the Ag release simulations. We developed an expanded database of LLS simulated effective diffusivities for Ag, covering a wide range of microstructures and temperatures. Using this database, we developed a framework for incorporating LLS effects into the engineering-scale Ag release UQ. We developed both parametric and non-parametric approaches for bridging the length scales. We then investigated the inverse UQ results in light of the AGR-2/3/4 data and the predictive UQ results in light of the AGR-1 data, and compared the LLS-informed approach and the Arrhenius equation, which does not include microstructure information. Finally, we discussed implementing parallel active learning capabilities in the Multiphysics Object Oriented Simulation Environment (MOOSE)/BISON to reduce the computational cost (i.e., computational resources and elapsed time) of Bayesian UQ. For verification purposes, we first tested these new capabil ities on a species interaction problem. We then demonstrated them on the TRISO Ag release application, showing that parallel active learning capabilities can enhance the accuracy of UQ while also substantially reducing the computational cost in comparison to the reference methods developed in FY-23.

11 NUCLEAR FUEL CYCLE AND FUEL MATERIALS↗
DOE OSTI2025

MOOSE ProbML: Parallelizable Probabilistic Machine Learning and Uncertainty Quantification Capabilities

The Multiphysics Object Oriented Simulation Environment (MOOSE) is a widely used open- source finite element software for performing multiphysics multiscale simulations in a massively parallel fashion. Recently, the computational team at Idaho National Laboratory (INL) has implemented Probabilistic Machine Learning (ProbML) capabilities in MOOSE—in a parallelized fashion—and enable active learning with large-scale computational models for tasks such as surrogate model development, scale bridging, forward/inverse uncertainty quantification (UQ), Bayesian optimization, etc. This presentation summarizes these developments in MOOSE along with demonstrations on several real applications relevant to nuclear energy. At the fundamental level, samplers like Monte Carlo/Latin Hypercube, variance reduction, parallelized Markov Chain Monte Carlo (MCMC) support uncertainty propagation in both forward and inverse settings. These samplers can be integrated with the Gaussian processes (GP) suite in MOOSE, which offer several variants like scalar GPs, multi-output GPs, and deep GPs, to enable active learning. These GPs can be tuned using gradient-based optimization methods like Adam and its variants or gradient-free methods like the elliptical slice sampler (a variant of MCMC adept under Gaussian settings) for more complex covariance kernels or likelihoods whose gradient computations can be cumbersome. A variety of batch acquisition functions permit parallelized evaluation of the computational model and support different learning objectives with high efficiency like Bayesian inference, global surrogate development, optimization, etc. Furthermore, libtorch integration supports training, evaluation, and re-training of neural networks and other complex machine learning models in active learning settings. The impacts of these developments are shown on several real applications: (1) nuclear fuel inverse UQ and model inadequacy assessment using the Kennedy O’Hagan framework; (2) uncertainty aware surrogate modeling for additive manufacturing to predict field quantities; (3) nuclear reactor rare events analysis; and (4) complex fluid flow prediction using a global surrogate with quantified prediction uncertainty. Finally, the outlook of MOOSE ProbML is discussed for both outer-loop and inner-loop computations in the broad view to accelerate fuels and materials qualification, address gaps in knowledge and data, and assess new reactor/fuel systems.

11 - NUCLEAR FUEL CYCLE AND FUEL MATERIALS↗
DOE OSTI2022

Posterior Regularized Bayesian Neural Network incorporating soft and hard knowledge constraints

Neural Networks (NNs) have been widely used in supervised learning due to their ability to model complex nonlinear patterns, often presented in high-dimensional data such as images and text. However, traditional NNs often lack the ability for uncertainty quantification. Bayesian NNs (BNNS) could help measure the uncertainty by considering the distributions of the NN model parameters. Besides, domain knowledge is commonly available and could improve the performance of BNNs if it can be appropriately incorporated. In this work, we propose a novel Posterior-Regularized Bayesian Neural Network (PR-BNN) model by incorporating different types of knowledge constraints, such as the soft and hard constraints, as a posterior regularization term. Furthermore, we propose to combine the augmented Lagrangian method and the existing BNN solvers for efficient inference. Furthermore, the experiments in simulation and two case studies about aviation landing prediction and solar energy output prediction have shown the knowledge constraints and the performance improvement of the proposed model over traditional BNNs without the constraints.

14 SOLAR ENERGY↗
DOE OSTI2022

Variational encoder geostatistical analysis (VEGAS) with an application to large scale riverine bathymetry

Estimation of riverbed profiles, also known as bathymetry, plays a vital role in many applications, such as safe and efficient inland navigation, prediction of bank erosion, land subsidence, and flood risk management. The high cost and complex logistics of direct bathymetry surveys, i.e, depth imaging, have encouraged the use of indirect measurements such as surface flow velocities. However, estimating high-resolution bathymetry from indirect measurements is an inverse problem that can be computationally challenging. Here, we propose a reduced-order model (ROM) based approach that utilizes a variational autoencoder (VAE), a type of deep neural network with a narrow layer in the middle, to compress bathymetry and flow velocity information and accelerate bathymetry inverse problems from flow velocity measurements. In our application, the shallow-water equations (SWE) with appropriate boundary conditions (BCs), e.g., the discharge and/or the free surface elevation, constitute the forward problem, to predict flow velocity. Then, ROMs of the SWEs are constructed on a nonlinear manifold of low dimensionality through a variational encoder and the bathymetry inversion problem is derived on the low-dimensional latent space in a Hierarchical Bayesian setting. Further, the reformulation allows variational inference with a small number (e.g., $\mathscr{O}$ (100) of ROM runs and efficient uncertainty quantification. We have tested our inversion approach on a one-mile reach of the Savannah River, GA, USA. Once the neural network is trained (offline stage), the proposed technique can perform the inversion operation orders of magnitude faster than traditional inversion methods that are commonly based on linear projections, such as principal component analysis (PCA), or the principal component geostatistical approach (PCGA). Furthermore, tests show that the algorithm can estimate the bathymetry with good accuracy even with sparse flow velocity measurements.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2021

Autonomous Experiments in Scanning Probe Microscopy and Spectroscopy: Choosing Where to Explore Polarization Dynamics in Ferroelectrics

Polarization dynamics in ferroelectric materials are explored via automated experiment in piezoresponse force microscopy/spectroscopy (PFM/S). A Bayesian optimization (BO) framework for imaging is developed, and its performance for a variety of acquisition and pathfinding functions is explored using previously acquired data. The optimized algorithm is then deployed on an operational scanning probe microscope (SPM) for finding areas of large electromechanical response in a thin film of PbTiO 3 , with results showing that, with just 20% of the area sampled, most high-response clusters were captured. Furthermore, this approach can allow performing more complex spectroscopies in SPM that were previously not possible due to time constraints and sample stability. Improvements to the framework to enable the incorporation of more prior information and improve efficiency further are modeled and discussed.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2024

Scaling and merging time-resolved pink-beam diffraction with variational inference

Time-resolved x-ray crystallography (TR-X) at synchrotrons and free electron lasers is a promising technique for recording dynamics of molecules at atomic resolution. While experimental methods for TR-X have proliferated and matured, data analysis is often difficult. Extracting small, time-dependent changes in signal is frequently a bottleneck for practitioners. Recent work demonstrated this challenge can be addressed when merging redundant observations by a statistical technique known as variational inference (VI). However, the variational approach to time-resolved data analysis requires identification of successful hyperparameters in order to optimally extract signal. In this case study, we present a successful application of VI to time-resolved changes in an enzyme, DJ-1, upon mixing with a substrate molecule, methylglyoxal. We present a strategy to extract high signal-to-noise changes in electron density from these data. Furthermore, we conduct an ablation study, in which we systematically remove one hyperparameter at a time to demonstrate the impact of each hyperparameter choice on the success of our model. We expect this case study will serve as a practical example for how others may deploy VI in order to analyze their time-resolved diffraction data.

47 OTHER INSTRUMENTATION↗
DOE OSTI2026

A Statistician’s Overview of Physics-Informed Neural Networks for Spatio-Temporal Data

The recent success of deep neural network models with physical constraints (so-called, Physics-Informed Neural Networks, PINNs) has led to renewed interest in the incorporation of mechanistic information in predictive models. Statisticians and others have long been interested in this problem, which has led to several practical and innovative solutions dating back decades. In this overview, we focus on the problem of data-driven prediction and inference of dynamic spatio-temporal processes that include mechanistic information, such as would be available from partial differential equations, with a strong focus on the quantification of uncertainty associated with data, process, and parameters. Here, we give a brief review of several paradigms and focus our attention on Bayesian implementations given they naturally accommodate uncertainty quantification. We then show that it is straight-forward to include the Bayesian PINN (B-PINN) within the Bayesian hierarchical model (BHM) framework that has long been considered for modeling dynamic spatio-temporal processes. Such a BHM-PINN is illustrated via a simulation study in which a latent nonlinear Burgers’ equation PDE governs the dynamics of Poisson distributed spatio-temporal data. Supplementary materials for this article are available online, including a standardized description of the materials available for reproducing the work.

Bayesian↗
DOE OSTI2017

A Bayesian framework for cosmic string searches in CMB maps

There exists various proposals to detect cosmic strings from Cosmic Microwave Background (CMB) or 21 cm temperature maps. Current proposals do not aim to find the location of strings on sky maps, all of these approaches can be thought of as a statistic on a sky map. We propose a Bayesian interpretation of cosmic string detection and within that framework, we derive a connection between estimates of cosmic string locations and cosmic string tension G μ. We use this Bayesian framework to develop a machine learning framework for detecting strings from sky maps and outline how to implement this framework with neural networks. The neural network we trained was able to detect and locate cosmic strings on noiseless CMB temperature map down to a string tension of G μ=5 ×10{sup −9} and when analyzing a CMB temperature map that does not contain strings, the neural network gives a 0.95 probability that G μ≤2.3×10{sup −9}.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗