Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Bayesian Monte Carlo”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 181 records · Page 10
All sourcesNASA NTRSDOE OSTI
DOE OSTI2022

SPARC: Structural properties associated with residue constraints

SPARC facilitates the generation of plausible hypotheses regarding underlying biochemical mechanisms by structurally characterizing protein sequence constraints. Such constraints appear as residues co-conserved in functionally related subgroups, as subtle pairwise correlations (i.e., direct couplings), and as correlations among these sequence features or with structural features. SPARC performs three types of analyses. First, based on pairwise sequence correlations, it estimates the biological relevance of alternative conformations and of homomeric contacts, as illustrated here for death domains. Second, it estimates the statistical significance of the correspondence between directly coupled residue pairs and interactions at heterodimeric interfaces. Third, given molecular dynamics simulated structures, it characterizes interactions among constrained residues or between such residues and ligands that: (a) are stably maintained during the simulation; (b) undergo correlated formation and/or disruption of interactions with other constrained residues; or (c) switch between alternative interactions. We illustrate this for two homohexameric complexes: the bacterial enhancer binding protein (bEBP) NtrC1, which activates transcription by remodeling RNA polymerase (RNAP) containing σ 54 , and for DnaB helicase, which opens DNA at the bacterial replication fork. Based on the NtrC1 analysis, we hypothesize possible mechanisms for inhibiting ATP hydrolysis until ADP is released from an adjacent subunit and for coupling ATP hydrolysis to restructuring of σ 54 binding loops. Based on the DnaB analysis, we hypothesize that DnaB ‘grabs’ ssDNA by flipping every fourth base and inserting it into cavities between subunits and that flipping of a DnaB-specific glutamine residue triggers ATP hydrolysis.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2023

Uncertainty quantification for high explosive reactant and product equations of state

Equations of state (EOSs) are typically represented as physics-informed models with tunable parameters that are adjusted to replicate calibration data as closely as possible. Uncertainty quantification (UQ) allows for the development of an ensemble of EOS parameters that are consistent with the calibration data instead of a single EOS. In this work, we perform UQ for the reactant and product EOSs for a variety of high explosives (HEs). In doing so, we demonstrate a strategy for dealing with heterogeneous (both experimental and calculated) data. We also use a statistical distance metric to quantify the differences between the various HEs using the UQ results.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
DOE OSTI2021

Bayesian learning of orthogonal embeddings for multi-fidelity Gaussian Processes

Uncertainty propagation in complex engineering systems often poses significant computational challenges related to modeling and quantifying probability distributions of model outputs, as those emerge as the result of various sources of uncertainty that are inherent in the system under investigation. Gaussian Processes regression (GPs) is a robust meta-modeling technique that allows for fast model prediction and exploration of response surfaces. Multi-fidelity variations of GPs further leverage information from cheap and low fidelity model simulations in order to improve their predictive performance on the high fidelity model. In order to cope with the high volume of data required to train GPs in high dimensional design spaces, a common practice is to introduce latent design variables that are typically projections of the original input space to a lower dimensional subspace, and therefore substitute the problem of learning the initial high dimensional mapping, with that of training a GP on a low dimensional space. Here in this paper, we present a Bayesian approach to identify optimal transformations that map the input points to low dimensional latent variables. The \projection" mapping consists of an orthonormal matrix that is considered a priori unknown and needs to be inferred jointly with the GP parameters, conditioned on the available training data. The proposed Bayesian inference scheme relies on a two-step iterative algorithm that samples from the marginal posteriors of the GP parameters and the projection matrix respectively, both using Markov Chain Monte Carlo (MCMC) sampling. In order to take into account the orthogonality constraints imposed on the orthonormal projection matrix, a Geodesic Monte Carlo sampling algorithm is employed, that is suitable for exploiting probability measures on manifolds. We extend the proposed framework to multi-fidelity models using GPs including the scenarios of training multiple outputs together. We validate our framework on three synthetic problems with a known lower-dimensional subspace. The benefits of our proposed framework, are illustrated on the computationally challenging aerodynamic optimization of a last-stage blade for an industrial gas turbine, where we study the effect of an 85-dimensional shape parameterization of a three-dimensional airfoil on two output quantities of interest, specifically on the aerodynamic efficiency and the degree of reaction

42 ENGINEERING↗
DOE OSTI2023

nautilus : boosting Bayesian importance nested sampling with deep learning

ABSTRACT We introduce a novel approach to boost the efficiency of the importance nested sampling (INS) technique for Bayesian posterior and evidence estimation using deep learning. Unlike rejection-based sampling methods such as vanilla nested sampling (NS) or Markov chain Monte Carlo (MCMC) algorithms, importance sampling techniques can use all likelihood evaluations for posterior and evidence estimation. However, for efficient importance sampling, one needs proposal distributions that closely mimic the posterior distributions. We show how to combine INS with deep learning via neural network regression to accomplish this task. We also introduce nautilus, a reference open-source python implementation of this technique for Bayesian posterior and evidence estimation. We compare nautilus against popular NS and MCMC packages, including emcee, dynesty, ultranest, and pocomc, on a variety of challenging synthetic problems and real-world applications in exoplanet detection, galaxy SED fitting and cosmology. In all applications, the sampling efficiency of nautilus is substantially higher than that of all other samplers, often by more than an order of magnitude. Simultaneously, nautilus delivers highly accurate results and needs fewer likelihood evaluations than all other samplers tested. We also show that nautilus has good scaling with the dimensionality of the likelihood and is easily parallelizable to many CPUs.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2022

GIGA-Lens: Fast Bayesian Inference for Strong Gravitational Lens Modeling

We present GIGA-Lens: a gradient-informed, GPU-accelerated Bayesian framework for modeling strong gravitational lensing systems, implemented in TensorFlow and JAX. The three components, optimization using multistart gradient descent, posterior covariance estimation with variational inference, and sampling via Hamiltonian Monte Carlo, all take advantage of gradient information through automatic differentiation and massive parallelization on graphics processing units (GPUs). We test our pipeline on a large set of simulated systems and demonstrate in detail its high level of performance. The average time to model a single system on four Nvidia A100 GPUs is 105 s. The robustness, speed, and scalability offered by this framework make it possible to model the large number of strong lenses found in current surveys and present a very promising prospect for the modeling of ${ \mathcal O }({10}^{5})$ lensing systems expected to be discovered in the era of the Vera C. Rubin Observatory, Euclid, and the Nancy Grace Roman Space Telescope.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2025

Detecting outbreaks using a spatial latent field

In this paper, we present a method for estimating the infection-rate of a disease as a spatial-temporal field. Our data comprises time-series case-counts of symptomatic patients in various areal units of a region. We extend an epidemiological model, originally designed for a single areal unit, to accommodate multiple units. The field estimation is framed within a Bayesian context, utilizing a parameterized Gaussian random field as a spatial prior. We apply an adaptive Markov chain Monte Carlo method to sample the posterior distribution of the model parameters condition on COVID-19 case-count data from three adjacent counties in New Mexico, USA. Our results suggest that the correlation between epidemiological dynamics in neighboring regions helps regularize estimations in areas with high variance (i.e., poor quality) data. Using the calibrated epidemic model, we forecast the infection-rate over each areal unit and develop a simple anomaly detector to signal new epidemic waves. Our findings show that anomaly detector based on estimated infection-rates outperforms a conventional algorithm that relies solely on case-counts.

Safta, Cosmin [Sandia National Laboratories (SNL-C↗
DOE OSTI2021

An adaptive Hessian approximated stochastic gradient MCMC method

Bayesian approaches have been successfully integrated into training deep neural networks. One popular family is stochastic gradient Markov chain Monte Carlo methods (SG-MCMC), which have gained increasing interest due to their ability to handle large datasets and the potential to avoid overfitting. Although standard SG-MCMC methods have shown great performance in a variety of problems, they may be inefficient when the random variables in the target posterior densities have scale differences or are highly correlated. Here, we present an adaptive Hessian approximated stochastic gradient MCMC method to incorporate local geometric information while sampling from the posterior. The idea is to apply stochastic approximation (SA) to sequentially update a preconditioning matrix at each iteration. The preconditioner possesses second-order information and can guide the random walk of a sampler efficiently. Instead of computing and saving the full Hessian of the log posterior, we use limited memory of the samples and their stochastic gradients to approximate the inverse Hessian-vector multiplication in the updating formula. Moreover, by smoothly optimizing the preconditioning matrix via SA, our proposed algorithm can asymptotically converge to the target distribution with a controllable bias under mild conditions. To reduce the training and testing computational burden, we adopt a magnitude-based weight pruning method to enforce the sparsity of the network. Our method is user-friendly and demonstrates better learning results compared to standard SG-MCMC updating rules. The approximation of inverse Hessian alleviates storage and computational complexities for large dimensional models. Numerical experiments are performed on several problems, including sampling from 2D correlated distribution, synthetic regression problems, and learning the numerical solutions of heterogeneous elliptic PDE. The numerical results demonstrate great improvement in both the convergence rate and accuracy.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2024

Statistical data analysis of x-ray spectroscopy data enabled by neural network accelerated Bayesian inference

Bayesian inference applied to x-ray spectroscopy data analysis enables uncertainty quantification necessary to rigorously test theoretical models. However, when comparing to data, detailed atomic physics and radiation transfer calculations of x-ray emission from non-uniform plasma conditions are typically too slow to be performed in line with statistical sampling methods, such as Markov Chain Monte Carlo sampling. Furthermore, differences in transition energies and x-ray opacities often make direct comparisons between simulated and measured spectra unreliable. Here, we present a spectral decomposition method that allows for corrections to line positions and bound–bound opacities to best fit experimental data, with the goal of providing quantitative feedback to improve the underlying theoretical models and guide future experiments. In this work, we use a neural network (NN) surrogate model to replace spectral calculations of isobaric hot-spots created in Kr-doped implosions at the National Ignition Facility. The NN was trained on calculations of x-ray spectra using an isobaric hot-spot model post-processed with Cretin, a multi-species atomic kinetics and radiation code. The speedup provided by the NN model to generate x-ray emission spectra enables statistical analysis of parameterized models with sufficient detail to accurately represent the physical system and extract the plasma parameters of interest.

47 OTHER INSTRUMENTATION↗
DOE OSTI2021

Robust Importance Sampling for Bayesian Model Calibration with Spatio-Temporal Data

This paper addresses two challenges in Bayesian calibration: 1) computational speed of existing sampling algorithms, and 2) calibration with spatio-temporal responses. The commonly used Markov Chain Monte Carlo (MCMC) approaches require many sequential model evaluations making the computational expense prohibitive. This paper proposes an efficient sampling algorithm: iterative importance sampling with genetic algorithm (IISGA). While iterative importance sampling enables computational efficiency, the genetic algorithm enables robustness by preventing sample degeneration and avoids getting stuck in multimodal search spaces. An inflated likelihood further enables robustness in high-dimensional parameter spaces by enlarging the target distribution. Spatio-temporal data complicate both surrogate modeling, which is necessary for expensive computational models, and the likelihood estimation. In this work, singular value decomposition is investigated for reducing the high-dimensional field data to a lower-dimensional space prior to Bayesian calibration. Then the likelihood is formulated and Bayesian inference is performed in the lower-dimension, latent space. An illustrative example is provided to demonstrate IISGA relative to existing sampling methods, and then IISGA is employed to calibrate a thermal battery model with 26 uncertain calibration parameters and spatio-temporal response data.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2023

Stochastic modeling and statistical calibration with model error and scarce data

This paper introduces a procedure to assess the predictive accuracy of stochastic models subject to model error and sparse data. Model error is introduced as uncertainty on the coefficients of appropriate polynomial chaos expansions (PCE). The error associated with finite sample size allows us to conceive of these coefficients as statistics of the data that we describe as random variables whose influence on output quantities of interest is evaluated through the extended polynomial chaos expansion (EPCE). A Bayesian data assimilation scheme is introduced to update these expansions by considering the resulting nested chaos expansion as a hierarchical probabilistic model. Stochastic models of quantities of interest (QoI) are thus constructed and efficiently evaluated. Here, the Metropolis–Hastings Markov chain Monte Carlo procedure is used to sample the posterior. Two illustrative analytical and numerical problems are used to demonstrate the proposed approach.

Bayesian inference↗
DOE OSTI2024

Online MCMC Thinning with Kernelized Stein Discrepancy

A fundamental challenge in Bayesian inference is efficient representation of a target distribution. Many nonparametric approaches do so by sampling a large number of points using variants of Markov chain Monte Carlo (MCMC). Here, we propose an MCMC variant that retains only those posterior samples which exceed a kernelized Stein discrepancy (KSD) threshold, which we call KSD thinning. We establish the convergence and complexity trade-offs for several settings of KSD thinning as a function of the KSD threshold parameter, sample size, and other problem parameters. We provide experimental comparisons against other online nonparametric Bayesian methods that generate low-complexity posterior representations. We observe superior consistency/complexity trade-offs across a range of settings including MCMC sampling on two Bayesian inference problems from the biological sciences, and 10 × inference speedup and storage reduction for Bayesian neural networks with no loss of accuracy and no increase in training time. Our code is available at https://github.com/colehawkins/KSD-Thinning.

Bayesian inference↗
DOE OSTI2020

TBASS: A Robust Adaptation of Bayesian Adaptive Spline Surfaces

The R package TBASS is an extension of the BASS package created by Francom and Sansó (2019). The package is used to fit a Bayesian multivariate adaptive spline to a dataset that either follows a Student’s t-distribution or has outliers. Much of the framework for TBASS is adapted from the concepts of Bayesian Multivariate Adaptive Regression Splines (BMARS), specifically the work done by Denison, Mallick, and Smith (1998). The spline function is fit using a Reversible-Jump Markov Chain Monte Carlo algorithm,. By including this more robust generalization, a dataset with outliers can be accurately fit using the BMARS model, without the possibility of overfitting or variance inflation.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2025

Bayesian prior construction for uncertainty quantification in first-principles statistical mechanics

First-principles statistical mechanics enables the prediction of thermodynamic and kinetic properties of materials, but is computationally expensive. Many approaches require surrogate models to calculate energies within Monte Carlo or molecular dynamics simulations. Inexpensive surrogates such as cluster expansions enable otherwise intractable calculations by interpolating data from higher accuracy methods, such as Density Functional Theory (DFT). Surrogate models introduce uncertainty into downstream calculations, in addition to any uncertainty inherent to DFT calculations. Bayesian frameworks address this by quantifying uncertainty and incorporating expert knowledge through priors. However, constructing effective priors remains challenging. This work introduces and describes practical strategies for building Bayesian cluster expansions, focusing on basis truncation, hyperparameter selection, and ground state replication. We analyze multiple basis truncation schemes, compare cross-validation to the evidence-approximation for hyperparameter optimization, and provide methods to find and enforce ground-state-preserving models through priors. Additionally, we compare the uncertainties between different approximations to DFT (LDA, PBE, SCAN) against the uncertainty introduced with the use of cluster expansion surrogate models. These approaches are demonstrated on the BCC Li x Mg 1-x and Li x Al 1-x alloys, which are both of interest for solid-state Li batteries. Our results provide guidelines for constructing and utilizing Bayesian cluster expansions, thereby improving the transparency of materials modeling. Furthermore, the approaches and insights developed in this work can be transferred to a wide range of cluster expansion surrogate models, including the atomic cluster expansion and related machine-learned interatomic potential architectures.

Alloy theory↗
DOE OSTI2025

Emulator-based Bayesian calibration of a subglacial drainage model

Subglacial drainage models, often motivated by the relationship between hydrology and ice flow, sensitively depend on numerous unconstrained parameters. We explore using borehole water-pressure time series to calibrate the uncertain parameters of a popular subglacial drainage model, taking a Bayesian perspective to quantify the uncertainty in parameter estimates and in the calibrated model predictions. To reduce the computation time associated with Markov Chain Monte Carlo sampling, we construct a fast Gaussian process emulator to stand in for the subglacial drainage model. We first carry out a calibration experiment using synthetic observations consisting of model simulations with hidden parameter values as a demonstration of the method. Using real borehole water pressures measured in western Greenland, we find meaningful constraints on four of the eight model parameters and a factor-of-three reduction in uncertainty of the calibrated model predictions. These experiments illustrate Gaussian process-based Bayesian inference as a useful tool for calibration and uncertainty quantification of complex glaciological models using field data. However, significant differences between the calibrated model and the borehole data suggest that structural limitations of the model, rather than poorly constrained parameters or computational cost, remain the most important constraint on subglacial drainage modelling.

58 GEOSCIENCES↗
DOE OSTI2020

Sequential ensemble transform for Bayesian inverse problems

In this work, we present the Sequential Ensemble Transform (SET) method, an approach for generating approximate samples from a Bayesian posterior distribution. The method explores the posterior distribution by solving a sequence of discrete optimal transport problems to produce a series of transport plans which map prior samples to posterior samples. We prove that the sequence of Dirac mixture distributions produced by the SET method converges weakly to the true posterior as the sample size approaches infinity. Furthermore, our numerical results indicate that, when compared to standard Sequential Monte Carlo (SMC) methods, the SET approach is more robust to the choice of Markov mutation kernels and requires less computational efforts to reach a similar accuracy when used to explore complex posterior distributions. Finally, we describe adaptive schemes that allow to completely automate the use of the SET method.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
DOE OSTI2020

Fast matrix algebra for Bayesian model calibration

In Bayesian model calibration, evaluation of the likelihood function usually involves finding the inverse and determinant of a covariance matrix. When Markov Chain Monte Carlo (MCMC) methods are used to sample from the posterior, hundreds of thousands of likelihood evaluations may be required. In this paper, we demonstrate that the structure of the covariance matrix can be exploited, leading to substantial time savings in practice. Here, we also derive two simple equations for approximating the inverse of the covariance matrix in this setting, which can be computed in near-quadratic time. The practical implications of these strategies are demonstrated using a simple numerical case study and the "quack" R package. For a covariance matrix with 1000 rows, application of these strategies for a million likelihood evaluations leads to a speedup of roughly 4000 compared to the naive implementation

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2022

Marginal unbiased score expansion and application to CMB lensing

Here, we present the marginal unbiased score expansion (MUSE) method, an algorithm for generic high-dimensional hierarchical Bayesian inference. MUSE performs approximate marginalization over arbitrary non-Gaussian latent parameter spaces, yielding Gaussianized asymptotically unbiased and near-optimal constraints on global parameters of interest. It is computationally much cheaper than exact alternatives like Hamiltonian Monte Carlo (HMC), excelling on funnel problems which challenge HMC, and does not require any problem-specific user supervision like other approximate methods such as variational inference or many simulation-based inference methods. MUSE makes possible the first joint Bayesian estimation of the delensed Cosmic Microwave Background (CMB) power spectrum and gravitational lensing potential power spectrum, demonstrated here on a simulated data set as large as the upcoming South Pole Telescope 3G 1500 deg 2 survey, corresponding to a latent dimensionality of ~6 million and of order 100 global bandpower parameters. On a subset of the problem where an exact but more expensive HMC solution is feasible, we verify that MUSE yields nearly optimal results. We also demonstrate that existing spectrum-based forecasting tools which ignore pixel-masking underestimate predicted error bars by only ~10%. This method is a promising path forward for fast lensing and delensing analyses which will be necessary for future CMB experiments such as SPT-3G, Simons Observatory, or CMB-S4, and can complement or supersede existing HMC approaches. The success of MUSE on this challenging problem strengthens its case as a generic procedure for a broad class of high-dimensional inference problems.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗
DOE OSTI2021

A Bayesian Approach to the Eagar–Tsai Model for Melt Pool Geometry Prediction with Implications in Additive Manufacturing of Metals

Here, this paper focuses on improving the melt pool geometry predictions and quantifying uncertainties using an adapted version of the Eagar–Tsai (E–T) model that incorporates temperature-dependent properties of the material as well as powder conditions. Additionally, Bayesian inference is employed to predict distributions for the E–T model input parameters of laser absorptivity and powder bed porosity by incorporating experimental results into the analysis. Monte Carlo uncertainty propagation is then used with these parameter distributions to estimate the melt pool depth and associated uncertainty. Our results for the 316L stainless steel suggest that both the absorptivity and powder bed porosity are strongly influenced by the laser power. In contrast, the scanning speed has only a marginal effect on both the absorptivity and powder bed porosity. We constructed a printability map using the Bayesian E–T model based on power-dependent input parameter values to demonstrate the merit of the approach. The Bayesian approach improved the accuracy in predicting the keyhole regions in the laser power-scan speed parameter space for the 316L stainless steel. Although applied to a specific adaptation of the E–T model, the method put forth can be extended to quantify uncertainties in other numerical models as well as in the estimation of unknown parameters.

316L Stainless Steel (SS).↗