Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Variational Bayesian”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 145 records · Page 8

Robust Optimal Experimental Design of Infinite-Dimensional Bayesian Nonlinear Inverse Problems

Abstract. We consider robust optimal experimental design (ROED) for nonlinear Bayesian inverse problems governed by partial differential equations (PDEs). An optimal design is one that maximizes some utility quantifying the quality of the solution of an inverse problem. However, the optimal design is dependent on elements of the inverse problem such as the simulation model, the prior, or the measurement error model. ROED aims to produce an optimal design that is aware of the additional uncertainties encoded in the inverse problem and remains optimal even after variations in them. We follow a worst-case scenario approach to develop a new framework for robust optimal design of nonlinear Bayesian inverse problems. The proposed framework (a) is scalable and designed for infinite-dimensional Bayesian nonlinear inverse problems constrained by PDEs; (b) develops efficient approximations of the utility, namely the expected information gain; (c) employs eigenvalue sensitivity techniques to develop analytical forms and efficient evaluation methods of the gradient of the utility with respect to the uncertainties against which we wish to be robust; and (d) employs a probabilistic optimization paradigm that properly defines and efficiently solves the resulting combinatorial max-min optimization problem. The effectiveness of the proposed approach is illustrated for optimal sensor placement problem in an inverse problem governed by an elliptic PDE.

Chowdhary, Abhijit↗

Sensitivity of modelled passive margin stratigraphy to variations in sea level, sediment supply and subsidence

Abstract We produced a 10 Myr synthetic stratigraphic section using a forward stratigraphic model that generates marine deltaic stratigraphy over geological timescales. We recursively fit the model using a Bayesian inversion algorithm to test: (1) if it could be accurately reconstructed; (2) if the parameters used to create it could be recovered; and (3) the sensitivity of the model output to given model parameters and the attendant physical processes. The original synthetic stratigraphic section was produced with cyclical sea‐level variations of 40 and 30 m with 2.4 and 10 Myr periods respectively. Sediment was also supplied cyclically, in 2.4 and 10 Myr cycles with amplitudes of 30 and 80 tons/100 kyr, respectively, varying from a mean of 232 tons/100 kyr. Parameter values were sampled to fit the model using a Markov chain Monte Carlo algorithm, resulting in a ±5 m (1σ) variation between the experimental output and the original. Sea level varied by ±7 m (1σ) within the posterior distribution of parameters. As a result, both the 10 Myr and 2.4 Myr sea‐level cycles could be extracted from the original output. The variation in sediment supply was approximately ±38 tons/100 kyr (1σ) and, as a result, only the larger long‐term supply variations could be accurately recovered in refitting the model. The variation in thermal, flexural and total subsidence across those parameter sets is less than ±10 m (1σ). The original section experienced 150 m of total subsidence at the depocentre. Our results demonstrate the distinct and interpretable imprint of sea level and subsidence on continental margin stratigraphy can be quantified. Moreover, we conclude that sea‐level change produces a defined effect on the geometries of stratigraphic architecture, and that techniques applied for the purpose of delineating sea‐level variation from continental margin strata have a well‐founded conceptual basis.

Schmelz, William J.↗

Sequential Image Recovery Using Joint Hierarchical Bayesian Learning

Abstract Recovering temporal image sequences (videos) based on indirect, noisy, or incomplete data is an essential yet challenging task. We specifically consider the case where each data set is missing vital information, which prevents the accurate recovery of the individual images. Although some recent (variational) methods have demonstrated high-resolution image recovery based on jointly recovering sequential images, there remain robustness issues due to parameter tuning and restrictions on the type of sequential images. Here, we present a method based on hierarchical Bayesian learning for the joint recovery of sequential images that incorporates prior intra- and inter-image information. Our method restores the missing information in each image by “borrowing” it from the other images. More precisely, we couple sequential images by penalizing their pixel-wise difference. The corresponding penalty terms (one for each pixel and pair of subsequent images) are treated as weakly-informative random variables that favor small pixel-wise differences but allow occasional outliers. As a result, all of the individual reconstructions yield improved accuracy. Our method can be used for various data acquisitions and allows for uncertainty quantification. Some preliminary results indicate its potential use for sequential deblurring and magnetic resonance imaging.

Xiao, Yao↗

Bayesian Poroelastic Aquifer Characterization From InSAR Surface Deformation Data. Part I: Maximum A Posteriori Estimate

Characterizing the properties of groundwater aquifers is essential for predicting aquifer response and managing groundwater resources. In this work, we develop a high-dimensional scalable Bayesian inversion framework governed by a three-dimensional quasi-static linear poroelastic model to characterize lateral permeability variations in groundwater aquifers. In this work, we determine the maximum a posteriori (MAP) point of the posterior permeability distribution from centimeter-level surface deformation measurements obtained from Interferometric Synthetic Aperture Radar (InSAR). The scalability of our method to high parameter dimension is achieved through the use of adjoint-based derivatives, inexact Newton methods to determine the MAP point, and a Mat´ern class sparse prior precision operator. Together, these guarantee that the MAP point is found at a cost, measured in number of forward/adjoint poroelasticity solves, that is independent of the parameter dimension. We apply our methodology to a test case for a municipal well in Mesquite, Nevada, in which InSAR and GPS surface deformation data are available. We solve problems with up to 320,824 state variable degrees of freedom (DOFs) and 16,896 parameter DOFs. A consistent treatment of noise level is employed so that the aquifer characterization result does not depend on the pixel spacing of surface deformation data. Our results show that the use of InSAR data significantly improves characterization of lateral aquifer heterogeneity, and the InSAR-based aquifer characterization recovers complex lateral displacement trends observed by independent daily GPS measurements.

54 ENVIRONMENTAL SCIENCES↗

A Statistical Evaluation of Rules for Biochemical Failure After Radiotherapy in Men Treated for Prostate Cancer

Purpose: The 'PSA nadir + 2 rule,' defined as any rise of 2 ng/ml above the current prostate-specific antigen (PSA) nadir, has replaced the American Society for Therapeutic Radiology and Oncology (ASTRO) rule, defined as three consecutive PSA rises, to indicate biochemical failure (BF) after radiotherapy in patients treated for prostate cancer. We propose an original approach to evaluate BF rules based on the PSAdt as the gold standard rule and on a simulation process allowing us to evaluate the BF rules under multiple settings (different frequency, duration of follow-up, PSA doubling time [PSAdt]). Methods and Materials: We relied on a retrospective, population-based cohort of individuals identified by the Connecticut Tumor Registry and treated for localized prostate cancer with radiotherapy. We estimated the 470 underlying true PSA trajectories, including the PSAdt, using a Bayesian hierarchical changepoint model. Next, we simulated realistic, sophisticated data sets that accurately reflect the systematic and random variations observed in PSA series. We estimated the sensitivity and specificity by comparing the simulated PSA series to the underlying true PSAdt. Results: For follow-up of more than 3 years, the specificity of the PSA nadir + 2 rule was systematically greater than that of the ASTRO criterion. In few settings, the nadir + 2 rule had a lower sensitivity than the ASTRO. The PSA nadir + 2 rule appeared less dependent on the frequency and duration of follow-up than the ASTRO. Conclusions: Our results provide some refinements to earlier findings as the BF rules were evaluated according to various parameters. In most settings, the PSA nadir + 2 rule outperforms the ASTRO criterion.

62 RADIOLOGY AND NUCLEAR MEDICINE↗

Bayesian analysis of energy and count rate data for detection of low count rate radioactive sources

We propose a radiation detection system which generates its own discrete sampling distribution based on past measurements of background. The advantage to this approach is that it can take into account variations in background with respect to time, location, energy spectra, detector-specific characteristics (i.e. different efficiencies at different count rates and energies), etc. This would therefore be a 'machine learning' approach, in which the algorithm updates and improves its characterization of background over time. The system would have a 'learning mode,' in which it measures and analyzes background count rates, and a 'detection mode,' in which it compares measurements from an unknown source against its unique background distribution. By characterizing and accounting for variations in the background, general purpose radiation detectors can be improved with little or no increase in cost. The statistical and computational techniques to perform this kind of analysis have already been developed. The necessary signal analysis can be accomplished using existing Bayesian algorithms which account for multiple channels, multiple detectors, and multiple time intervals. Furthermore, Bayesian machine-learning techniques have already been developed which, with trivial modifications, can generate appropriate decision thresholds based on the comparison of new measurements against a nonparametric sampling distribution. (authors)

97 MATHEMATICS AND COMPUTING↗

Preparation of the SU(3) lattice Yang-Mills vacuum with variational quantum methods

Studying QCD and other gauge theories on quantum hardware requires the preparation of physically interesting states. The variational quantum eigensolver provides a way of performing vacuum state preparation on quantum hardware. Here in this work, variational quantum eigensolver is applied to pure SU(3) lattice Yang-Mills on a single plaquette and one dimensional plaquette chains. Bayesian optimization and gradient descent were investigated for performing the classical optimization. Ansatz states for plaquette chains are constructed in a scalable manner from smaller systems using domain decomposition and a stitching procedure analogous to the density matrix renormalization group. Small examples are performed on IBM’s superconducting Manila processor.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗

Scalable and efficient learning from crowds with Gaussian processes

Over the last few years, multiply-annotated data has become a very popular source of information. Online platforms such as Amazon Mechanical Turk have revolutionized the labelling process needed for any classification task, sharing the effort between a number of annotators (instead of the classical single expert). This crowdsourcing approach has introduced new challenging problems, such as handling disagreements on the annotated samples or combining the unknown expertise of the annotators. Probabilistic methods, such as Gaussian Processes (GP), have proven successful to model this new crowdsourcing scenario. However, GPs do not scale up well with the training set size, which makes them prohibitive for medium-to-large datasets (beyond 10K training instances). This constitutes a serious limitation for current real-world applications. In this work, we introduce two scalable and efficient GP-based crowdsourcing methods that allow for processing previously-prohibitive datasets. The first one is an efficient and fast approximation to GP with squared exponential (SE) kernel. The second allows for learning a more flexible kernel at the expense of a heavier training (but still scalable to large datasets). Since the latter is not a GP-SE approximation, it can be also considered as a whole new scalable and efficient crowdsourcing method, useful for any dataset size. Both methods use Fourier features and variational inference, can predict the class of new samples, and estimate the expertise of the involved annotators. A complete experimentation compares them with state-of-the-art probabilistic approaches in synthetic and real crowdsourcing datasets of different sizes. Finally, they stand out as the best performing approach for large scale problems. Moreover, the second method is competitive with the current state-of-the-art for small datasets.

96 KNOWLEDGE MANAGEMENT AND PRESERVATION↗

mmbo (Multi-modal Bayesian Optimization) [SWR-25-119]

This package implements Bayesian Neural Network (BNN) based surrogate models for multi-modal data. These models are fit using a custom Variational Bayes procedure leveraging conjugate posteriors in the last layer of the network, offering more accurate predictions with better-calibrated uncertainty quantification than mean-field Variational Bayes.

Taylor, Ian [National Laboratory of the Rockies (N↗

Jensen–Shannon divergence based novel loss functions for Bayesian neural networks

Bayesian neural networks (BNNs) are state-of-the-art machine learning methods that can naturally regularize and systematically quantify uncertainties using their stochastic parameters. Kullback–Leibler (KL) divergence-based variational inference used in BNNs suffer from unstable optimization and challenges in approximating light-tailed posteriors due to the unbounded nature of the KL divergence. To resolve these issues, we formulate a novel loss function for BNNs based on a new modification to the generalized Jensen–Shannon (JS) divergence, which is bounded. In addition, we propose a Geometric JS divergence-based loss, which is computationally efficient since it can be evaluated analytically. We found that the JS divergence-based variational inference is intractable, and hence employed a constrained optimization framework to formulate these losses. Our theoretical analysis and empirical experiments on multiple regression and classification data sets suggest that the proposed losses perform better than the KL divergence-based loss, especially when the data sets are noisy or biased. Specifically, there are approximately 5% and 8% improvements in accuracy for a noise-added CIFAR-10 dataset and a regression dataset, respectively. There is about 13% reduction in false negative predictions of a biased histopathology dataset. Additionally, we quantify and compare the uncertainty metrics for the regression and classification tasks.

97 MATHEMATICS AND COMPUTING↗

Assessing the Influence of Microphysical and Environmental Parameter Perturbations on Orographic Precipitation

Microphysical (MP) schemes contain parameters whose values can impact the amount and location of forecasted precipitation, and sensitivity is typically explored by perturbing one parameter at a time while holding the rest constant. Although much can be learned from these “one-at-a-time” studies, the results are limited as these methods do not allow for nonlinear interactions of multiple perturbed parameters. This study applies the Morris one-at-a-time (MOAT) method, a robust statistical tool allowing for simultaneous perturbation of numerous parameters, to explore orographic precipitation sensitivity to changes in microphysical and environmental parameters within an environment characteristic of an atmospheric river. Results show parameters associated with snow fall speed coefficient A s and density ρ s , ice-cloud water collection efficiency (ECI), rain accretion (WRA), relative humidity, zonal wind speed, and surface potential temperature cause the largest influence on simulated precipitation. MP and environmental parameter perturbations can cause precipitation changes of similar magnitude, but results vary by location on the mountain. Different environments are also tested, with A s being the most influential MP parameter regardless of environment. Fewer MP parameters influence precipitation in a faster-wind-speed environment, possibly due to the stronger dynamical forcing upwind and different wave dynamics downwind, compared to a slower-wind-speed environment. Finally, perturbing MP parameters within a single scheme can result in precipitation variations of similar magnitude compared to using entirely different microphysics schemes. MOAT results presented in this study have implications for Bayesian parameter estimation methods and stochastic parameterization within ensemble forecasting.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

A Bayesian Measurment Error Model for Misaligned Radiographic Data

An understanding of the inherent variability in micro-computed tomography (micro-CT) data is essential to tasks such as statistical process control and the validation of radiographic simulation tools. The data present unique challenges to variability analysis due to the relatively low resolution of radiographs, and also due to minor variations from run to run which can result in misalignment or magnification changes between repeated measurements of a sample. Positioning changes artificially inflate the variability of the data in ways that mask true physical phenomena. We present a novel Bayesian nonparametric regression model that incorporates both additive and multiplicative measurement error in addition to heteroscedasticity to address this problem. We also use this model to assess the effects of sample thickness and sample position on measurement variability for an aluminum specimen. Supplementary materials for this article are available online.

97 MATHEMATICS AND COMPUTING↗

A non-stationary image prior combination in super-resolution

Here, a new Bayesian Super-Resolution(SR) image registration and reconstruction method is proposed. The new method utilizes a prior distribution based on a general combination of spatially adaptive, or non-stationary, image filters, which includes an adaptive local strength parameter able to preserve both image edges and textures. With the application of variational techniques, the proposed method allows for the automatic estimation of all problem unknowns. An experimental comparison between state of the art methods and the proposed SR approach has been performed on both synthetic and real images.

97 MATHEMATICS AND COMPUTING↗

Analyzing degradation data with a random effects spline regression model

This study proposes using a random effects spline regression model to analyze degradation data. Spline regression avoids having to specify a parametric function for the true degradation of an item. A distribution for the spline regression coefficients captures the variation of the true degradation curves from item to item. We illustrate the proposed methodology with a real example using a Bayesian approach. The Bayesian approach allows prediction of degradation of a population over time and estimation of reliability is easy to perform.

97 MATHEMATICS AND COMPUTING↗

Data-driven prediction of scaling and ignition of inertial confinement fusion experiments

Recent advances in inertial confinement fusion (ICF) at the National Ignition Facility (NIF), including ignition and energy gain, are enabled by a close coupling between experiments and high-fidelity simulations. Neither simulations nor experiments can fully constrain the behavior of ICF implosions on their own, meaning pre- and postshot simulation studies must incorporate experimental data to be reliable. Linking past data with simulations to make predictions for upcoming designs and quantifying the uncertainty in those predictions has been an ongoing challenge in ICF research. We have developed a data-driven approach to prediction and uncertainty quantification that combines large ensembles of simulations with Bayesian inference and deep learning. The approach builds a predictive model for the statistical distribution of key performance parameters, which is jointly informed by past experiments and physics simulations. The prediction distribution captures the impact of experimental uncertainty, expert priors, design changes, and shot-to-shot variations. We have used this new capability to predict a 10× increase in ignition probability between Hybrid-E shots driven with 2.05 MJ compared to 1.9 MJ, and validated our predictions against subsequent experiments. We describe our new Bayesian postshot and prediction capabilities, discuss their application to NIF ignition and validate the results, and finally investigate the impact of data sparsity on our prediction results.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗

Understanding the Scalability of Bayesian Network Inference Using Clique Tree Growth Curves

One of the main approaches to performing computation in Bayesian networks (BNs) is clique tree clustering and propagation. The clique tree approach consists of propagation in a clique tree compiled from a Bayesian network, and while it was introduced in the 1980s, there is still a lack of understanding of how clique tree computation time depends on variations in BN size and structure. In this article, we improve this understanding by developing an approach to characterizing clique tree growth as a function of parameters that can be computed in polynomial time from BNs, specifically: (i) the ratio of the number of a BN s non-root nodes to the number of root nodes, and (ii) the expected number of moral edges in their moral graphs. Analytically, we partition the set of cliques in a clique tree into different sets, and introduce a growth curve for the total size of each set. For the special case of bipartite BNs, there are two sets and two growth curves, a mixed clique growth curve and a root clique growth curve. In experiments, where random bipartite BNs generated using the BPART algorithm are studied, we systematically increase the out-degree of the root nodes in bipartite Bayesian networks, by increasing the number of leaf nodes. Surprisingly, root clique growth is well-approximated by Gompertz growth curves, an S-shaped family of curves that has previously been used to describe growth processes in biology, medicine, and neuroscience. We believe that this research improves the understanding of the scaling behavior of clique tree clustering for a certain class of Bayesian networks; presents an aid for trade-off studies of clique tree clustering using growth curves; and ultimately provides a foundation for benchmarking and developing improved BN inference and machine learning algorithms.

Mengshoel, Ole J.↗

Scalable posterior approximations for large-scale Bayesian inverse problems via likelihood-informed parameter and state reduction

Two prime bottlenecks to the solution of large-scale Bayesian inverse problems are the scaling of posterior sampling algorithms to high-dimensional parameter spaces and the computational cost of forward model evaluations. However, incomplete or noisy data, the state variation and parameter dependence of the forward model, and correlations in the prior collectively provide useful structure that can be exploited for dimension reduction in this setting—both in the parameter space of the inverse problem and in the state space of the forward model. To this end, we show how to jointly construct low-dimensional subspaces of the parameter space and the state space in order to accelerate the Bayesian solution of the inverse problem. As a byproduct of state dimension reduction, we also show how to identify low-dimensional subspaces of the data in problems with high-dimensional observations. These subspaces enable approximation of the posterior as a product of two factors: (i) a projection of the posterior onto a low-dimensional parameter subspace, wherein the original likelihood is replaced by an approximation involving a reduced model; and (ii) the marginal prior distribution on the high-dimensional complement of the parameter subspace. We introduce and compare several strategies for constructing these subspaces using only a limited number of forward and adjoint model simulations. The resulting posterior approximations can rapidly be characterized using standard sampling techniques, e.g., Markov chain Monte Carlo. Two numerical examples demonstrate the accuracy and efficiency of our approach: inversion of an integral equation in atmospheric remote sensing, where the data dimension is very high; and the inference of a heterogeneous transmissivity field in a groundwater system, which involves a partial differential equation forward model with high dimensional state and parameters.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗