Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Variational Bayesian”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 91 records · Page 5

Is Bayesian inference "brittle"?

In a recent report, “Bayesian Brittleness: Why no Bayesian model is ‘good enough,’” (arXiv, 1304.6772v1) the authors prove rigorously that model variations that are arbitrarily small, in a particular technical sense, the posterior expectation of a function can achieve essentially any value that the function alone can achieve, so that Bayesian inference appears to have no robustness whatsoever. We explain this puzzling result, and show why it does not imply a breakdown of Bayesian inference. The explanation is that the models leading to the extreme results depend on the observed data.

97 MATHEMATICS AND COMPUTING↗

Modeling Radiation Detection Using a Mixed-Effects Statistical Model for Background Variation

In this paper, we consider radiation source localization in an urban environment as an inverse problem and use Bayesian parameter estimation techniques to infer the source location and intensity. Since parameter estimation generally requires a large number of model evaluations, we employ a simplified photon transport model, as in a 2017 paper by Stefanescu, et al., to reduce computational requirements. However, the model in the 2017 paper does not account for the physical reality that background radiation varies with position within an urban locale. We incorporate statistical mixed-effects modeling into the simplified transport equation from the Stefanescu, et al. paper to reflect individual variation among the background radiation measurements at multiple detector locations. With our new model, we use a Bayesian DRAM-within-Gibbs technique, which incorporates the Delayed Rejection Adaptive Metropolis (DRAM) algorithm, to estimate the mixed-effects parameters. We note that without adequate prior information, we were unable to simultaneously estimate the set of parameters corresponding to the source intensity, source location, and individual background radiation counts. By calibrating the background in the absence of a source, we obtained narrow prior distributions, which permitted us to jointly infer the source intensity and location and the background radiation distribution.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗

Avian collision risk models for wind energy impact assessments

With the increasing global development of wind energy, collision risk models (CRMs) are routinely used to assess the potential impacts of wind turbines on birds. We reviewed and compared the avian collision risk models currently available in the scientific literature, exploring aspects such as the calculation of a collision probability, inclusion of stationary components e.g. the tower, angle of approach and uncertainty. 10 models were cited in the literature and of these, all included a probability of collision of a single bird colliding with a wind turbine during passage through the rotor swept area, and the majority included a measure of the number of birds at risk. 7 out of the 10 models calculated the probability of birds colliding, whilst the remainder used a constant. We identified four approaches to calculate the probability of collision and these were used by others. 6 of the 10 models were deterministic and included the most frequently used models in the UK, with only 4 including variation or uncertainty in some way, the most recent using Bayesian methods. Despite their appeal, CRMs have their limitations and can be ‘data hungry’ as well as assuming much about bird movement and behaviour. As data become available, these assumptions should be tested to ensure that CRMs are functioning to adequately answer the questions posed by the wind energy sector. - Highlights: • We highlighted ten models available to assess avian collision risk. • Only 4 of the models included variability or uncertainty. • Collision risk models have limitations and can be ‘data hungry’. • It is vital that the most appropriate model is used for a given task.

54 ENVIRONMENTAL SCIENCES↗

A unifying Bayesian framework for merging X-ray diffraction data

Novel X-ray methods are transforming the study of the functional dynamics of biomolecules. Key to this revolution is detection of often subtle conformational changes from diffraction data. Diffraction data contain patterns of bright spots known as reflections. To compute the electron density of a molecule, the intensity of each reflection must be estimated, and redundant observations reduced to consensus intensities. Systematic effects, however, lead to the measurement of equivalent reflections on different scales, corrupting observation of changes in electron density. Here, we present a modern Bayesian solution to this problem, which uses deep learning and variational inference to simultaneously rescale and merge reflection observations. We successfully apply this method to monochromatic and polychromatic single-crystal diffraction data, as well as serial femtosecond crystallography data. We find that this approach is applicable to the analysis of many types of diffraction experiments, while accurately and sensitively detecting subtle dynamics and anomalous scattering.

59 BASIC BIOLOGICAL SCIENCES↗

How to distinguish between cloudy mini-Neptunes and water/volatile-dominated super-Earths

One of the most profound questions about the newly discovered class of low-density super-Earths is whether these exoplanets are predominately H{sub 2}-dominated mini-Neptunes or volatile-rich worlds with gas envelopes dominated by H{sub 2}O, CO{sub 2}, CO, CH{sub 4}, or N{sub 2}. Transit observations of the super-Earth GJ 1214b rule out cloud-free H{sub 2}-dominated scenarios, but are not able to determine whether the lack of deep spectral features is due to high-altitude clouds or the presence of a high mean molecular mass atmosphere. Here, we demonstrate that one can unambiguously distinguish between cloudy mini-Neptunes and volatile-dominated worlds based on wing steepness and relative depths of absorption features in moderate-resolution near-infrared transmission spectra (R ∼ 100). In a numerical retrieval study, we show for GJ 1214b that an unambiguous distinction between a cloudy H{sub 2}-dominated atmosphere and cloud-free H{sub 2}O atmosphere will be possible if the uncertainties in the spectral transit depth measurements can be reduced by a factor of ∼3 compared to the published Hubble Space Telescope Wide-Field Camera 3 and Very Large Telescope transit observations by Berta et al. and Bean et al. We argue that the required precision for the distinction may be achievable with currently available instrumentation by stacking 10-15 repeated transit observations. We provide a scaling law that scales our quantitative results to other transiting super-Earths and Neptunes such as HD 97658b, 55 Cnc e, GJ 3470b and GJ 436b. The analysis in this work is performed using an improved version of our Bayesian atmospheric retrieval framework. The new framework not only constrains the gas composition and cloud/haze parameters, but also determines our confidence in having detected molecules and cloud/haze species through Bayesian model comparison. Using the Bayesian tool, we demonstrate quantitatively that the subtle transit depth variation in the Berta et al. data is not sufficient to claim the detection of water absorption.

79 ASTRONOMY AND ASTROPHYSICS↗

Efficient high-dimensional variational data assimilation with machine-learned reduced-order models

Abstract. Data assimilation (DA) in geophysical sciences remains the cornerstone of robust forecasts from numerical models. Indeed, DA plays a crucial role in the quality of numerical weather prediction and is a crucial building block that has allowed dramatic improvements in weather forecasting over the past few decades. DA is commonly framed in a variational setting, where one solves an optimization problem within a Bayesian formulation using raw model forecasts as a prior and observations as likelihood. This leads to a DA objective function that needs to be minimized, where the decision variables are the initial conditions specified to the model. In traditional DA, the forward model is numerically and computationally expensive. Here we replace the forward model with a low-dimensional, data-driven, and differentiable emulator. Consequently, gradients of our DA objective function with respect to the decision variables are obtained rapidly via automatic differentiation. We demonstrate our approach by performing an emulator-assisted DA forecast of geopotential height. Our results indicate that emulator-assisted DA is faster than traditional equation-based DA forecasts by 4 orders of magnitude, allowing computations to be performed on a workstation rather than a dedicated high-performance computer. In addition, we describe accuracy benefits of emulator-assisted DA when compared to simply using the emulator for forecasting (i.e., without DA). Our overall formulation is denoted AIEADA (Artificial Intelligence Emulator-Assisted Data Assimilation).

58 GEOSCIENCES↗

Unbinned extraction of $γ$ from $B\to DK$ with normalizing flows

We introduce an unbinned method for extracting the CKM angle $γ$ from the decay chain $B^\pm \to (D \to K_S π^+ π^-) K^\pm$ using normalizing flows (NFs). The NFs, trained on $D$ decay data, learn a faithful continuous representation of the amplitude and strong phase variation over the $D\to K_Sπ^+π^-$ Dalitz plot whose fidelity improves with increased data sample sizes. With this input, the $B$ decay data can be used to extract the parameters $r_B$, $δ_B$, and $γ$. We test the method on Monte Carlo generated data, where it successfully recovers the injected value of $γ$ within uncertainties. The present implementation propagates statistical uncertainties from finite training data via an ensemble of independently trained flows, and does not attempt to capture the effects of systematic experimental errors. We explore two versions of the method that differ in how the trigonometric constraint on phase variation is encoded, and comment on the possible extension to Bayesian NFs, which would provide direct uncertainty estimates on the learned densities without requiring ensemble training.

Grossman, Yuval [Cornell U., LEPP]↗

A Variational Inference Approach to Inverse Problems with Gamma Hyperpriors

Hierarchical models with gamma hyperpriors provide a flexible, sparse-promoting framework to bridge L 1 and L 2 regularizations in Bayesian formulations to inverse problems. Despite the Bayesian motivation for these models, existing methodologies are limited to maximum a posteriori estimation. The potential to perform uncertainty quantification has not yet been realized. This paper introduces a variational iterative alternating scheme for hierarchical inverse problems with gamma hyperpriors. The proposed variational inference approach yields accurate reconstruction, provides meaningful uncertainty quantification, and is easy to implement. In addition, it lends itself naturally to conduct model selection for the choice of hyperparameters. Here, we illustrate the performance of our methodology in several computed examples, including a deconvolution problem and sparse identification of dynamical systems from time series data.

Bayesian shrinkage↗

B-PINNs: Bayesian Physics-informal Neural Networks for Forward and Inverse PDE Problems with Noisy Data

We propose a Bayesian physics-informed neural network (B-PINN) to solve both forward and inverse nonlinear problems described by partial differential equations (PDEs) and noisy data. In this Bayesian framework, the Bayesian neural network (BNN) combined with a PINN for PDEs serves as the prior while the Hamiltonian Monte Carlo (HMC) or the variational inference (VI) could serve as an estimator of the posterior. B-PINNs make use of both physical laws and scattered noisy measurements to provide predictions and quantify the aleatoric uncertainty arising from the noisy data in the Bayesian framework. Compared with PINNs, in addition to uncertainty quantification, B-PINNs obtain more accurate predictions in scenarios with large noise due to their capability of avoiding overfitting. We conduct a systematic comparison between the two different approaches for the B-PINNs posterior estimation (i.e., HMC or VI), along with dropout used for quantifying uncertainty in deep neural networks. Our experiments show that HMC is more suitable than VI with mean field Gaussian approximation for the B-PINNs posterior estimation, while dropout employed in PINNs can hardly provide accurate predictions with reasonable uncertainty. Finally, we replace the BNN in the prior with a truncated Karhunen-Loève (KL) expansion combined with HMC or a deep normalizing flow (DNF) model as posterior estimators. The KL is as accurate as BNN and much faster but this framework cannot be easily extended to high-dimensional problems unlike the BNN based framework.

Non linear PDEs, Noisy data, Bysian physics inform↗

TYPE Ia SUPERNOVA LIGHT-CURVE INFERENCE: HIERARCHICAL BAYESIAN ANALYSIS IN THE NEAR-INFRARED

We present a comprehensive statistical analysis of the properties of Type Ia supernova (SN Ia) light curves in the near-infrared using recent data from Peters Automated InfraRed Imaging TELescope and the literature. We construct a hierarchical Bayesian framework, incorporating several uncertainties including photometric error, peculiar velocities, dust extinction, and intrinsic variations, for principled and coherent statistical inference. SN Ia light-curve inferences are drawn from the global posterior probability of parameters describing both individual supernovae and the population conditioned on the entire SN Ia NIR data set. The logical structure of the hierarchical model is represented by a directed acyclic graph. Fully Bayesian analysis of the model and data is enabled by an efficient Markov Chain Monte Carlo algorithm exploiting the conditional probabilistic structure using Gibbs sampling. We apply this framework to the JHK{sub s} SN Ia light-curve data. A new light-curve model captures the observed J-band light-curve shape variations. The marginal intrinsic variances in peak absolute magnitudes are sigma(M{sub J}) = 0.17 +- 0.03, sigma(M{sub H}) = 0.11 +- 0.03, and sigma(M{sub Ks}) = 0.19 +- 0.04. We describe the first quantitative evidence for correlations between the NIR absolute magnitudes and J-band light-curve shapes, and demonstrate their utility for distance estimation. The average residual in the Hubble diagram for the training set SNe at cz > 2000kms{sup -1} is 0.10 mag. The new application of bootstrap cross-validation to SN Ia light-curve inference tests the sensitivity of the statistical model fit to the finite sample and estimates the prediction error at 0.15 mag. These results demonstrate that SN Ia NIR light curves are as effective as corrected optical light curves, and, because they are less vulnerable to dust absorption, they have great potential as precise and accurate cosmological distance indicators.

79 ASTRONOMY AND ASTROPHYSICS↗

Soil moisture estimation using tomographic ground penetrating radar in a MCMC–Bayesian framework

In this paper, we focus on a hydrogeological inverse problem specifically targeting monitoring soil moisture variations using tomographic ground penetrating radar (GPR) travel time data. Technical challenges exist in the inversion of GPR tomographic data for handling non-uniqueness, nonlinearity and high-dimensionality of unknowns. We have developed a new method for estimating soil moisture fields from crosshole GPR data. It uses a pilot-point method to provide a low-dimensional representation of the relative dielectric permittivity field of the soil, which is the primary object of inference: the field can be converted to soil moisture using a petrophysical model. We integrate a multi-chain Markov chain Monte Carlo (MCMC)–Bayesian inversion framework with the pilot point concept, a curved-ray GPR travel time model, and a sequential Gaussian simulation algorithm, for estimating the dielectric permittivity at pilot point locations distributed within the tomogram, as well as the corresponding geostatistical parameters (i.e., spatial correlation range). We infer the dielectric permittivity as a probability density function, thus capturing the uncertainty in the inference. The multi-chain MCMC enables addressing high-dimensional inverse problems as required in the inversion setup. The method is scalable in terms of number of chains and processors, and is useful for computationally demanding Bayesian model calibration in scientific and engineering problems. The proposed inversion approach can successfully approximate the posterior density distributions of the pilot points, and capture the true values. The computational efficiency, accuracy, and convergence behaviors of the inversion approach were also systematically evaluated, by comparing the inversion results obtained with different levels of noises in the observations, increased observational data, as well as increased number of pilot points.

54 ENVIRONMENTAL SCIENCES↗

A neural networks framework for real-time unfolding of neutron spectroscopic data at JET

A determination of fast ion population parameters such as intensity and kinetic temperature is important for fusion reactors. This becomes more challenging with finer time resolution of the measurements, since the limited data in each time slice cause increasing statistical variations in the data. This paper describes a framework using Bayesian-regularized neural networks (NNs) designed for such a task. The method is applied to the TOFOR 2.5 MeV fusion neutron spectrometer at JET. NN training data are generated by random sampling of variables in neutron spectroscopy models. Ranges and probability distributions of the parameters are chosen to match the experimental data. Results have shown good performance both on synthetic and experimental data. The latter was assessed by statistical considerations and by examining the robustness and time consistency of the results. The regularization of the training algorithm allowed for higher time resolutions than simple forward methods. The fast execution time makes this approach suitable for real-time analysis with a time resolution limit in the microsecond time scale.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗

Adaptive mitigation of time-varying quantum noise

Current quantum computers suffer from non-stationary noise channels with high error rates, which undermines their reliability and reproducibility. We propose a Bayesian inference based adaptive algorithm that can learn and mitigate quantum noise in response to changing channel conditions. Our study emphasizes the need for dynamic inference of critical channel parameters to improve program accuracy. We use the Dirichlet distribution to model the stochasticity of the Pauli channel. This allows us to perform Bayesian inference, which can improve the performance of probabilistic error cancellation (PEC) under time-varying noise. Our work demonstrates the importance of characterizing and mitigating temporal variations in quantum noise, which is crucial for developing more accurate and reliable quantum technologies. Our results demonstrate that Bayesian PEC can outperform non-adaptive approaches by a factor of 4.5x when measured using Hellinger distance from the ideal distribution.

Dasgupta, Samudra↗

Aided Active Learning (AAL) for Enhanced Critical Heat Flux Prediction

Accurate prediction of critical heat flux (CHF) is crucial for the safe and efficient operation of nuclear reactors. Traditional CHF modeling methods often require extensive experimental data, which are hard to obtain. This study introduces the Aided Active Learning (AAL) framework, which strategically minimizes data requirements without sacrificing model accuracy. Unlike conventional Active Learning (AL), AAL introduces an additional step of randomly selecting a subset from the sample pool before applying the query strategy. To evaluate the performance of AAL, two query strategies—uncertainty-based sampling and error-reduction sampling—were evaluated across the following models: random forest (RF), feedforward neural network (FNN), and variational feedforward neural network (vFNN). The proposed framework demonstrated that AAL effectively reduces the number of training samples needed to achieve comparable predictive accuracy. For the RF model, AL required only 710 samples to achieve an R2 score of 0.98, as compared to the 4,785 samples needed by random sampling. Similarly, the FNN model achieved the same R2 score with just 355 samples when using AL, a significant improvement over the 825 samples required by random sampling. In case of uncertainty-based sampling strategy, vFNN attained an R2 of 0.98 with 3,420 samples, reducing the sample requirement by 47% relative to the 6,440 samples needed for random sampling. Its performance suggests that larger training data are required to fully leverage its uncertainty quantification capabilities.

22 - GENERAL STUDIES OF NUCLEAR REACTORS↗

Marginal unbiased score expansion and application to CMB lensing

Here, we present the marginal unbiased score expansion (MUSE) method, an algorithm for generic high-dimensional hierarchical Bayesian inference. MUSE performs approximate marginalization over arbitrary non-Gaussian latent parameter spaces, yielding Gaussianized asymptotically unbiased and near-optimal constraints on global parameters of interest. It is computationally much cheaper than exact alternatives like Hamiltonian Monte Carlo (HMC), excelling on funnel problems which challenge HMC, and does not require any problem-specific user supervision like other approximate methods such as variational inference or many simulation-based inference methods. MUSE makes possible the first joint Bayesian estimation of the delensed Cosmic Microwave Background (CMB) power spectrum and gravitational lensing potential power spectrum, demonstrated here on a simulated data set as large as the upcoming South Pole Telescope 3G 1500 deg 2 survey, corresponding to a latent dimensionality of ~6 million and of order 100 global bandpower parameters. On a subset of the problem where an exact but more expensive HMC solution is feasible, we verify that MUSE yields nearly optimal results. We also demonstrate that existing spectrum-based forecasting tools which ignore pixel-masking underestimate predicted error bars by only ~10%. This method is a promising path forward for fast lensing and delensing analyses which will be necessary for future CMB experiments such as SPT-3G, Simons Observatory, or CMB-S4, and can complement or supersede existing HMC approaches. The success of MUSE on this challenging problem strengthens its case as a generic procedure for a broad class of high-dimensional inference problems.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗

Particle image velocimetry analysis with simultaneous uncertainty quantification using Bayesian neural networks

Particle image velocimetry (PIV) is an effective tool in experimental fluid mechanics for extracting flow fields from images. Recently, convolutional neural networks (CNNs) have been used to perform PIV analysis with accuracy on par with classical methods. Here we extend the use of CNNs to analyze PIV data while providing simultaneous uncertainty quantification on the inferred flow field. The method we apply in this paper is a Bayesian convolutional neural network (BCNN) which learns distributions of the CNN weights through variational Bayes. In order to demonstrate the utility of BCNNs for the PIV task, we compare the performance of three distinct BCNN models with simple architectures. The first network estimates flow velocity from image interrogation regions only. Our second model learns to infer velocity from both the image interrogation regions and interrogation region cross-correlation maps. Finally, our best performing network infers velocities from interrogation region cross-correlation maps only. We find that BCNNs using interrogation region cross-correlation maps as inputs perform better than those using interrogation windows only as inputs and discuss reasons why this may be the case. Additionally, we test the best performing BCNN on a full synthetic test image pair and a real image pair from the 1st International PIV Challenge. We show that ~98% of true particle displacements from the full synthetic image pair can be captured within the BCNN's 95% confidence intervals, and that the BCNN's performance on the real image pair is quantitatively similar to that of algorithms tested in the 1st International PIV Challenge. Finally, we show that BCNNs can be generalized to be used with multi-pass PIV algorithms with a moderate loss in accuracy, which may be overcome by future work on finetuning and training schemes. So to our knowledge, this is the first use of Bayesian neural networks to perform PIV.

47 OTHER INSTRUMENTATION↗

An investigation on machine learning predictive accuracy improvement and uncertainty reduction using VAE-based data augmentation

The confluence of ultrafast computers with large memory, rapid progress in Machine Learning (ML) algorithms, and the availability of large datasets place multiple engineering fields at the threshold of dramatic progress. However, a unique challenge in nuclear engineering is data scarcity because experimentation on nuclear systems is usually more expensive and time-consuming than most other disciplines. One potential way to resolve the data scarcity issue is deep generative learning, which uses certain ML models to learn the underlying distribution of existing data and generate synthetic samples that resemble the real data. In this way, one can significantly expand the dataset to train more accurate predictive ML models. In this study, our objective is to evaluate the effectiveness of data augmentation using variational autoencoder (VAE)-based deep generative models. We investigated whether the data augmentation leads to improved accuracy in the predictions of a deep neural network (DNN) model trained using the augmented data. Additionally, the DNN prediction uncertainties are quantified using Bayesian Neural Networks (BNN) and conformal prediction (CP) to assess the impact on predictive uncertainty reduction. To test the proposed methodology, we used TRACE simulations of steady-state void fraction data based on the NUPEC Boiling Water Reactor Full-size Fine-mesh Bundle Test (BFBT) benchmark. Here, we found that augmenting the training dataset using VAEs has improved the DNN model’s predictive accuracy, improved the prediction confidence intervals, and reduced the prediction uncertainties.

Bayesian neural network↗

STUDIES IN ASTRONOMICAL TIME SERIES ANALYSIS. VI. BAYESIAN BLOCK REPRESENTATIONS

This paper addresses the problem of detecting and characterizing local variability in time series and other forms of sequential data. The goal is to identify and characterize statistically significant variations, at the same time suppressing the inevitable corrupting observational errors. We present a simple nonparametric modeling technique and an algorithm implementing it-an improved and generalized version of Bayesian Blocks-that finds the optimal segmentation of the data in the observation interval. The structure of the algorithm allows it to be used in either a real-time trigger mode, or a retrospective mode. Maximum likelihood or marginal posterior functions to measure model fitness are presented for events, binned counts, and measurements at arbitrary times with known error distributions. Problems addressed include those connected with data gaps, variable exposure, extension to piecewise linear and piecewise exponential representations, multivariate time series data, analysis of variance, data on the circle, other data modes, and dispersed data. Simulations provide evidence that the detection efficiency for weak signals is close to a theoretical asymptotic limit derived by Arias-Castro et al. In the spirit of Reproducible Research all of the code and data necessary to reproduce all of the figures in this paper are included as supplementary material.

79 ASTRONOMY AND ASTROPHYSICS↗