Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Bayesian Monte Carlo”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 145 records · Page 8
All sourcesNASA NTRSDOE OSTI
DOE OSTI2023

Bayesian projection pursuit regression

In projection pursuit regression (PPR), a univariate response variable is approximated by the sum of $M$ “ridge functions,” which are flexible functions of one-dimensional projections of a multivariate input variable. Traditionally, optimization routines are used to choose the projection directions and ridge functions via a sequential algorithm, and $M$ is typically chosen via cross-validation. Here, we introduce a novel Bayesian version of PPR, which has the benefit of accurate uncertainty quantification. To infer appropriate projection directions and ridge functions, we apply novel adaptations of methods used for the single ridge function case ($M$=1), called the Bayesian Single Index Model; and use a Reversible Jump Markov chain Monte Carlo algorithm to infer the number of ridge functions $M$. We evaluate the predictive ability of our model in 20 simulated scenarios and for 23 real datasets, in a bake-off against an array of state-of-the-art regression methods. Finally, we generalize this methodology and demonstrate the ability to accurately model multivariate response variables. Its effective performance indicates that Bayesian Projection Pursuit Regression is a valuable addition to the existing regression toolbox.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2023

Monte Carlo goodness-of-fit tests for degree corrected and related stochastic blockmodels

Abstract We construct Bayesian and frequentist finite-sample goodness-of-fit tests for three different variants of the stochastic blockmodel for network data. Since all of the stochastic blockmodel variants are log-linear in form when block assignments are known, the tests for the latent block model versions combine a block membership estimator with the algebraic statistics machinery for testing goodness-of-fit in log-linear models. We describe Markov bases and marginal polytopes of the variants of the stochastic blockmodel and discuss how both facilitate the development of goodness-of-fit tests and understanding of model behaviour. The general testing methodology developed here extends to any finite mixture of log-linear models on discrete data, and as such is the first application of the algebraic statistics machinery for latent-variable models.

Karwa, Vishesh↗
DOE OSTI2025

Accelerating multilevel Markov Chain Monte Carlo using machine learning models

Here, this work presents an efficient approach for accelerating multilevel Markov Chain Monte Carlo (MCMC) sampling for large-scale problems using low-fidelity machine learning models. While conventional techniques for large-scale Bayesian inference often substitute computationally expensive high-fidelity models with machine learning models, thereby introducing approximation errors, our approach offers a computationally efficient alternative by augmenting high-fidelity models with low-fidelity ones within a hierarchical framework. The multilevel approach utilizes the low-fidelity machine learning model (MLM) for inexpensive evaluation of proposed samples thereby improving the acceptance of samples by the high-fidelity model. The hierarchy in our multilevel algorithm is derived from geometric multigrid hierarchy. We utilize an MLM to accelerate the coarse level sampling. Training machine learning model for the coarsest level significantly reduces the computational cost associated with generating training data and training the model. We present an MCMC algorithm to accelerate the coarsest level sampling using MLM and account for the approximation error introduced. We provide theoretical proofs of detailed balance and demonstrate that our multilevel approach constitutes a consistent MCMC algorithm. Additionally, we derive the expression for cost reduction due to machine learning model to facilitate cost analysis of the hierarchical sampling algorithm. Our technique is demonstrated on a standard benchmark inference problem in groundwater flow, where we estimate the probability density of a quantity of interest using a four-level MCMC algorithm. Our proposed algorithm accelerates multilevel sampling by a factor of two while achieving similar accuracy compared to sampling using the standard multilevel algorithm.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2021

TensorBNN: Bayesian inference for neural networks using TensorFlow

We report that TensorBNN is a new package based on TensorFlow that implements Bayesian inference for modern neural network models. The posterior density of neural network model parameters is represented as a point cloud sampled using Hamiltonian Monte Carlo. The TensorBNN package leverages TensorFlow's architecture and its ability to use modern graphics processing units in both the training and prediction stages.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
DOE OSTI2023

DeepONet-grid-UQ: A trustworthy deep operator framework for predicting the power grid’s post-fault trajectories

This paper proposes a novel data-driven method for the reliable prediction of the power grid’s post-fault trajectories, i.e., the power grid’s dynamic response after a disturbance or fault. Here, the proposed method is based on the recently proposed concept of Deep Operator Networks (DeepONets). Unlike traditional neural networks that learn to approximate functions, DeepONets are designed to approximate nonlinear operators, i.e., mappings between infinite-dimensional spaces. Under this operator framework, we design a novel and efficient DeepONet that (i) takes as inputs the trajectories collected before and during the fault and (ii) outputs the predicted post-fault trajectories. In addition, we endow our method with the much-needed ability to balance efficiency with reliable/trustworthy predictions via uncertainty quantification. To this end, we propose and compare two novel methods that enable quantifying the predictive uncertainty. First, we propose a Bayesian DeepONet (B-DeepONet) that uses stochastic gradient Hamiltonian Monte-Carlo to sample from the posterior distribution of the DeepONet trainable parameters. Then, we design a Probabilistic DeepONet (Prob-DeepONet) that uses a probabilistic training strategy to enable quantifying uncertainty at virtually no extra computational cost. Finally, we validate the proposed methods’ predictive power and uncertainty quantification capability using the New York-New England power grid model.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2020

Automated calculation and convergence of defect transport tensors

Defect diffusion is a key process in materials science and catalysis, but as migration mechanisms are often too complex to enumerate a priori, calculation of transport tensors typically have no measure of convergence and require significant end-user intervention. These two bottlenecks prevent high-throughput implementations essential to propagate model-form uncertainty from interatomic interactions to predictive simulations. In order to address these issues, we extend a massively parallel accelerated sampling scheme, autonomously controlled by Bayesian estimators of statewide sampling completeness, to build atomistic kinetic Monte Carlo models on a state-space irreducible under exchange and space group symmetries. Focusing on isolated defects, we derive analytic expressions for drift and diffusion coefficients, providing a convergence metric by calculating the Kullback–Leibler divergence across the ensemble of diffusion processes consistent with the sampling uncertainty. The autonomy and efficacy of the method is demonstrated on surface trimers in tungsten and Hexa-interstitials in magnesium oxide, both of which exhibit complex, correlated migration mechanisms.

36 MATERIALS SCIENCE↗
DOE OSTI2020

Bayesian differential programming for robust systems identification under uncertainty

This paper presents a machine learning framework for Bayesian systems identification from noisy, sparse and irregular observations of nonlinear dynamical systems. The proposed method takes advantage of recent developments in differentiable programming to propagate gradient information through ordinary differential equation solvers and perform Bayesian inference with respect to unknown model parameters using Hamiltonian Monte Carlo sampling. This allows an efficient inference of the posterior distributions over plausible models with quantified uncertainty, while the use of sparsity-promoting priors enables the discovery of interpretable and parsimonious representations for the underlying latent dynamics. A series of numerical studies is presented to demonstrate the effectiveness of the proposed methods, including nonlinear oscillators, predator–prey systems and examples from systems biology. Taken together, our findings put forth a flexible and robust workflow for data-driven model discovery under uncertainty. All codes and data accompanying this article are available at https://bit.ly/34FOJMj .

Science & Technology - Other Topics↗
DOE OSTI2022

Gaussian processes meet NeuralODEs: a Bayesian framework for learning the dynamics of partially observed systems from scarce and noisy data

We present a machine learning framework (GP-NODE) for Bayesian model discovery from partial, noisy and irregular observations of nonlinear dynamical systems. The proposed method takes advantage of differentiable programming to propagate gradient information through ordinary differential equation solvers and perform Bayesian inference with respect to unknown model parameters using Hamiltonian Monte Carlo sampling and Gaussian Process priors over the observed system states. This allows us to exploit temporal correlations in the observed data, and efficiently infer posterior distributions over plausible models with quantified uncertainty. The use of the Finnish Horseshoe as a sparsity-promoting prior for free model parameters also enables the discovery of parsimonious representations for the latent dynamics. A series of numerical studies is presented to demonstrate the effectiveness of the proposed GP-NODE method including predator–prey systems, systems biology and a 50-dimensional human motion dynamical system. This article is part of the theme issue ‘Data-driven prediction in dynamical systems’.

Science & Technology - Other Topics↗
DOE OSTI2020

Quantifying Uncertainties In Neutron-Alpha Scattering With Chiral Nucleon-Nucleon And Three-Nucleon Forces

We report that modern ab initio theory combined with high-quality nucleon-nucleon (NN) and three-nucleon (3N) interactions from chiral effective field theory (EFT) can provide a predictive description of low-energy light-nuclei reactions relevant for astrophysics and fusion-energy applications. However, the high cost of computations has so far impeded a complete analysis of the uncertainty budget of such calculations. Starting from NN potentials up to fifth order (N 4 LO) combined with leading-order 3N forces, we study how the order-by-order convergence of the chiral expansion and confidence intervals for the 3N contact and contact-plus-one-pion-exchange low-energy constants (c E and c D ) contribute to the overall uncertainty budget of many-body calculations of neutron- 4 He (n-α) elastic scattering. We compute structure and reaction observables for three-, four-, and five-nucleon systems within the ab initio frameworks of the no-core shell model and no-core shell model with continuum. Using a small set of design runs, we construct a Gaussian process model (GPM) that acts as a statistical emulator for the theory. With this, we gain insight into how uncertainties in the 3N low-energy constants propagate throughout the calculation and determine the Bayesian posterior distribution of these parameters with Markov-Chain Monte Carlo. We find rapidly converging n-α phase shifts with respect to the chiral order. With the adopted leading-order 3N force, calculations based on the NN interaction at N 4 LO of Entem, Machleidt, and Nosyk are unable to reproduce the experimental phase shifts in the 3/2 - channel within the estimated chiral truncation errors. Closer agreement with empirical data is found when using an older parametrization of the NN interaction at order N 3 LO, and the position and width of the P-wave resonances can be used to reduce the uncertainty of the 3N low-energy constants. The present results point to a lack of spin-orbit strength when the newer parametrization of the chiral NN force up to fifth order is combined with the leading-order 3N force. The inclusion of higher-order 3N-force terms may be required to recover the missing strength. GPMs can act as fast and accurate emulators of ab initio many-body calculations of low-energy scattering and reactions of light nuclei, opening the way to a robust quantification of theoretical uncertainties grounded in the description of the underlying chiral Hamiltonian.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗
DOE OSTI2021

Bayesian analysis of a future β decay experiment's sensitivity to neutrino mass scale and ordering

Bayesian modeling techniques enable sensitivity analyses that incorporate detailed expectations regarding future experiments. A model-based approach also allows one to evaluate inferences and predicted outcomes, by calibrating (or measuring) the consequences incurred when certain results are reported. In this work, we present procedures for calibrating predictions of an experiment's sensitivity to both continuous and discrete parameters. Using these procedures and a new Bayesian model of the β-decay spectrum, we assess a high-precision β-decay experiment's sensitivity to the neutrino mass scale and ordering for one assumed design scenario. We find that such an experiment could measure the electron-weighted neutrino mass within ~40 meV after 1 year (90% credibility). Neutrino masses > 500 meV could be measured within ≈5 meV. Using only β decay and external reactor neutrino data, we find that next-generation β-decay experiments could potentially constrain the mass ordering using a two-neutrino spectral model analysis. By calibrating mass ordering results, we identify reporting criteria that can be tuned to suppress false ordering claims. In some cases, a two-neutrino analysis can reveal that the mass ordering is inverted, an unobtainable result for the traditional one-neutrino analysis approach.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗
DOE OSTI2024

Calibrating Bayesian generative machine learning for Bayesiamplification

Recently, combinations of generative and Bayesian deep learning have been introduced in particle physics for both fast detector simulation and inference tasks. These neural networks aim to quantify the uncertainty on the generated distribution originating from limited training statistics. The interpretation of a distribution-wide uncertainty however remains ill-defined. We show a clear scheme for quantifying the calibration of Bayesian generative machine learning models. For a Continuous Normalizing Flow applied to a low-dimensional toy example, we evaluate the calibration of Bayesian uncertainties from either a mean-field Gaussian weight posterior, or Monte Carlo sampling network weights, to gauge their behaviour on unsteady distribution edges. Well calibrated uncertainties can then be used to roughly estimate the number of uncorrelated truth samples that are equivalent to the generated sample and clearly indicate data amplification for smooth features of the distribution.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2025

Validating sequential Monte Carlo for gravitational-wave inference

Nested sampling (NS) is the preferred stochastic sampling algorithm for gravitational-wave inference for compact binary coalescences. It can handle the complex nature of the gravitational-wave likelihood surface and provides an estimate of the Bayesian model evidence. However, there is another class of algorithms that meets the same requirements, but has not been used for gravitational-wave analyses: sequential Monte Carlo (SMC), an extension of importance sampling that maps samples from an initial density to a target density via a series of intermediate densities. In this work, we validate a type of SMC algorithm, called persistent sampling (PS), for gravitational-wave inference. We consider a range of different scenarios including binary black holes and binary neutron stars and real and simulated data and show that PS produces results that are consistent with NS whilst being, on average, 2 times more efficient and 2.74 times faster. This demonstrates that PS is a viable alternative to NS that should be considered for future gravitational-wave analyses.

black hole mergers↗
DOE OSTI2026

Hierarchical Gaussian Random Field Sampling for Multilevel Markov Chain Monte Carlo: Coupling Stochastic Partial Differential Equation and the Karhunen–Loève Decomposition

This work introduces structure preserving hierarchical decompositions for sampling Gaussian random fields (GRFs) within the context of multilevel Bayesian inference in high-dimensional space. Existing scalable hierarchical sampling methods, such as those based on stochastic partial differential equations (SPDEs), often reduce the dimensionality of the sample space at the cost of accuracy of inference. Other approaches, such that those based on Karhunen-Loève (KL) expansions, offer sample space dimensionality reduction but sacrifice GRF representation accuracy and ergodicity of the Markov chain Monte Carlo (MCMC) sampler and are computationally expensive for high-dimensional problems. The proposed method integrates the dimensionality reduction capabilities of KL expansions with the scalability of SPDE-based sampling, thereby providing a robust, unified framework for high-dimensional uncertainty quantification (UQ) that is scalable and accurate, preserves ergodicity, and offers dimensionality reduction of the sample space. The hierarchy in our multilevel algorithm is derived from the geometric multigrid hierarchy. By constructing a hierarchical decomposition that maintains the covariance structure across the levels in the hierarchy, the approach enables efficient coarse-to-fine sampling while ensuring that all samples are drawn from the desired distribution. The effectiveness of the proposed method is demonstrated on a benchmark subsurface flow problem, demonstrating its effectiveness in improving computational efficiency and statistical accuracy. Furthermore, our proposed technique is more efficient and accurate and displays better convergence properties than existing methods for high-dimensional Bayesian inference problems.

Gaussian random fields↗
DOE OSTI2024

Microcanonical Hamiltonian Monte Carlo

We develop Microcanonical Hamiltonian Monte Carlo (MCHMC), a class of models that follow fixed energy Hamiltonian dynamics, in contrast to Hamiltonian Monte Carlo (HMC), which follows canonical distribution with different energy levels. MCHMC tunes the Hamiltonian function such that the marginal of the uniform distribution on the constant-energy-surface over the momentum variables gives the desired target distribution. We show that MCHMC requires occasional energy-conserving billiard-like momentum bounces for ergodicity, analogous to momentum resampling in HMC. We generalize the concept of bounces to a continuous version with partial direction preserving bounces at every step, which gives energy-conserving underdamped Langevin-like dynamics with non-Gaussian noise (MCLMC). MCHMC and MCLMC exhibit favorable scalings with condition number and dimensionality. We develop an efficient hyperparameter tuning scheme that achieves high performance and consistently outperforms NUTS HMC on several standard benchmark problems, in some cases by orders of magnitude.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2020

Radiation Source Localization Using Surrogate Models Constructed from 3-D Monte Carlo Transport Physics Simulations

Recent research has focused on the development of surrogate models for radiation source localization in a simulated urban domain. We employ the Monte Carlo N-Particle (MCNP) code to provide high- delity simulations of radiation transport within an urban domain. The model is constructed to employ a source location (x, y, z) as input and return the estimated count rate for a set of speci ed detector locations. Because MCNP simulations are computationally expensive, we develop e cient and accurate surrogate models of the detector responses. We construct surrogate models using Gaussian processes (GP) and neural networks (NN) that we train and verify using the MCNP simulations. The trained surrogate models provide an e cient framework for Bayesian inference and experimental design. We employ Delayed Rejection Adaptive Metropolis (DRAM), a Markov Chain Monte Carlo (MCMC) algorithm, to infer the location and intensity of an unknown source. The DRAM results yield a posterior probability distribution for the source's location conditioned on the observed detector count rates. The posterior distribution exhibits regions of high and low probability within the simulated environment identifying potential source locations. In this manner, we can quantify the source location to within at least one of these regions of high probability in the considered cases. Employing these methods, we are able to reduce the space of potential source locations by at least 60%.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗
DOE OSTI2024

Hybrid Parameter Search and Dynamic Model Selection for Mixed-Variable Bayesian Optimization

Herein this article presents a new type of hybrid model for Bayesian optimization (BO) adept at managing mixed variables, encompassing both quantitative (continuous and integer) and qualitative (categorical) types. Our proposed new hybrid models (named hybridM) merge the Monte Carlo Tree Search structure (MCTS) for categorical variables with Gaussian Processes (GP) for continuous ones. hybridM leverages the upper confidence bound tree search (UCTS) for MCTS strategy, showcasing the tree architecture’s integration into Bayesian optimization. Our innovations, including dynamic online kernel selection in the surrogate modeling phase and a unique UCTS search strategy, position our hybrid models as an advancement in mixed-variable surrogate models. Numerical experiments underscore the superiority of hybrid models, highlighting their potential in Bayesian optimization.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2022

Bayesian optimization to design a novel x-ray shaping device

In radiation therapy, x-ray dose must be precisely sculpted to the tumor, while simultaneously avoiding surrounding organs at risk. This requires modulation of x-ray intensity in space and/or time. Typically, this is achieved using a multi leaf collimator (MLC) - a complex mechatronic device comprising over one hundred individually powered tungsten ‘leaves’ that move in or out of the radiation field as required. Here, an all-electronic x-ray collimation concept with no moving parts is presented, termed “SPHINX”: Scanning Pencil-beam High-speed Intensity-modulated X-ray source. SPHINX utilizes a spatially distributed bremsstrahlung target and collimator array in conjunction with magnetic scanning of a high energy electron beam to generate a plurality of small x-ray “beamlets.” A simulation framework was developed in Topas Monte Carlo incorporating a phase space electron source, transport through user defined magnetic fields, bremsstrahlung x-ray production, transport through a SPHINX collimator, and dose in water. This framework was completely parametric, meaning a simulation could be built and run for any supplied geometric parameters. This functionality was coupled with Bayesian optimization to find the best parameter set based on an objective function which included terms to maximize dose rate for a user defined beamlet width while constraining inter-channel cross talk and electron contamination. Designs for beamlet widths of 5, 7, and 10 mm 2 were generated. Each optimization was run for 300 iterations and took approximately 40 h on a 24-core computer. For the optimized 7-mm model, a simulation of all beamlets in water was carried out including a linear scanning magnet calibration simulation. Finally, a back-of-envelope dose rate formalism was developed and used to estimate dose rate under various conditions. The optimized 5–, 7–, and 10-mm models had beamlet widths of 5.1 , 7.2 , and 10.1 mm 2 and dose rates of 3574, 6351, and 10 015 Gy/C, respectively. The reduction in dose rate for smaller beamlet widths is a result of both increased collimation and source occlusion. For the simulation of all beamlets in water, the scanning magnet calibration reduced the offset between the collimator channels and beam centroids from 2.9 ±1.9 mm to 0.01 ±0.03 mm. A slight reduction in dose rate of approximately 2% per degree of scanning angle was observed. Based on a back-of-envelope dose rate formalism, SPHINX in conjunction with next-generation linear accelerators has the potential to achieve substantially higher dose rates than conventional MLC-based delivery, with delivery of an intensity modulated 100 x 100 mm 2 field achievable in 0.9 to 10.6 s depending on the beamlet widths used. Bayesian optimization was coupled with Monte Carlo modeling to generate SPHINX geometries for various beamlet widths. A complete Monte Carlo simulation for one of these designs was developed, including electron beam transport of all beamlets through scanning magnets, x-ray production and collimation, and dose in water. These results demonstrate that SPHINX is a promising candidate for sculpting radiation dose with no moving parts, and has the potential to vastly improve both the speed and robustness of radiotherapy delivery. A multi-beam SPHINX system may be a candidate for delivering magavoltage FLASH RT in humans.

62 RADIOLOGY AND NUCLEAR MEDICINE↗