SEARCH · Engineering Papers

Results for “Bayesian Monte Carlo”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 37 records · Page 2

All sources NASA NTRS DOE OSTI

DOE OSTI2022

Challenges in Markov Chain Monte Carlo for Bayesian Neural Networks

Markov chain Monte Carlo (MCMC) methods have not been broadly adopted in Bayesian neural networks (BNNs). This paper initially reviews the main challenges in sampling from the parameter posterior of a neural network via MCMC. Such challenges culminate to lack of convergence to the parameter posterior. Nevertheless, this paper shows that a nonconverged Markov chain, generated via MCMC sampling from the parameter space of a neural network, can yield via Bayesian marginalization a valuable posterior predictive distribution of the output of the neural network. Further, classification examples based on multilayer perceptrons showcase highly accurate posterior predictive distributions. The postulate of limited scope for MCMC developments in BNNs is partially valid; an asymptotically exact parameter posterior seems less plausible, yet an accurate posterior predictive distribution is a tenable research avenue.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2022

Deterministic and Monte Carlo Nuclear Data Adjustment Methods [Slides]

For the Bayesian Monte Carlo methodology, a need to understand convergence of the posterior moments as a function of the number of parameter realizations is required. In high-dimensional systems, it can be very costly to sample entire parameter space and perform functional evaluation for every realization. Bayesian Monte Carlo allows one to relax the GLLS approximations of model linearity and prior/posterior PDF shape. The Bayesian Stochastic Collocation Method is a deterministic approach to “sample” the parameter space. It allows one to relax the GLLS approximations of model linearity and posterior PDF shape. Higher-order posterior moments (i.e., skewness, kurtosis, etc.) can be studied through polynomial expansion. Tensor product quadrature scales poorly and can use sparse grid quadrature methods.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗

DOE OSTI2025

Accelerating Hamiltonian Monte Carlo for Bayesian inference in neural networks and neural operators

Hamiltonian Monte Carlo (HMC) is a powerful and accurate method to sample from the posterior distribution in Bayesian inference. However, HMC techniques are computationally demanding for Bayesian neural networks due to the high dimensionality of the network’s parameter space and the non-convexity of their posterior distributions. Therefore, various approximation techniques, such as variational inference (VI) or stochastic gradient MCMC, are often employed to infer the posterior distribution of the network parameters. Such approximations introduce inaccuracies in the inferred distributions, resulting in unreliable uncertainty estimates. In this work, we propose a hybrid approach that combines inexpensive VI and accurate HMC methods to efficiently and accurately quantify uncertainties in neural networks and neural operators. The proposed approach leverages an initial VI training on the full network. We examine the influence of individual parameters on the prediction uncertainty, which shows that a large proportion of the parameters do not contribute substantially to uncertainty in the network predictions. This information is then used to significantly reduce the dimension of the parameter space, and HMC is performed only for the subset of network parameters that strongly influence prediction uncertainties. This yields a framework for accelerating the full batch HMC for posterior inference in neural networks. We demonstrate the efficiency and accuracy of the proposed framework on deep neural networks and operator networks, showing that inference can be performed for large networks with tens to hundreds of thousands of parameters. Finally, we show that this method can effectively learn surrogates for complex physical systems by modeling the operator that maps from upstream conditions to wall-pressure data on a cone in hypersonic flow.

Bayesian inference↗

DOE OSTI2025

A Markov chain Monte Carlo (MCMC) Bayesian inference approach to analyze apparent activation barriers and reaction orders from microreactor data

Statistical analysis of steady-state catalytic kinetic data is often limited by data sparsity due to the slow pace at which the data is collected. Data sparsity and limitations in statistical analysis make it difficult to differentiate between mechanistic models and catalytic sites. A Bayesian inference tool is reported for catalysis researchers to estimate error in the determination of reaction orders from steady state microreactor data. The benefits of a Bayesian inference approach are discussed, as an alternative to the more common frequentist approach. The approach incorporates prior knowledge of the system and the data collected to form an error estimate on reaction orders. We investigated the effects of three distinct data treatments—individual fitting of trials, pooled analysis, and constrained regression methods—on the precision and uncertainty of reaction order determinations. To assess the robustness of our findings, we conducted sensitivity analyses to evaluate the influence of Bayesian parameters on uncertainty estimation. Additionally, we utilized synthetic data to illustrate how data quality impacts the precision of uncertainty assessments. We show Bayesian analysis can obtain a more precise estimation of error with a sparse data set than a frequentist analysis. Finally, this work provides strong evidence that the adoption of Bayesian analysis of kinetic data may help researchers make more precise arguments as to the strength of their evidence for a particular mechanistic hypothesis, or in comparing across different catalysts.

42 ENGINEERING↗

DOE OSTI2021

Bayesian Monte-Carlo Evaluation Framework for Imperfect Data [Slides]

BMC evaluation is a tool to address imperfect data & models, non-linear models, and non-normal PDFs. New posterior PDFs may need new storage formats to allow storage of non-normal PDFs. Storing posterior sets allows for: variance, covariance, skewness, etc.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2021

Bayesian Monte-Carlo Evaluation Framework for Imperfect Nuclear Data [Slides]

BMC evaluation is a tool used to address imperfect data and models, non-linear models, and non-normal PDFs. ENDF-6 format does not allow non-normal parameter PDFs. Storing posterior sets allows for variance, covariance, skewness, etc. To better predict criticality, we should document non-normal parameter PDFs (i.e. asymmetric uncertainty) and consider non-linear sensitivity of $k_{\text{eff}}$ to resonance parameters.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗

DOE OSTI2022

Bayesian Monte-Carlo Framework: New Methods for Resonance Parameter Evaluation [Slides]

BMC evaluation is a tool to address imperfect data & models, non-linear models, and non-normal PDFs. ENDF-6 format does not allow non-normal parameter PDFs. Storing posterior sets allow for variance, covariance, skewness, etc. To better predict criticality, we could document non-normal parameter PDFs (i.e. asymmetric uncertainty), consider non-linear sensitivity of $\kappa$ eff to resonance parameters, and reduce uncertainty in key resonance parameters.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗

DOE OSTI2022

Parameter estimation for X-ray scattering analysis with Hamiltonian Markov Chain Monte Carlo

Bayesian-inference-based approaches, in particular the random-walk Markov Chain Monte Carlo (MCMC) method, have received much attention recently for X-ray scattering analysis. Hamiltonian MCMC, a state-of-the-art development in the field of MCMC, has become popular in recent years. It utilizes Hamiltonian dynamics for indirect but much more efficient drawings of the model parameters. We described the principle of the Hamiltonian MCMC for inversion problems in X-ray scattering analysis by estimating high-dimensional models for several motivating scenarios in small-angle X-ray scattering, reflectivity, and X-ray fluorescence holography. Hamiltonian MCMC with appropriate preconditioning can deliver superior performance over the random-walk MCMC, and thus can be used as an efficient tool for the statistical analysis of the parameter distributions, as well as model predictions and confidence analysis.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗

DOE OSTI2026

Dark matter clumps as sources of gravitational-wave glitches in LIGO-Virgo-KAGRA data

We consider the hypothetical possibility that nonstationary glitch features in the noise of ground-based gravitational-wave detectors could be produced by small dark matter clumps that pass through the earth in the vicinity of gravitational-wave detectors. Here, we first derive the gravitational-wave strain that would be generated by the passage of such a dark matter clump. We find that the strain is primarily sourced by the Newtonian gravitational acceleration of the mirrors toward the clump and by the Shapiro time delay of the photons in the laser beams as they pass through the gravitational potential created by the dark matter clump. We also find that the Newtonian acceleration effect dominates the gravitational-wave strain for both ground and space-based interferometers. We then compare our dark matter clump, gravitational-wave strain model to 84 Koi-Fish glitches detected during the second observing run of the LIGO/Virgo/KAGRA Collaboration through a Markov chain Monte Carlo Bayesian analysis. We find that all glitches but nine can be confidently rejected as having originated from dark matter clumps. For the remaining glitches, the dark matter hypothesis cannot be excluded, and the maximum a posteriori parameters yield minimum densities of about 10 −7 g⁡/cm 3 , within the model. These results allow us to place the first direct upper limits with gravitational-wave detectors on the local over-density of dark matter in the form of clumps in the local neighborhood of Earth, namely 𝜌 DM ⁢clumps ≲ 10 −15 g⁡/cm −3 .

Astronomy and AstroPhysics↗

NASA NTRS1992

The statistical analysis of circadian phase and amplitude in constant-routine core-temperature data

Accurate estimation of the phases and amplitude of the endogenous circadian pacemaker from constant-routine core-temperature series is crucial for making inferences about the properties of the human biological clock from data collected under this protocol. This paper presents a set of statistical methods based on a harmonic-regression-plus-correlated-noise model for estimating the phases and the amplitude of the endogenous circadian pacemaker from constant-routine core-temperature data. The methods include a Bayesian Monte Carlo procedure for computing the uncertainty in these circadian functions. We illustrate the techniques with a detailed study of a single subject's core-temperature series and describe their relationship to other statistical methods for circadian data analysis. In our laboratory, these methods have been successfully used to analyze more than 300 constant routines and provide a highly reliable means of extracting phase and amplitude information from core-temperature data.

NASA Discipline Regulatory Physiology↗

NASA NTRS2017

Assimilation of All-Weather GMI and ATMS Observations into HWRF

Moradi, Isaac↗

NASA NTRS2018

Assimilation of All-Weather GMI and ATMS Observations into HWRF

Moradi, Isaac↗

NASA NTRS2018

Assimilation of GMI and ATMS Observations in the Rainbands of Hurricanes

We propose a novel Bayesian Monte Carlo Integration (BMCI) technique to retrieve the profiles of temperature, water vapor, and cloud liquid/ice water content from microwave cloudy measurements in the presence of tropical cyclones (TC). These retrievals then can either be directly used by meteorologists to analyze the structure of TCs or be assimilated into numerical models to provide accurate initial conditions for the NWP models. The BMCI technique is applied to the data from the Advanced Technology Microwave Sounder (ATMS) onboard Suomi National Polar-orbiting Partnership (NPP) and Global Precipitation Measurement (GPM) Microwave Imager (GMI). The retrieved profiles are then assimilated into Hurricane WRF using GSI data assimilation system. This talk presents the development of the BMCI retrieval system as well as the preliminary results of assimilating the retrievals into NASA GEOS-5 model.

Moradi, Isaac↗

NASA NTRS2019

Assimilation of Microwave Observations in the Rainbands of Tropical Cyclones

We propose a novel Bayesian Monte Carlo Integration (BMCI) technique to retrieve the profiles of temperature, water vapor, and cloud liquid/ice water content from microwave cloudy measurements in the presence of tropical cyclones (TC). These retrievals then can either be directly used by meteorologists to analyze the structure of TCs or be assimilated into numerical models to provide accurate initial conditions for the NWP (Numerical Weather Prediction) models. The BMCI technique is applied to the data from the Advanced Technology Microwave Sounder (ATMS) onboard Suomi National Polar-orbiting Partnership (NPP) and Global Precipitation Measurement (GPM) Microwave Imager (GMI). The retrieved profiles are then assimilated into Hurricane WRF (Weather Research and Forecasting) using the GSI (Gridpoint Statistical Interpolation) data assimilation system.

Moradi, Isaac↗

NASA NTRS2020

Tests of General Relativity with Stellar-Mass Black Hole Binaries Observed by LISA

We consider the observation of stellar-mass black holes binaries with the Laser Interferometer Space Antenna(LISA). Preliminary results based on Fisher information matrix analyses have suggested that gravitational waves from those sources could be very sensitive to possible deviations from the theory of general relativity (GR) andfrom the strong equivalence principle during the low-frequency binary inspiral. We perform a full Markov Chain Monte Carlo Bayesian analysis to quantify the sensitivity of these signals to two phenomenological modifications of GR, namely a putative gravitational dipole emission and a non-zero mass for the graviton, properly accounting for the detector’s response. Moreover, we consider a scenario where those sources could be observed also with Earth-based detectors, which should measure the coalescence time with precision better than 1 ms. This constraint on the coalescence time further improves the bounds that we can set on those phenomenological deviations from GR. We show that tests of dipole radiation and the graviton’s mass should improve respectively by seven and half an order(s) of magnitude over current bounds. Finally, we discuss under which conditions one may claim the detection of a modification to General Relativity.

Alexandre Toubiana↗

NASA NTRS2020

Assimilation of satellite microwave observations over the rainbands of tropical cyclones

A novel Bayesian Monte Carlo integration (BMCI) technique was developed to retrieve geophysical variables from satellite microwave radiometer data in the presence of tropical cyclones. The BMCI technique includes three steps: generating a stochastic database, simulating satellite brightness temperatures using a radiative transfer model, and retrieving geophysical variables such as profiles of temperature, relative humidity, and cloud liquid and ice water content from real observations. The technique also provides uncertainty estimates for each retrieval and can output the error covariance matrix of selected parameters. The measurements from the Advanced Technology Microwave Sounder (ATMS) on board Suomi National Polar-Orbiting Partnership (Suomi NPP) and the Global Precipitation Measurement (GPM) Microwave Imager (GMI) were used as input. A new technique was developed to correct the ATMS and GMI observations for the beam-filling effect, which is due to small-scale variability of precipitation and clouds when compared with the instrument footprint and also the nonlinear relation between the brightness temperature and precipitation. In addition, the assimilation of the BMCI retrievals into the NASA GEOS model is discussed for Hurricane Maria. The results show that assimilating the BMCI retrievals can influence the dynamical features of the cyclone, including a stronger warm core, a symmetric eye, and vertically aligned wind columns. Two possible factors that may limit the impact of the BMCI retrievals include 1) the resolution of the model (about 25 km), which was too coarse to show the potential of the BMCI data in improving the representation of tropical storms in the model forecast, and 2) the data assimilation system not being able to consider vertically correlated observation errors.

Isaac Moradi↗

DOE OSTI2022

Modernization efforts for the R -Matrix code SAMMY [Abstract]

The R-Matrix code SAMMY is a widely used nuclear data evaluation code focused on the resolved range, which includes corrections for experimental effects. The code is still mostly written in Fortran 77, and uses a memory management system suitable for the time of its initial writing (1984). A modernization effort is under way to bring the code in-line with modern software development practices. A continuous-integration testing framework was added, automating the large existing set of test cases. It is run on every commit. The memory management was updated to current standard practices suitable for modern software analysis tools. The code can be obtained from https://code.ornl.gov/RNSD/SAMMY. The resonance parameters and covariance information are now stored in C++ objects shared by SAMMY and AMPX, the processing code that generates nuclear data libraries for SCALE. This allows for easier maintenance and access to the resonance parameters inside and outside of SAMMY. This feature is already used by accessing and changing parameters in memory in the Bayesian Monte Carlo Evaluation Framework for Cross Sections Nuclear Data and Integral Benchmark Experiments project, Further plans include the switch to the ENDF reading and writing routines in AMPX, as these routines are more robust, easier to maintain, and support more features. Of note here is support for the new GNDS format. Previously it wasn’t easy to share the full covariance matrix for evaluations containing more than one isotope due to limitations on the ENDF format; this is now supported in GNDS. The data are currently available in a binary SAMMY format and can be exported to GNDS to make them more widely available and sharable. The next step will be to use the same resonance processing code at 0K in AMPX and SAMMY as one of the available Reich-Moore R-Matrix formalism. The first step toward this goal is to isolate the reconstruction into a module that takes resonance parameters as its input and does not depend on SAMMY global parameters. This goal has been achieved and it should now be possible to more easily change the resonance formalism and add enhancements as the Phenomenological R-Matrix parameterization of direct, doorway, and compound nuclear reactions discussed elsewhere on this conference. This concerted modernization and enhancement effort provides multiple advantages to the nuclear data community. It will allow parameter optimization using enhanced formalisms, including experimental effects, that better match complex experimental data. Then those evaluated parameters can immediately be passed off to AMPX to be reconstructed with the exact same cross section model and be put into a data library for subsequent testing using SCALE and the Valid Benchmark suite or other suitable benchmark suites.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗

DOE OSTI2024

Sequential Monte Carlo for Cut-Bayesian Posterior Computation

We propose a sequential Monte Carlo (SMC) method to efficiently and accurately compute cut-Bayesian posterior quantities of interest, variations of standard Bayesian approaches constructed primarily to account for model misspecification. We prove finite sample concentration bounds for estimators derived from the proposed method along with a linear tempering extension and apply these results to a realistic setting where a computer model is misspecified. We then illustrate the SMC method for inference in a modular chemical reactor example that includes submodels for reaction kinetics, turbulence, mass transfer, and diffusion. The samples obtained are commensurate with a direct-sampling approach that consists of running multiple Markov chains, with computational efficiency gains using the SMC method. Overall, the SMC method presented yields a novel, rigorous approach to computing with cut-Bayesian posterior distributions.

97 MATHEMATICS AND COMPUTING↗