Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Bayesian Inference”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,117 records · Page 62

Scalable Bayesian Physics-Informed Kolmogorov-Arnold Networks

Uncertainty quantification (UQ) plays a pivotal role in scientific machine learning, especially when surrogate models are used to approximate complex systems. Although multilayer perceptions (MLPs) are commonly employed as surrogates, they often suffer from overfitting due to their large number of parameters. Kolmogorov-Arnold networks (KANs) offer an alternative solution with fewer parameters. However, gradient-based inference methods, such as Hamiltonian Monte Carlo (HMC), may result in computational inefficiency when applied to KANs, especially for large-scale datasets, due to the high cost of back-propagation. To address these challenges, we propose a novel approach, combining the dropout Tikhonov ensemble Kalman inversion (DTEKI) with Chebyshev KANs. This gradient-free method effectively mitigates overfitting and enhances numerical stability. In addition, we incorporate the active subspace method to reduce the parameter-space dimensionality, allowing us to improve the accuracy of predictions and obtain more reliable uncertainty estimates. Extensive experiments demonstrate the efficacy of our approach in various test cases, including scenarios with large datasets and high noise levels. Our results show that the new method achieves comparable or better accuracy, much higher efficiency as well as stability compared to HMC, in addition to scalability. Moreover, by leveraging the low-dimensional parameter subspace, our method preserves prediction accuracy while substantially reducing further the computational cost.

97 MATHEMATICS AND COMPUTING↗

BASS: An R Package for Fitting and Performing Sensitivity Analysis of Bayesian Adaptive Spline Surfaces

In this work, we present the R package BASS as a tool for nonparametric regression. The primary focus of the package is fitting fully Bayesian adaptive spline surface (BASS) models and performing global sensitivity analyses of these models. The BASS framework is similar to that of Bayesian multivariate adaptive regression splines (BMARS) from Denison, Mallick, and Smith (1998), but with many added features. The software is built to efficiently handle significant amounts of data with many continuous or categorical predictors and with functional response. Under our Bayesian framework, most priors are automatic but these can be modified by the user to focus on parsimony and the avoidance of overfitting. If directed to do so, the software uses parallel tempering to improve the reversible jump Markov chain Monte Carlo (RJMCMC) methods used to perform inference. We discuss the implementation of these features and present the performance of BASS in a number of analyses of simulated and real data.

97 MATHEMATICS AND COMPUTING↗

Reduced chemistry model calibration using unsteady flames with repetitive extinction and ignition

In this paper, combustion in a narrow channel with an imposed temperature gradient is used to calibrate a reduced chemical mechanism. The focus is placed on unsteady flames with repetitive extinction and ignition (FREI) at high pressure. The reduced mechanism follows a hybrid approach, combining a small fuel-dependent submechanism with a detailed fuel-independent submechanism for the core species chemistry. The fuel-dependent submechanism consists of three species and nine reactions describing both the low and high-temperature fuel decomposition pathways. The sensitivity of the extinction and ignition locations to the fuel-dependent reactions is assessed and multiple reactions are found to affect the FREI behavior. The sensitive reactions are calibrated following a Bayesian approach. The calibrated reduced model is compared to a detailed model and good agreement in ignition delay times and laminar flame speeds is observed. Finally, this illustrates the potential of a micro flow reactor with a controlled temperature profile to test fuels, calibrate chemical models, and infer kinetic data.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

Technical Report: Algorithm and Implementation for Quasispecies Abundance Inference with Confidence Intervals from Metagenomic Sequence Data

This report describes the design and implementation of an algorithm for estimating relative microbial abundances, together with confidence limits, using data from metagenomic DNA sequencing. For the background behind this project and a detailed discussion of our modeling approach for metagenomic data, we refer the reader to our earlier technical report, dated March 4, 2014. Briefly, we described a fully Bayesian generative model for paired-end sequence read data, incorporating the effects of the relative abundances, the distribution of sequence fragment lengths, fragment position bias, sequencing errors and variations between the sampled genomes and the nearest reference genomes. A distinctive feature of our modeling approach is the use of a Chinese restaurant process (CRP) to describe the selection of genomes to be sampled, and thus the relative abundances. The CRP component is desirable for fitting abundances to reads that may map ambiguously to multiple targets, because it naturally leads to sparse solutions that select the best representative from each set of nearly equivalent genomes.

59 BASIC BIOLOGICAL SCIENCES↗

A Graphical Model for Fusing Diverse Microbiome Data

This paper develops a Bayesian graphical model for fusing disparate types of count data. The motivating application is the study of bacterial communities from diverse high-dimensional features, in this case, transcripts, collected from different treatments. In such datasets, there are no explicit correspondences between the communities and each corresponds to different factors, making data fusion challenging. We introduce a flexible multinomial-Gaussian generative model for jointly modeling such count data. This latent variable model jointly characterizes the observed data through a common multivariate Gaussian latent space that parameterizes the set of multinomial probabilities of the transcriptome counts. The covariance matrix of the latent variables induces a covariance matrix of co-dependencies between all the transcripts, effectively fusing multiple data sources. We present a computationally scalable variational Expectation-Maximization (EM) algorithm for inferring the latent variables and the parameters of the model. Here, the inferred latent variables provide a common dimensionality reduction for visualizing the data and the inferred parameters provide a predictive posterior distribution. In addition to simulation studies that demonstrate the variational EM procedure, we apply our model to a bacterial microbiome dataset.

59 BASIC BIOLOGICAL SCIENCES↗

Bayesian time-varying occupancy model for West Nile virus in Ontario, Canada

Occupancy models determine the true presence or absence of a species by adjusting for imperfect detection in surveys. They often assume that species presences can be detected only if sites are occupied during a sampling season. We extended these models to estimate occupancy rates that vary throughout a sampling season as well as account for spatial dependence among sites. For these methods, we constructed a fast Gibbs sampler with the Pólya-Gamma augmentation strategy to conduct inference on covariate effects. We applied these methods to evaluate how environmental conditions and surveillance practices are associated with the presence of West Nile virus in mosquito traps across Ontario, Canada from 2002 to 2017. We found that urban land cover and warm temperatures drove viral occupancy, whereas viral testing on pools with higher proportions of Culex mosquitoes was more likely to result in a positive test for West Nile virus. Models with time-varying occupancy effects achieved much lower Watanabe-Akaike information criteria than models without such effects. Our final model had strong predictive performance on test data that included some of the most extreme seasons, demonstrating the promise of these methods in the study of pathogens spread by mosquito vectors.

60 APPLIED LIFE SCIENCES↗

Inference of Multichannel r -process Element Enrichment in the Milky Way Using Binary Neutron Star Merger Observations

Observations of GW170817 strongly suggest that binary neutron star (BNS) mergers produce rapid neutron-capture nucleosynthesis ( r -process) elements. However, it remains an open question whether these mergers can account for all the r -process element enrichment in the Milky Way’s history. Here, we constrain the contributions of the BNS channel using astrophysical neutron star observations. The rate and mass distributions are constrained by LIGO/Virgo/Kagra through the latest catalog GWTC-3, the neutron star equation of state by gravitational-wave, radio, and X-ray observations, and the delay time distribution by short gamma-ray burst (GRB) host galaxy associations. We present a Bayesian framework to consistently combine these observations with abundance information to quantify the contribution and uncertainties of single and multiple astrophysical enrichment sources, and obtain a distribution of per-event BNS r -process element yields consistent with geophysical and astrophysical abundance constraints. We then adopt a Galactic chemical evolution model assuming an instantaneous and fixed amount of Fe enrichment from core-collapse supernovae, and show that BNS-only enrichment scenarios remain inconsistent with the observed r-process abundance trend of disk stars in the Galaxy even with the uncertainties in BNS merger observations. Using stellar abundance observations instead of the short GRB constraints, we can infer a shorter BNS delay time distribution with power-law index α ≤ −2.0 and minimum delay time ${t}_{{\rm{\min }}}\leqslant 40$ Myr at 90% confidence, consistent with detailed Galactic chemical evolution models. Such delay times are in tension with those predicted by standard BNS formation models. Alternatively, we confirm that a two-channel scenario, in which the second channel tracks the star formation history without significant delay, can account for both Galactic stellar and short GRB observations. We estimate that 45%–90% of the r -process abundance in the Milky Way today would have been produced by this star formation-tracking channel, rather than BNS mergers with significant delay times.

gravitational wave astronomy↗

A Statistician’s Overview of Physics-Informed Neural Networks for Spatio-Temporal Data

The recent success of deep neural network models with physical constraints (so-called, Physics-Informed Neural Networks, PINNs) has led to renewed interest in the incorporation of mechanistic information in predictive models. Statisticians and others have long been interested in this problem, which has led to several practical and innovative solutions dating back decades. In this overview, we focus on the problem of data-driven prediction and inference of dynamic spatio-temporal processes that include mechanistic information, such as would be available from partial differential equations, with a strong focus on the quantification of uncertainty associated with data, process, and parameters. Here, we give a brief review of several paradigms and focus our attention on Bayesian implementations given they naturally accommodate uncertainty quantification. We then show that it is straight-forward to include the Bayesian PINN (B-PINN) within the Bayesian hierarchical model (BHM) framework that has long been considered for modeling dynamic spatio-temporal processes. Such a BHM-PINN is illustrated via a simulation study in which a latent nonlinear Burgers’ equation PDE governs the dynamics of Poisson distributed spatio-temporal data. Supplementary materials for this article are available online, including a standardized description of the materials available for reproducing the work.

Bayesian↗

Operational, gauge-free quantum tomography

As increasingly impressive quantum information processors are realized in laboratories around the world, robust and reliable characterization of these devices is now more urgent than ever. These diagnostics can take many forms, but one of the most popular categories is tomography, where an underlying parameterized model is proposed for a device and inferred by experiments. Here, we introduce and implement efficient operational tomography, which uses experimental observables as these model parameters. This addresses a problem of ambiguity in representation that arises in current tomographic approaches (the gauge problem). Solving the gauge problem enables us to efficiently implement operational tomography in a Bayesian framework computationally, and hence gives us a natural way to include prior information and discuss uncertainty in fit parameters. We demonstrate this new tomography in a variety of different experimentally-relevant scenarios, including standard process tomography, Ramsey interferometry, randomized benchmarking, and gate set tomography.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

Quantifying uncertainties due to optical potentials in one-neutron knockout reactions

One-neutron knockout reactions have been widely used to extract information about the single-particle structure of nuclei from the valley of stability to the drip lines. The interpretation of knockout data relies on reaction models, where the uncertainties are typically not accounted for. In this work, we quantify uncertainties of optical potentials used in these reaction models and propagate them, for the first time, to knockout observables using a Bayesian analysis. We study two reactions in the present paper, the first of which involves a loosely bound halo projectile, 11 Be, and the second a tightly bound projectile, 12 C. We first quantify the parametric uncertainties associated with phenomenological optical potentials. Complementing this approach, we also quantify the model uncertainties associated with the chiral forces that can be used to construct microscopic optical potentials. For the phenomenological study, we investigate the impact of the imaginary terms of the optical potential on the breakup and stripping components of the knockout cross sections as well as the impact of the angular range. For the 11 Be case, the theoretical uncertainty from the phenomenological method is on the order of the experimental uncertainty of the knockout observables; however, for the 12 C case, the theoretical uncertainty is significantly larger. Additionally, the widths of the uncertainty bands for the knockout observables obtained for the microscopic study and the phenomenological approach are of similar orders of magnitude. Based on this work we conclude that structure information inferred from the ratio of the knockout cross sections will carry a theoretical uncertainty of at least 20% for halo nuclei and at least 40% for tightly bound nuclei.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗

Joint velocity and density reconstruction of the Universe with nonlinear differentiable forward modeling

Reconstructing the initial conditions of the Universe from late-time observations has the potential to optimally extract cosmological information. Due to the high dimensionality of the parameter space, a differentiable forward model is needed for convergence, and recent advances have made it possible to perform reconstruction with nonlinear models based on galaxy (or halo) positions. In addition to positions, future surveys will provide measurements of galaxies' peculiar velocities through the kinematic Sunyaev-Zel'dovich effect (kSZ), type Ia supernovae, the fundamental plane relation, and the Tully-Fisher relation. Here we develop the formalism for including halo velocities, in addition to halo positions, to enhance the reconstruction of the initial conditions. We show that using velocity information can significantly improve the reconstruction accuracy compared to using only the halo density field. Here we study this improvement as a function of shot noise, velocity measurement noise, and angle to the line of sight. We also show how halo velocity data can be used to improve the reconstruction of the final nonlinear matter overdensity and velocity fields. We have built our pipeline into the differentiable Particle-Mesh FlowPM package, paving the way to perform field-level cosmological inference with joint velocity and density reconstruction. This is especially useful given the increased ability to measure peculiar velocities in the near future.

79 ASTRONOMY AND ASTROPHYSICS↗

DETERMINATION OF TRANSVERSE DENSITY STRUCTURING FROM PROPAGATING MAGNETOHYDRODYNAMIC WAVES IN THE SOLAR ATMOSPHERE

We present a Bayesian seismology inversion technique for propagating magnetohydrodynamic transverse waves observed in coronal waveguides. The technique uses theoretical predictions for the spatial damping of propagating kink waves in transversely inhomogeneous coronal waveguides. It combines wave amplitude damping length scales along the waveguide with theoretical results for resonantly damped propagating kink waves to infer the plasma density variation across the oscillating structures. Provided that the spatial dependence of the velocity amplitude along the propagation direction is measured and the existence of two different damping regimes is identified, the technique would enable us to fully constrain the transverse density structuring, providing estimates for the density contrast and its transverse inhomogeneity length scale.

79 ASTRONOMY AND ASTROPHYSICS↗

Nuclear masses learned from a probabilistic neural network

Machine learning methods and uncertainty quantification have been gaining interest throughout the last several years in low-energy nuclear physics. In particular, Gaussian processes and Bayesian neural networks have increasingly been applied to improve mass model predictions while providing well-quantified uncertainties. In this work, we use the probabilistic Mixture Density Network (MDN) to directly predict the mass excess of the 2016 Atomic Mass Evaluation within the range of measured data, and we extrapolate the inferred models beyond available experimental data. The MDN provides not only mean values but also full posterior distributions both within the training set and extrapolated testing set. We show that the addition of physical information to the feature space increases the accuracy of the match to the training data as well as provides for more physically meaningful extrapolations beyond the the limits of experimental data.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗

Variation-Resilient FeFET-Based In-Memory Computing Leveraging Probabilistic Deep Learning

Reliability issues stemming from device level nonidealities of nonvolatile emerging technologies like ferroelectric field-effect transistors (FeFETs), especially at scaled dimensions, cause substantial degradation in the accuracy of in-memory crossbar-based AI systems. Here, in this work, we present a variation-aware design technique to characterize the device level variations and to mitigate their impact on hardware accuracy employing a Bayesian neural network (BNN) approach. An effective conductance variation model is derived from the experimental measurements of cycle-to-cycle (C2C) and device-to-device (D2D) variations performed on FeFET devices fabricated using 28 nm high-k metal gate technology. The variations were found to be a function of different conductance states within the given programming range, which sharply contrasts earlier efforts where a fixed variation dispersion was considered for all conductance values. Such variation characteristics formulated for three different device sizes at different read voltages were provided as prior variation information to the BNN to yield a more exact and reliable inference. Near-ideal accuracy for shallow networks (MLP5 and LeNet models) on the MNIST dataset and limited accuracy decline by ~3.8%–16.1% for deeper AlexNet models on CIFAR10 dataset under a wide range of variations corresponding to different device sizes and read voltages, demonstrates the efficacy of our proposed device-algorithm co-design technique.

97 MATHEMATICS AND COMPUTING↗

Tokamak divertor plasma emulation with machine learning

Abstract Future tokamak devices that aim to create conditions relevant to power plant operations must consider strategies for mitigating damage to plasma facing components in the divertor. One of the goals of MAST-U tokamak operations is to inform these considerations by researching advanced divertor configurations that aid stable plasma detachment. Machine design, scenario planning and detachment control would all greatly benefit from tools that enable rapid calculation of scenario-relevant quantities given some input parameters. This paper presents a method for generating large, simulated scrape-off layer data sets, which was applied to generate a data set of steady-state Hermes-3 simulations of the MAST-U tokamak. A machine learning model was constructed using a Bayesian approach to hyperparameter optimisation to predict diagnosable output quantities given control-relevant input features. The resulting best-performing model, which is based on a feedforward neural network, achieves high accuracy when predicting electron temperature at the divertor target and carbon impurity radiation front position and runs in around 1 ms in inference mode. Techniques for interpreting the predictions made by the model were applied, and a high-resolution parameter scan of upstream conditions was performed to demonstrate the utility of rapidly generating accurate predictions using the emulator. This work represents a step forward in the design of machine learning-driven emulators of tokamak exhaust simulation codes in operational modes relevant to divertor detachment control and plasma scenario design.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗

Methane emissions from dairies in the Los Angeles Basin

We estimate the amount of methane (CH 4 ) emitted by the largest dairies in the southern California region by combining measurements from four mobile solar-viewing ground-based spectrometers (EM27/SUN), in situ isotopic 13/12 CH 4 measurements from a CRDS analyzer (Picarro), and a high-resolution atmospheric transport simulation with a Weather Research and Forecasting model in large-eddy simulation mode (WRF-LES). The remote sensing spectrometers measure the total column-averaged dry-air mole fractions of CH 4 and CO 2 (X CH 4 and X CO 2 ) in the near infrared region, providing information on total emissions of the dairies at Chino. Differences measured between the four EM27/SUN ranged from 0.2 to 22ppb (part per billion) and from 0.7 to 3ppm (part per million) for X CH 4 and X CO 2 , respectively. To assess the fluxes of the dairies, these differential measurements are used in conjunction with the local atmospheric dynamics from wind measurements at two local airports and from the WRF-LES simulations at 111m resolution. Our top-down CH 4 emissions derived using the Fourier transform spectrometers (FTS) observations of 1.4 to 4.8ppts -1 are in the low end of previous top-down estimates, consistent with reductions of the dairy farms and urbanization in the domain. However, the wide range of inferred fluxes points to the challenges posed by the heterogeneity of the sources and meteorology. Inverse modeling from WRF-LES is utilized to resolve the spatial distribution of CH 4 emissions in the domain. Both the model and the measurements indicate heterogeneous emissions, with contributions from anthropogenic and biogenic sources at Chino. A Bayesian inversion and a Monte Carlo approach are used to provide the CH 4 emissions of 2.2 to 3.5ppts -1 at Chino.

54 ENVIRONMENTAL SCIENCES↗

The Attraction Indian Buffet Distribution

We propose the attraction Indian buffet distribution (AIBD), a distribution for binary feature matrices influenced by pairwise similarity information. Binary feature matrices are used in Bayesian models to uncover latent variables (i.e., features) that explain observed data. The Indian buffet process (IBP) is a popular exchangeable prior distribution for latent feature matrices. In the presence of additional information, however, the exchangeability assumption is not reasonable or desirable. The AIBD can incorporate pairwise similarity information, yet it preserves many properties of the IBP, including the distribution of the total number of features. Thus, much of the interpretation and intuition that one has for the IBP directly carries over to the AIBD. A temperature parameter controls the degree to which the similarity information affects feature-sharing between observations. Unlike other nonexchangeable distributions for feature allocations, the probability mass function of the AIBD has a tractable normalizing constant, making posterior inference on hyperparameters straight-forward using standard MCMC methods. A novel posterior sampling algorithm is proposed for the IBP and the AIBD. We demonstrate the feasibility of the AIBD as a prior distribution in feature allocation models and compare the performance of competing methods in simulations and an application.

97 MATHEMATICS AND COMPUTING↗

An exponential–gamma mixture model for extreme Santa Ana winds

We analyze the behavior of extreme winds occurring in Southern California during the Santa Ana wind season using a latent mixture model. This mixture representation is formulated as a hierarchical Bayesian model and fit using Markov chain Monte Carlo. The two‐stage model results in generalized Pareto margins for exceedances and generates temporal dependence through a latent Markov process. This construction induces asymptotic independence in the response, while allowing for dependence at extreme, but subasymptotic, levels. We compare this model with a frequentist analogue where inference is performed via maximum pairwise likelihood. We use interval censoring to account for data quantization and estimate the extremal index and probabilities of multiday occurrences of extreme Santa Ana winds over a range of high thresholds.

Bopp, Gregory P.↗