Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Bayesian Inference”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,135 records · Page 63

Jensen–Shannon divergence based novel loss functions for Bayesian neural networks

Bayesian neural networks (BNNs) are state-of-the-art machine learning methods that can naturally regularize and systematically quantify uncertainties using their stochastic parameters. Kullback–Leibler (KL) divergence-based variational inference used in BNNs suffer from unstable optimization and challenges in approximating light-tailed posteriors due to the unbounded nature of the KL divergence. To resolve these issues, we formulate a novel loss function for BNNs based on a new modification to the generalized Jensen–Shannon (JS) divergence, which is bounded. In addition, we propose a Geometric JS divergence-based loss, which is computationally efficient since it can be evaluated analytically. We found that the JS divergence-based variational inference is intractable, and hence employed a constrained optimization framework to formulate these losses. Our theoretical analysis and empirical experiments on multiple regression and classification data sets suggest that the proposed losses perform better than the KL divergence-based loss, especially when the data sets are noisy or biased. Specifically, there are approximately 5% and 8% improvements in accuracy for a noise-added CIFAR-10 dataset and a regression dataset, respectively. There is about 13% reduction in false negative predictions of a biased histopathology dataset. Additionally, we quantify and compare the uncertainty metrics for the regression and classification tasks.

97 MATHEMATICS AND COMPUTING↗

A Bayesian Modeling Approach for Estimation of a Shape-Free Groundwater Age Distribution using Multiple Tracers

The mixing of groundwaters with different ages in aquifers, groundwater age is more appropriately represented by a distribution rather than a scalar number. To infer a groundwater age distribution from environmental tracers, a mathematical form is often assumed for the shape of the distribution and the parameters of the mathematical distribution are estimated using deterministic or stochastic inverse methods. We found that the prescription of the mathematical form limits the exploration of the age distribution to the shapes that can be described by the selected distribution. In this paper, the use of freeform histograms as groundwater age distributions is evaluated. A Bayesian Markov Chain Monte Carlo approach is used to estimate the fraction of groundwater in each histogram bin. This method was able to capture the shape of a hypothetical gamma distribution from the concentrations of four age tracers. The number of bins that can be considered in this approach is limited based on the number of tracers available. The histogram method was also tested on tracer data sets from Holten (The Netherlands; 3 H, 3 He, 85 Kr, 39 Ar) and the La Selva Biological Station (Costa-Rica; SF 6 , CFCs, 3 H, 4 He and 14 C), and compared to a number of mathematical forms. According to standard Bayesian measures of model goodness, the best mathematical distribution performs better than the histogram distributions in terms of the ability to capture the observed tracer data relative to their complexity. Among the histogram distributions, the four bin histogram performs better in most of the cases. The Monte Carlo simulations showed strong correlations in the posterior estimates of bin contributions, indicating that these bins cannot be well constrained using the available age tracers. The fact that mathematical forms overall perform better than the freeform histogram does not undermine the benefit of the freeform approach, especially for the cases where a larger amount of observed data is available and when the real groundwater distribution is more complex than can be represented by simple mathematical forms.

54 ENVIRONMENTAL SCIENCES↗

Improving Trustworthiness of Data-Driven Power Grid Contingency Analysis With Bayesian Residual Graph Neural Networks

The evolving energy landscape requires novel tools to efficiently perform contingency analysis and reliability assessment of power grids, potentially in real-time. The high computational cost of traditional power flow solvers limits their applicability in practice. Machine learning (ML) surrogates such as deep neural networks (NNs) accelerate power flow solvers computations, enabling high-order contingency analysis and real-time decision-making by learning highly nonlinear functions and integrating grid topology via graph architectures. However, (graph) NNs lack predictive power away from training data and do not provide predictive confidence estimates. Here, we present a Bayesian residual graph NN that integrates knowledge from low-fidelity data via residual training and embeds granular quantification of uncertainties, improving trustworthiness critical for high-consequence decision-making. Applying Bayesian concepts to NNs is challenging due to the high-dimensionality of both the parameter space, complicating derivation of a meaningful prior, and the output space in large grid systems, requiring enhanced techniques to assess the predicted high-dimensional uncertainties. Our contributions include: (1) Deriving a prior for fully connected and graph NNs that leverages low-fidelity data to guide mean predictions and appropriately control prior predictive uncertainty. (2) Integrating this prior within an ensembling with anchoring scheme for efficient approximate posterior inference. (3) Deriving enhanced metrics to assess accuracy of both the mean and uncertainty predictions in high dimensions, appropriately accounting for correlations propagated through graph layers. The resulting Bayesian residual graph NN is tested on a contingency analysis task for 14-bus and 118-bus grids.

24 - POWER TRANSMISSION AND DISTRIBUTION↗

Hierarchical Model for Bayesian Hypothesis Testing

A common problem in data analysis is testing a point hypothesis. For example, one might test whether or not the mean of a distribution is zero. We describe the standard Bayesian approach, which requires the choice of a width parameter to define a prior distribution for an alternative hypothesis. Standard practice is to choose this width parameter to be the least favorable choice for the original, point hypothesis. This approach, however, is ad-hoc and not informed by the data. We examine this practice in more detail than in previous work by constructing a hierarchical model, where the width parameter is an unknown value subject to inference. This yields a marginal posterior distribution for the width parameter. We then show that using the median of this marginal posterior distribution as the width parameter in the Bayesian comparison between the hypotheses gives results which are very similar to those obtained from the existing approaches. This gives a more rigorous and data-informed justification for the standard practice of choosing a width parameter which is least favorable to the point hypothesis.

97 MATHEMATICS AND COMPUTING↗

Acceptance sampling using judgmental and randomly selected samples

We present a Bayesian model for acceptance sampling where the population consists of two groups, each with different levels of risk of containing unacceptable items. Expert opinion, or judgment, may be required to distinguish between the high and low-risk groups. Hence, high-risk items are likely to be identifed (and sampled) using expert judgment, while the remaining low-risk items are sampled randomly. We focus on the situation where all observed samples must be acceptable. Consequently, the objective of the statistical inference is to quantify the probability that a large percentage of the unsampled items in the population are also acceptable. We demonstrate that traditional (frequentist) acceptance sampling and simpler Bayesian formulations of the problem are essentially special cases of the proposed model. We explore the properties of the model in detail, and discuss the conditions necessary to ensure that required samples sizes are non-decreasing function of the population size. The method is applicable to a variety of acceptance sampling problems, and, in particular, to environmental sampling where the objective is to demonstrate the safety of reoccupying a remediated facility that has been contaminated with a lethal agent.

54 ENVIRONMENTAL SCIENCES↗

System risk quantification and decision making support using functional modeling and dynamic Bayesian network

Risk-informed decision-making requires a probabilistic assessment of the likelihood of success of control action, given the system status. This paper presents a systematic state transition modeling approach integrating dynamic probabilistic risk assessment with a decision-making process using a dynamic Bayesian network (DBN) coupled with functional modeling. A functional model designed with multilevel flow modeling (MFM) technique was used to build a system state structure inferred by energy, mass, and information flow so that one can verify the developed model with respect to system functionality. The MFM model represents the causal relationship among the nodes, which captures the structure of process parameters and control units. Each node may have multiple possible states, and the DBN structured by the MFM model represents the time-domain transitions among the defined states. Furthermore, the MFM-DBN integrated state transition modeling is a white-box approach that allows one to draw the system's risk profile by updating the system states and supports the decisions probabilistically with physical inference. An example of a simple heating system has been used to illustrate this process, including decision-making support based on quantitative risk profile. For demonstrating its applicability to a complex system operational decision making, a case study of station blackout accident scenario leading to the seal loss of coolant accident in a nuclear power plant is presented. The proposed approach effectively provided the risk profile along time for each option so that the operators can make the best decision, which minimizes the plant risk.

42 ENGINEERING↗

Bayesian Framework For Bioburden Density Calculations To Perform Planetary Protection Probabilistic Risk Assessment

The planetary protection discipline aims to minimize the microbial contamination on spacecraft to prevent the inadvertent contamination of other planetary bodies, known as forward planetary protection (PP). Planetary protection probabilistic risk assessment (PRA) relies on two core methodologies-the contamination probability event tree analysis and statistical parameter estimation. Planetary protection engineers combine several techniques to estimate the bioburden present on spacecraft components. A direct assay to enumerate CFU (colony forming units) is the preferred methodology, but given a similar processing environment the bioburden present on certain components is inferred using: (1) a NASA defined bioburden estimate based upon the biological cleanliness of the manufacturing/assembly environment or (2) sampled data from a similar spacecraft component. The paper presents an empirical Bayesian framework to systematically treat bioburden estimation and its uncertainties on different levels starting with measurement procedures to combining different components to subsystems and whole spacecraft. It is shown that the Bayesian approach can effectively handle estimations and their uncertainties at different levels and produce a reliable estimate for bioburden to be used to evaluate the probability of contamination.

Seuylemezian, Arman↗

Direct nonparametric multimessenger constraints on the equation of state of cold dense nuclear matter

We utilize the now substantial amount of astrophysical observations of neutron stars (NSs), along with perturbative quantum chromodynamics (pQCD) calculations at high density, to directly constrain the NS equation of state (EOS). To this end, we construct nonparametric EOS priors by using Gaussian processes trained on 75 EOSs, which include models with either hadrons, hyperons, or quarks at high densities. We create a prior using the full EOS sample (model agnostic), and one prior for each EOS family to test model discrimination. We introduce a novel inference approach, which allows the simultaneous sampling of intrinsic and extrinsic parameters of binary NS mergers, as well as a nonparametric equation of state. We showcase this method in a Bayesian updating scheme by first performing a complete analysis of the binary NS merger event GW170817 with minimal assumptions, and sequentially adding information from x-ray and radio NS observations, along with pQCD calculations. Besides providing standard constraints, such as the pressure at twice nuclear saturation density 𝑝⁡(2⁢𝜌 sat ) = 4.3$^{+0.6}_{−0.6}$ × 10 34 dyne/cm 2 , at 95% confidence level, for the model agnostic prior, our methodology shows how the choice of EOS families used in conditioning changes the inferred astrophysical properties of the EOS, namely tidal deformability and maximum supported NS mass. We find hyperonic priors predicting higher tidal deformabilities for a 1.4⁢𝑀 ⊙ NS, and hadronic priors being preferred by the considered astrophysical data.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗

PINN surrogate of Li-ion battery models for parameter inference, Part I: Implementation and multi-fidelity hierarchies for the single-particle model

To plan and optimize energy storage demands that account for Li-ion battery aging dynamics, techniques need to be developed to diagnose battery internal states accurately and rapidly. Here, this study seeks to reduce the computational resources needed to determine a battery's internal states by replacing physics-based Li-ion battery models - such as the single-particle model (SPM) and the pseudo-2D (P2D) model - with a physics-informed neural network (PINN) surrogate. The surrogate model makes high-throughput techniques, such as Bayesian calibration, tractable to determine battery internal parameters from voltage responses. This manuscript is the first of a two-part series that introduces PINN surrogates of Li-ion battery models for parameter inference (i.e., state-of-health diagnostics). In this first part, a method is presented for constructing a PINN surrogate of the SPM. A multi-fidelity hierarchical training, where several neural nets are trained with multiple physics-loss fidelities is shown to significantly improve the surrogate accuracy when only training on the governing equation residuals. The implementation is made available in a companion repository (https://github.com/NREL/PINNSTRIPES). The techniques used to develop a PINN surrogate of the SPM are extended in Part II for the PINN surrogate for the P2D battery model, and explore the Bayesian calibration capabilities of both surrogates.

25 ENERGY STORAGE↗

GEOMETRIC AND DYNAMICAL MODELS OF REVERBERATION MAPPING DATA

We present a general method to analyze reverberation (or echo) mapping data that simultaneously provides estimates for the black hole mass and for the geometry and dynamics of the broad-line region (BLR) in active galactic nuclei (AGNs). While previous methods yield a typical scale size of the BLR or a reconstruction of the transfer function, our method directly infers the spatial and velocity distribution of the BLR from the data, from which a transfer function can be easily derived. Previous echo mapping analysis requires an independent estimate of a scaling factor known as the virial coefficient to infer the mass of the black hole, but this is not needed in our more direct approach. We use the formalism of Bayesian probability theory and implement a Markov Chain Monte Carlo algorithm to obtain estimates and uncertainties for the parameters of our BLR models. Fitting of models to the data requires knowledge of the continuum flux at all times, not just the measured times. We use Gaussian Processes to interpolate and extrapolate the continuum light curve data in a fully consistent probabilistic manner, taking the associated errors into account. We illustrate our method using simple models of BLR geometry and dynamics and show that we can recover the parameter values of our test systems with realistic uncertainties that depend upon the variability of the AGN and the quality of the reverberation mapping observing campaign. With a geometry model we can recover the mean radius of the BLR to within {approx}0.1 dex random uncertainty for simulated data with an integrated line flux uncertainty of 1.5%, while with a dynamical model we can recover the black hole mass and the mean radius to within {approx}0.05 dex random uncertainty, for simulated data with a line profile average signal-to-noise ratio of 4 per spectral pixel. These uncertainties do not include modeling errors, which are likely to be present in the analysis of real data, and should therefore be considered as lower limits to the accuracy of the method.

79 ASTRONOMY AND ASTROPHYSICS↗

Active Galactic Nuclei Continuum Reverberation Mapping Based on Zwicky Transient Facility Light Curves

We perform a systematic survey of active galactic nuclei (AGNs) continuum lags using ~3 days cadence gri-band light curves from the Zwicky Transient Facility. We select a sample of 94 type 1 AGNs at z < 0.8 with significant and consistent inter-band lags based on the interpolated cross-correlation function method and the Bayesian method JAVELIN. Within the framework of the "lamp-post" reprocessing model, our findings are: (1) The continuum emission (CE) sizes inferred from the data are larger than the disk sizes predicted by the standard thin-disk model. (2) For a subset of the sample, the CE size exceeds the theoretical limit of the self-gravity radius (12 lt-days) for geometrically thin disks. (3) The CE size scales with continuum luminosity as R CE ∝ L 0.48±0.04 with a scatter of 0.2 dex, analogous to the well-known radius–luminosity relation of broad Hβ. These findings suggest a significant contribution of diffuse continuum emission from the broad-line region (BLR) to AGN continuum lags. We find that the R CE –L relation can be explained by a photoionization model that assumes ~23% of the total flux comes from the diffuse BLR emission. In addition, the ratio of the CE size and model-predicted disk size anticorrelates with the continuum luminosity, which is indicative of a potential nondisk BLR lag contribution evolving with the luminosity. Finally, a robust positive correlation between the CE size and black hole mass is detected.

79 ASTRONOMY AND ASTROPHYSICS↗

Rule groupings in expert systems using nearest neighbour decision rules, and convex hulls

Expert System shells are lacking in many areas of software engineering. Large rule based systems are not semantically comprehensible, difficult to debug, and impossible to modify or validate. Partitioning a set of rules found in CLIPS (C Language Integrated Production System) into groups of rules which reflect the underlying semantic subdomains of the problem, will address adequately the concerns stated above. Techniques are introduced to structure a CLIPS rule base into groups of rules that inherently have common semantic information. The concepts involved are imported from the field of A.I., Pattern Recognition, and Statistical Inference. Techniques focus on the areas of feature selection, classification, and a criteria of how 'good' the classification technique is, based on Bayesian Decision Theory. A variety of distance metrics are discussed for measuring the 'closeness' of CLIPS rules and various Nearest Neighbor classification algorithms are described based on the above metric.

Anastasiadis, Stergios↗

Bond Line Thickness Estimation in Composite Structures Using Multiple Inspection Techniques

Imaging and other nondestructive evaluation techniques are commonly used for material characterization and defect recognition in safety critical aerospace applications, with data fusion providing the framework for uncertainty quantification in these contexts. Most commonly, forward physics-based modeling predicts the response conditioned on material properties and defect assumptions, and probabilistic methods are used to infer the hidden state of subject of the inspection from a combination of prior information, likelihoods, and inspection data. In this paper Bayesian methods are used to estimate bond thickness in lap joints comprised of aluminum adherends using a combination of infrared thermography, digital radiography, and ultrasound. The accuracy of the fused estimates are validated against data generated from synthetic specimens, and by comparison against high resolution X-ray computed tomography inspections of built specimens.

thermal nondestructive evaluation↗

Simultaneous global and local clustering in multiplex networks with covariate information

Understanding both global and layer-specific group structures is useful for uncovering complex patterns in networks with multiple interaction types. In this work, we introduce a new model, the hierarchical multiplex stochastic blockmodel, which simultaneously detects communities within individual layers of a multiplex network while inferring a global node clustering across the layers. A stochastic blockmodel is assumed in each layer, with probabilities of layer-level group memberships determined by a node’s global group assignment. Our model uses a Bayesian framework, employing a probit stick-breaking process to construct node-specific mixing proportions over a set of shared Griffiths–Engen–McCloseky distributions. These proportions determine layer-level community assignment, allowing for an unknown and varying number of groups across layers, while incorporating nodal covariate information to inform the global clustering. We propose a scalable variational inference procedure with parallelisable updates for application to large networks. Extensive simulation studies demonstrate our model’s ability to accurately recover both global and layer-level clusters in complicated settings, and applications to real data showcase the model’s effectiveness in uncovering interesting latent network structure.

community detection↗

TDCOSMO - IV. Hierarchical time-delay cosmography – joint inference of the Hubble constant and galaxy density profiles

The H0LiCOW collaboration inferred via strong gravitational lensing time delays a Hubble constant value of H0 = 73.3−1.8+1.7 km s−1 Mpc−1, describing deflector mass density profiles by either a power-law or stars (constant mass-to-light ratio) plus standard dark matter halos. The mass-sheet transform (MST) that leaves the lensing observables unchanged is considered the dominant source of residual uncertainty in H0. We quantify any potential effect of the MST with a flexible family of mass models, which directly encodes it, and they are hence maximally degenerate with H0. Our calculation is based on a new hierarchical Bayesian approach in which the MST is only constrained by stellar kinematics. The approach is validated on mock lenses, which are generated from hydrodynamic simulations. We first applied the inference to the TDCOSMO sample of seven lenses, six of which are from H0LiCOW, and measured H0 = 74.5−6.1+5.6 km s−1 Mpc−1. Secondly, in order to further constrain the deflector mass density profiles, we added imaging and spectroscopy for a set of 33 strong gravitational lenses from the Sloan Lens ACS (SLACS) sample. For nine of the 33 SLAC lenses, we used resolved kinematics to constrain the stellar anisotropy. From the joint hierarchical analysis of the TDCOSMO+SLACS sample, we measured H0 = 67.4−3.2+4.1 km s−1 Mpc−1. This measurement assumes that the TDCOSMO and SLACS galaxies are drawn from the same parent population. The blind H0LiCOW, TDCOSMO-only and TDCOSMO+SLACS analyses are in mutual statistical agreement. The TDCOSMO+SLACS analysis prefers marginally shallower mass profiles than H0LiCOW or TDCOSMO-only. Without relying on the form of the mass density profile used by H0LiCOW, we achieve a ∼5% measurement of H0. While our new hierarchical analysis does not statistically invalidate the mass profile assumptions by H0LiCOW – and thus the H0 measurement relying on them – it demonstrates the importance of understanding the mass density profile of elliptical galaxies. The uncertainties on H0 derived in this paper can be reduced by physical or observational priors on the form of the mass profile, or by additional data.Key words: gravitational lensing: strong / galaxies: general / galaxies: kinematics and dynamics / distance scale / cosmological parameters / cosmology: observations⋆ The full analysis is available at https://github.com/TDCOSMO/hierarchy_analysis_2020_public.

79 ASTRONOMY AND ASTROPHYSICS↗

The tale of a modern animal plague: Tracing the evolutionary history and determining the time-scale for foot and mouth disease virus

Despite significant advances made in the understanding of its epidemiology, foot and mouth disease virus (FMDV) is among the most unexpected agricultural devastating plagues. While the disease manifests itself as seven immunologically distinct strains their origin, population dynamics, migration patterns and divergence times remain unknown. Herein we have assembled a comprehensive data set of gene sequences representing the global diversity of the disease and inferred the time-scale and evolutionary history for FMDV. Serotype-specific rates of evolution and divergence times were estimated using a Bayesian coalescent framework. We report that an ancient precursor FMDV gave rise to two major diversification events spanning a relatively short interval of time. This radiation event is estimated to have taken place towards the end of the 17th and the beginning of the 18th century giving us the present circulating Euro-Asiatic and South African viral strains. Furthermore our results hint that Europe acted as a possible hub for the disease from where it successfully dispersed elsewhere via exploration and trading routes.

60 APPLIED LIFE SCIENCES↗

Bond Line Thickness Estimation in Composite Structures Using Multiple Inspection Techniques

Imaging and other nondestructive evaluation techniques are commonly used for material characterization and defect recognition in safety critical aerospace applications, with data fusion providing the framework for uncertainty quantification in these contexts. Most commonly, forward physics-based modeling predicts the response conditioned on material properties and defect assumptions, and probabilistic methods are used to infer the hidden state of the subject of the inspection from a combination of prior information, likelihoods, and inspection data. In this paper Bayesian methods are used to estimate bond thickness in lap joints comprised of aluminum adherends using a combination of infrared thermography and ultrasound. The concept of the conflation of probability distributions is applied to combine the posterior distributions derived from thermography and ultrasound and the quality of the fused estimates are compared against the individual estimates against synthetic data that was created to mimic the inspection of a lap joint comprised of aluminum adherends.

thermal nondestructive evaluation↗

Bayesian hierarchical models for soil CO{sub 2} flux and leak detection at geologic sequestration sites

Proper characterizations of background soil CO{sub 2} respiration rates are critical for interpreting CO{sub 2} leakage monitoring results at geologic sequestration sites. In this paper, a method is developed for determining temperature-dependent critical values of soil CO{sub 2} flux for preliminary leak detection inference. The method is illustrated using surface CO{sub 2} flux measurements obtained from the AmeriFlux network fit with alternative models for the soil CO{sub 2} flux versus soil temperature relationship. The models are fit first to determine pooled parameter estimates across the sites, then using a Bayesian hierarchical method to obtain both global and site-specific parameter estimates. Model comparisons are made using the deviance information criterion (DIC), which considers both goodness of fit and model complexity. The hierarchical models consistently outperform the corresponding pooled models, demonstrating the need for site-specific data and estimates when determining relationships for background soil respiration. A hierarchical model that relates the square root of the CO{sub 2} flux to a quadratic function of soil temperature is found to provide the best fit for the AmeriFlux sites among the models tested. This model also yields effective prediction intervals, consistent with the upper envelope of the flux data across the modeled sites and temperature ranges. Calculation of upper prediction intervals using the proposed method can provide a basis for setting critical values in CO{sub 2} leak detection monitoring at sequestration sites.

54 ENVIRONMENTAL SCIENCES↗