Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Bayesian Inference”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,261 records · Page 70

The Sensitivity of Variational Bayesian Neural Network Performance to Hyperparameters

In scientific applications, predictive modeling is often of limited use without accurate uncertainty quantification (UQ) to indicate when a model may be extrapolating or when more data needs to be collected. Bayesian Neural Networks (BNNs) produce predictive uncertainty by propagating uncertainty in neural network (NN) weights and offer the promise of obtaining not only an accurate predictive model but also accurate UQ. However, in practice, obtaining accurate UQ with BNNs is difficult due in part to the approximations used for model training (such as those made in variational inference) and in part to the need to choose a suitable set of hyperparameters; these hyperparameters outnumber those needed for traditional NNs and often have opaque effects on the results. We aim to shed light on the effects of hyperparameter choices for variational BNNs by performing a global sensitivity analysis of variational BNN performance under varying hyperparameter settings. Our results indicate that many of the hyperparameters interact with each other to affect both predictive accuracy and UQ. For improved usage of variational BNNs in real-world applications, we suggest that thorough hyperparameter tuning, including tuning of prior hyperparameters and loss function parameters, is essential for accurate UQ in variational BNNs.

97 MATHEMATICS AND COMPUTING↗

The statistical analysis of circadian phase and amplitude in constant-routine core-temperature data

Accurate estimation of the phases and amplitude of the endogenous circadian pacemaker from constant-routine core-temperature series is crucial for making inferences about the properties of the human biological clock from data collected under this protocol. This paper presents a set of statistical methods based on a harmonic-regression-plus-correlated-noise model for estimating the phases and the amplitude of the endogenous circadian pacemaker from constant-routine core-temperature data. The methods include a Bayesian Monte Carlo procedure for computing the uncertainty in these circadian functions. We illustrate the techniques with a detailed study of a single subject's core-temperature series and describe their relationship to other statistical methods for circadian data analysis. In our laboratory, these methods have been successfully used to analyze more than 300 constant routines and provide a highly reliable means of extracting phase and amplitude information from core-temperature data.

NASA Discipline Regulatory Physiology↗

Inferring the Gibbs state of a small quantum system

Gibbs states are familiar from statistical mechanics, yet their use is not limited to that domain. For instance, they also feature in the maximum entropy reconstruction of quantum states from incomplete measurement data. Outside the macroscopic realm, however, estimating a Gibbs state is a nontrivial inference task, due to two complicating factors: the proper set of relevant observables might not be evident a priori; and whenever data are gathered from a small sample only, the best estimate for the Lagrange parameters is invariably affected by the experimenter's prior bias. I show how the two issues can be tackled with the help of Bayesian model selection and Bayesian interpolation, respectively, and illustrate the use of these Bayesian techniques with a number of simple examples.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

Parameter and Structure Inference for Nonlinear Dynamical Systems

A great many systems can be modeled in the non-linear dynamical systems framework, as x = f(x) + xi(t), where f() is the potential function for the system, and xi is the excitation noise. Modeling the potential using a set of basis functions, we derive the posterior for the basis coefficients. A more challenging problem is to determine the set of basis functions that are required to model a particular system. We show that using the Bayesian Information Criteria (BIC) to rank models, and the beam search technique, that we can accurately determine the structure of simple non-linear dynamical system models, and the structure of the coupling between non-linear dynamical systems where the individual systems are known. This last case has important ecological applications.

Morris, Robin D.↗

Determining frequentist confidence limits using a directed parameter space search

We consider the problem of inferring constraints on a high-dimensional parameter space with a computationally expensive likelihood function. We propose a machine learning algorithm that maps out the Frequentist confidence limit on parameter space by intelligently targeting likelihood evaluations so as to quickly and accurately characterize the likelihood surface in both low- and high-likelihood regions. We compare our algorithm to Bayesian credible limits derived by the well-tested Markov Chain Monte Carlo (MCMC) algorithm using both multi-modal toy likelihood functions and the seven yr Wilkinson Microwave Anisotropy Probe cosmic microwave background likelihood function. We find that our algorithm correctly identifies the location, general size, and general shape of high-likelihood regions in parameter space while being more robust against multi-modality than MCMC.

79 ASTRONOMY AND ASTROPHYSICS↗

Assessing correlated truncation errors in modern nucleon-nucleon potentials

We test the BUQEYE model of correlated effective field theory (EFT) truncation errors on Reinert, Krebs, and Epelbaum's semilocal momentum-space implementation of the chiral EFT (𝜒⁢EFT ) expansion of the nucleon-nucleon (NN) potential. This Bayesian model hypothesizes that dimensionless coefficient functions extracted from the order-by-order corrections to NN observables can be treated as draws from a Gaussian process (GP). We combine a variety of graphical and statistical diagnostics to assess when predicted observables have a 𝜒⁢EFT convergence pattern consistent with the hypothesized GP statistical model. Our conclusions are that, first, the BUQEYE model is generally applicable to the potential investigated here, which enables statistically principled estimates of the impact of higher EFT orders on observables. Second, parameters defining the extracted coefficients such as the expansion parameter 𝑄 must be well chosen for the coefficients to exhibit a regular convergence pattern—a property we exploit to obtain posterior distributions for such quantities. Third, the assumption of GP stationarity across lab energy and scattering angle is not generally met; this necessitates adjustments in future work. We provide a workflow and interpretive guide for our analysis framework, and show what can be inferred about probability distributions for 𝑄, the EFT breakdown scale Λ 𝑏 , the scale associated with soft physics in the 𝜒⁢EFT potential 𝑚 eff , and the GP hyperparameters. All our results can be reproduced using a publicly available Jupyter notebook, which can be straightforwardly modified to analyze other 𝜒⁢EFT NN potentials.

Bayesian methods↗

Unmixing hyperspectral images using Markov random fields

This paper proposes a new spectral unmixing strategy based on the normal compositional model that exploits the spatial correlations between the image pixels. The pure materials (referred to as endmembers) contained in the image are assumed to be available (they can be obtained by using an appropriate endmember extraction algorithm), while the corresponding fractions (referred to as abundances) are estimated by the proposed algorithm. Due to physical constraints, the abundances have to satisfy positivity and sum-to-one constraints. The image is divided into homogeneous distinct regions having the same statistical properties for the abundance coefficients. The spatial dependencies within each class are modeled thanks to Potts-Markov random fields. Within a Bayesian framework, prior distributions for the abundances and the associated hyperparameters are introduced. A reparametrization of the abundance coefficients is proposed to handle the physical constraints (positivity and sum-to-one) inherent to hyperspectral imagery. The parameters (abundances), hyperparameters (abundance mean and variance for each class) and the classification map indicating the classes of all pixels in the image are inferred from the resulting joint posterior distribution. To overcome the complexity of the joint posterior distribution, Markov chain Monte Carlo methods are used to generate samples asymptotically distributed according to the joint posterior of interest. Simulations conducted on synthetic and real data are presented to illustrate the performance of the proposed algorithm.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

Cellular, molecular, and therapeutic characterization of pilocarpine-induced temporal lobe epilepsy

Animal models have expanded our understanding of temporal lobe epilepsy (TLE). However, translating these to cell-specific druggable hypotheses is not explored. Herein, we conducted an integrative insilico-analysis of an available transcriptomics dataset obtained from animals with pilocarpine-induced-TLE. A set of 119 genes with subtle-to-moderate impact predicted most forms of epilepsy with ~ 97% accuracy and characteristically mapped to upregulated homeostatic and downregulated synaptic pathways. The deconvolution of cellular proportions revealed opposing changes in diverse cell types. The proportion of nonneuronal cells increased whereas that of interneurons, except for those expressing vasoactive intestinal peptide (Vip), decreased, and pyramidal neurons of the cornu-ammonis (CA) subfields showed the highest variation in proportion. A probabilistic Bayesian-network demonstrated an aberrant and oscillating physiological interaction between nonneuronal cells involved in the blood–brain-barrier and Vip interneurons in driving seizures, and their role was evaluated insilico using transcriptomic changes induced by valproic-acid, which showed opposing effects in the two cell-types. Additionally, we revealed novel epileptic and antiepileptic mechanisms and predicted drugs using causal inference, outperforming the present drug repurposing approaches. These well-powered findings not only expand the understanding of TLE and seizure oscillation, but also provide predictive biomarkers of epilepsy, cellular and causal micro-circuitry changes associated with it, and a drug-discovery method focusing on these events.

60 APPLIED LIFE SCIENCES↗

3D Sedimentological and geophysical studies of clastic reservoir analogs: Facies architecture, reservoir properties, and flow behavior within delta front facies elements of the Cretaceous Wall Creek Member, Frontier Formation, Wyoming

Significant volumes of oil and gas occur in reservoirs formed by ancient river deltas. This has implications for the spatial distribution of rock types and the variation of transport properties. A between mudstones and sandstones may form baffles that influence productivity and recovery efficiency. Diagenetic processes such as compaction, dissolution, and cementation can also alter flow properties. A better understanding of these properties and improved methods will allow improved reservoir development planning and increased recovery of oil and gas from deltaic reservoirs. Surface exposures of ancient deltaic rocks provide a high-resolution view of variability. Insights gleaned from these exposures can be used to model analogous reservoirs, for which data is sparser. The Frontier Formation in central Wyoming provides an opportunity for high-resolution models. The same rocks exposed in the Tisdale anticline are productive in nearby oil fields. Kilometers of exposure are accessible, and bedding-plane exposures allow use of high-resolution ground-penetrating radar. This study combined geologic interpretations, maps, vertical sections, core data, and ground-penetrating radar to construct geostatistical and flow models. Strata-conforming grids were use to reproduce the observed geometries. A new Bayesian method integrates outcrop, core, and radar amplitude and phase data. The proposed method propagates measurement uncertainty and yields an ensemble of plausible models for calcite concretions. These concretions affect flow significantly. Models which integrate more have different flow responses from simpler models, as demonstrated an exhaustive two-dimensional reference image and in three dimensions. This method is simple to implement within widely available geostatistics packages. Significant volumes of oil and gas occur in reservoirs that are inferred to have been formed by ancient river deltas. This geologic setting has implications for the spatial distribution of rock types (\Eg sandstones and mudstones) and the variation of transport properties (\Eg permeability and porosity) within bodies of a particular rock type. Both basin-wide processes such as sea-level change and the autocyclicity of deltaic processes commonly cause deltaic reservoirs to have large variability in rock properties; in particular, alternations between mudstones and sandstones may form baffles and trends in rock body permeability can influence productivity and recovery efficiency. In addition, diagenetic processes such as compaction, dissolution, and cementation can alter the spatial pattern of flow properties. A better understanding of these properties, and improved methods to model the properties and their effects, will allow improved reservoir development planning and increased recovery of oil and gas from deltaic reservoirs. Surface exposures of ancient deltaic rocks provide a high resolution, low uncertainty view of subsurface variability. Patterns and insights gleaned from these exposures can be used to model analogous reservoirs, for which data is much sparser. This approach is particularly attractive when reservoir formations are exposed at the surface. The Frontier Formation in central Wyoming provides an opportunity for high resolution characterization. The same rocks exposed in the vicinity of the Tisdale anticline are productive in nearby oil fields, including Salt Creek. Many kilometers of good-quality exposure are accessible, and the common bedding-plane exposures allow use of shallow-penetration, high-resolution electromagnetic methods known as ground-penetrating radar. This study combined geologic interpretations, maps, vertical sections, core data, and ground-penetrating radar to construct high-resolution geostatistical and flow models for the Wall Creek Member of the Frontier Formation. Stratal-conforming grids were use to reproduce the progradational and aggradational geometries observed in outcrop and radar data. A new, Bayesian method integrates outcrop--derived statistics, core observations of concretions, and radar amplitude and phase data. The proposed method consistently propagates measurement uncertainty through the model-building process, and yields an ensemble of plausible models for diagenetic calcite concretions. These concretions have a statistically significant on flow. Furthermore, neither geostatistical data from the outcrops nor geophysical data from radar is sufficient: models which integrate these data have significantly different flow responses. This was demonstrated both for an exhaustive two-dimensional reference image and in three dimensions, using flow simulations. This project wholly supported one PhD student and part of the education of an additional MS and PhD student. It helped to sponsor 6 refereed articles and 8 conference or similar presentations.

02 PETROLEUM↗

Modeling Images of Natural 3D Surfaces: Overview and Potential Applications

Generative models of natural images have long been used in computer vision. However, since they only describe the of 2D scenes, they fail to capture all the properties of the underlying 3D world. Even though such models are sufficient for many vision tasks a 3D scene model is when it comes to inferring a 3D object or its characteristics. In this paper, we present such a generative model, incorporating both a multiscale surface prior model for surface geometry and reflectance, and an image formation process model based on realistic rendering, the computation of the posterior model parameter densities, and on the critical aspects of the rendering. We also how to efficiently invert the model within a Bayesian framework. We present a few potential applications, such as asteroid modeling and Planetary topography recovery, illustrated by promising results on real images.

Jalobeanu, Andre↗

Bayesian classification of partially observed outbreaks using time-series data.

Results show that a time-series based classification may be possible. For the test cases considered, the correct model can be selected and the number of index case can be captured within {+-} {sigma} with 5-10 days of data. The low signal-to-noise ratio makes the classification difficult for small epidemics. The problem statement is: (1) Create Bayesian techniques to classify and characterize epidemics from a time-series of ICD-9 codes (will call this time-series a 'morbidity stream'); and (2) It is assumed the morbidity stream has already set off an alarm (through a Kalman filter anomaly detector) Starting with a set of putative diseases: Identify which disease or set of diseases 'fit the data best' and, Infer associated information about it, i.e. number of index cases, start time of the epidemic, spread rate, etc.

99 GENERAL AND MISCELLANEOUS↗

Probabilistic inference in very large universes

Our current favored cosmological theories allow for the striking and controversial possibility that the observable universe is just a small part of a much larger universe in which parameters that describe the effective, low-energy laws of physics vary from one region to another. The controversy is largely driven by the fact that such a “very large universe” is mostly observationally inaccessible to us, so the issue arises of how we can reasonably assess a theory that describes such a universe. In this paper, we propose a Bayesian method for theory assessment based on theory-generated probability distributions for our observations. We focus on the principles that define this method, leaving aside concerns about how, in practice, one would carry out the required calculations. (One important issue that we set aside is the measure problem.) We argue that cosmological theories can be tested by the standard method of Bayesian updating, but we need to use theoretical predictions for “first-person” probabilities—that is, probabilities that we should use for our observations, taking into account all relevant selection effects. These selection effects can vary from one observer to another and can vary with time, so, in principle, first-person probabilities are defined for each observer instant—an observer at a specific instant of time. Calculations of first-person probabilities should take into account everything that the observer believes about herself and her surroundings, which we refer to as her subjective state. If the universe is very large, a theory might predict that there are many observer instants in the same subjective state; we argue that first-person probabilities should be calculated using a principle of self-locating indifference (PSLI), the assumption that any real observer should make predictions for her future as if she were chosen randomly and uniformly from the theoretically predicted observer instants that share her subjective state. We believe the PSLI is intuitively very reasonable, but we also argue that, if the theory is correct, the use of this principle maximizes the expected fraction of observers who will make correct predictions. A further complication is that cosmological theories are not expected to fully predict the detailed properties of the universe, but rather will predict a set of possible universes, each with a probability. Different possible universes will generically have different numbers of observers. We argue that, in the calculation of first-person probabilities, the probability for each possible universe should be weighted by the number of observer instants in the specified subjective state that it contains. These issues have been controversial in the literature, so we also provide a rebuttal to the claim that principles like the PSLI involve a “selection fallacy”; a rebuttal to what we dub the principle of required certainty; an argument rejecting theories that predict a preponderance of Boltzmann brains; a rebuttal to a parable about humans and Jovians used by Hartle and Srednicki to argue that assumptions of typicality can lead to absurd consequences; and, finally, a discussion about how the use of “old evidence” can be fit into a Bayesian mold.

Azhar, Feraz [University of Notre Dame, IN (United↗

Flood Hazard Assessment from Storm Tides, Rain and Sea Level Rise for a Tidal River Estuary

Cities and towns along the tidal Hudson River are highly vulnerable to flooding through the combination of storm tides and high streamflows, compounded by sea level rise. Here a three-dimensional hydrodynamic model, validated by comparing peak water levels for 76 historical storms, is applied in a probabilistic flood hazard assessment. In simulations, the model merges streamflows and storm tides from tropical cyclones (TCs), offshore extratropical cyclones (ETCs) and inland "wet extratropical" cyclones (WETCs). The climatology of possible ETC and WETC storm events is represented by historical events (1931-2013), and simulations include gauged streamflows and inferred ungauged streamflows (based on watershed area) for the Hudson River and its tributaries. The TC climatology is created using a stochastic statistical model to represent a wider range of storms than is contained in the historical record. TC streamflow hydrographs are simulated for tributaries spaced along the Hudson, modeled as a function of TC attributes (storm track, sea surface temperature, maximum wind speed) using a statistical Bayesian approach. Results show WETCs are important to flood risk in the upper tidal river (e.g., Albany, New York), ETCs are important in the estuary (e.g., New York City) and lower tidal river, and TCs are important at all locations due to their potential for both high surge and extreme rainfall. The raising of floods by sea level rise is shown to be reduced by approximately 30-60 percent at Albany due to the dominance of streamflow for flood risk. This can be explained with simple channel flow dynamics, in which increased depth throughout the river reduces frictional resistance, thereby reducing the water level slope and the upriver water level.

Tidal river↗

HELIOS–RETRIEVAL: An Open-source, Nested Sampling Atmospheric Retrieval Code; Application to the HR 8799 Exoplanets and Inferred Constraints for Planet Formation

We present an open-source retrieval code named HELIOS–RETRIEVAL, designed to obtain chemical abundances and temperature–pressure profiles by inverting the measured spectra of exoplanetary atmospheres. In our forward model, we use an exact solution of the radiative transfer equation, in the pure absorption limit, which allows us to analytically integrate over all of the outgoing rays. Two chemistry models are considered: unconstrained chemistry and equilibrium chemistry (enforced via analytical formulae). The nested sampling algorithm allows us to formally implement Occam’s Razor based on a comparison of the Bayesian evidence between models. We perform a retrieval analysis on the measured spectra of the four HR 8799 directly imaged exoplanets. Chemical equilibrium is disfavored for HR 8799b and c. We find supersolar C/H and O/H values for the outer HR 8799b and c exoplanets, while the inner HR 8799d and e exoplanets have a range of C/H and O/H values. The C/O values range from being superstellar for HR 8799b to being consistent with stellar for HR 8799c and being substellar for HR 8799d and e. If these retrieved properties are representative of the bulk compositions of the exoplanets, then they are inconsistent with formation via gravitational instability (without late-time accretion) and consistent with a core accretion scenario in which late-time accretion of ices occurred differently for the inner and outer exoplanets. For HR 8799e, we find that spectroscopy in the K band is crucial for constraining C/O and C/H. HELIOS–RETRIEVAL is publicly available as part of the Exoclimes Simulation Platform (http://www.exoclime.org).

79 ASTRONOMY AND ASTROPHYSICS↗

Using discrete Bayesian networks for diagnosing and isolating cross-level faults in HVAC systems

Fault detection and diagnosis (FDD) technologies are critical to ensure satisfactory building performance, such as reducing energy wastes and negative impacts on occupant comfort and productivity. Existing FDD technologies mainly focus on component-level FDD solutions, which could lead to mis-diagnosis of cross-level faults in heating, ventilating, and air-conditioning (HVAC) systems. Cross-level faults are those faults that occur in one component or subsystem, but cause operational abnormalities in other components or subsystems, and result in a building level performance degradation. How to effectively diagnose the root cause of a cross-level fault is the focus of this study. Here, this paper presents a novel discrete Bayesian Network (DisBN)-based method for diagnosing cross-level faults in an HVAC system commonly used in commercial buildings. A two-level DisBN structure model is developed in this study. The parameters used in the DisBN model are obtained either from expert knowledge or through machine-learning strategies from normal system operation data. Meanwhile, the probability parameters are discretized to incorporate the uncertainties associated with typical expert knowledge. Thus, the developed DisBN method addresses the challenges many other BN based FDD methods face, i.e., the lack of fault data for BN parameter training. The developed DisBN represents causal relationships between a fault and its cross-level system impacts (i.e., fault symptoms or fault indicators) by considering how fault impacts propagate across different levels in an HVAC system. A weather and schedule information-based Pattern Matching (WPM) method is employed to automatically create WPM baseline data sets for each incoming real time snapshot data from the building systems. Consequently, BN inference and real-time diagnostics are achieved by comparing incoming snapshot data and the WPM baseline data set. The proposed method is evaluated using experimental fault data collected in a campus building. Fault diagnosis results demonstrate that the WPM-DisBN method is effective at locating the root causes of cross-level faults in an HVAC system.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗

Asteroid orbit determination using Bayesian probabilities

Bayesian a priori and a posteriori probability densities are employed to determine asteroid orbital elements from optical astrometric observations. For near-earth asteroids and Jupiter Trojans, the two-body approximation can yield grossly erroneous uncertainty estimates because of the scattering effect of close planetary encounters or prolonged weak planetary perturbations. For main-belt asteroids, the two-body approximation works efficiently, from which it is inferred that, although the ephemeris must be computed from an integrated orbit, only approximate partial derivatives are needed to attain sufficient accuracy in predicting ephemeris uncertainties. The semimajor and semiminor axes of the 2D 1-sigma sky-plane uncertainty ellipse during the years 1993 to 2005 in the many-body and two-body linear approximations are shown.

Muinonen, Karri↗

Ground Heat Flux Reconstruction Using Bayesian Uncertainty Quantification Machinery and Surrogate Modeling

Ground heat flux (G 0 ) is a key component of the land-surface energy balance of high-latitude regions. Despite its crucial role in controlling permafrost degradation due to global warming, G 0 is sparsely measured and not well represented in the outputs of global scale model simulation. In this study, an analytical heat transfer model is tested to reconstruct G 0 across seasons using soil temperature series from field measurements, Global Climate Model, and climate reanalysis outputs. The probability density functions of ground heat flux and of model parameters are inferred using available G 0 data (measured or modeled) for snow-free period as a reference. When observed G 0 is not available, a numerical model is applied using estimates of surface heat flux (dependent on parameters) as the top boundary condition. These estimates (and thus the corresponding parameters) are verified by comparing the distributions of simulated and measured soil temperature at several depths. Aided by state-of-the-art uncertainty quantification methods, the developed G 0 reconstruction approach provides novel means for assessing the probabilistic structure of the ground heat flux for regional permafrost change studies.

54 ENVIRONMENTAL SCIENCES↗

Cosmological Evidence Modelling: a new simulation-based approach to constrain cosmology on non-linear scales

Extracting accurate cosmological information from galaxy–galaxy and galaxy–matter correlation functions on non-linear scales (${\lesssim } 10 \, h^{-1}{\rm {Mpc}}$) requires cosmological simulations. Additionally, one has to marginalize over several nuisance parameters of the galaxy–halo connection. However, the computational cost of such simulations prohibits naive implementations of stochastic posterior sampling methods like Markov chain Monte Carlo (MCMC) that would require of order $\mathcal {O}(10^6)$ samples in cosmological parameter space. Several groups have proposed surrogate models as a solution: a so-called emulator is trained to reproduce observables for a limited number of realizations in parameter space. Afterwards, this emulator is used as a surrogate model in an MCMC analysis. In this work, we demonstrate a different method called Cosmological Evidence Modelling (CEM). First, for each simulation, we calculate the Bayesian evidence marginalized over the galaxy–halo connection by repeatedly populating the simulation with galaxies. We show that this Bayesian evidence is directly related to the posterior probability of cosmological parameters. Finally, we build a physically motivated model for how the evidence depends on cosmological parameters as sampled by the simulations. We demonstrate the feasibility of CEM by using simulations from the Aemulus simulation suite and forecasting cosmological constraints from BOSS CMASS measurements of redshift-space distortions. Our analysis includes exploration of how galaxy assembly bias affects cosmological inference. Overall, CEM has several potential advantages over the more common approach of emulating summary statistics, including the ability to easily marginalize over highly complex models of the galaxy–halo connection and greater accuracy, thereby reducing the number of simulations required.

79 ASTRONOMY AND ASTROPHYSICS↗