Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “BAYESIAN”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 343 records · Page 19
All sourcesNASA NTRSDOE OSTI
DOE OSTI2018

Bayesian Design of Experiments for Logistic Regression to Evaluate Multiple Forensic Algorithms

When evaluating the performance of several forensic classification algorithms, it is desirable to construct a design that considers a variety of performance levels for each of the algorithms. We describe a strategy to use Bayesian design of experiments with multiple prior estimates to capture anticipated performance. Our goal is to characterize results from the different classification algorithms as a function of multiple explanatory variables and use this to choose a design about which units to test. Bayesian design of experiments has been successful for generalized linear models, including logistic regression models. Here, we develop methodology for the case where there are several potentially nonoverlapping priors for anticipated performance under consideration. The weighted priors method performs well for a broad range of true underlying model parameter choices and is more robust when compared to other candidate design choices. Additionally, we show how this can be applied in the multivariate input case and provide some useful summary measures. The shared information plot is used to evaluate design point allocation, and the D-value difference plot allows for the comparison of design performance across multiple potential parameter values in higher dimensions. Here, we illustrate the methods with several examples.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2020

CheKiPEUQ Intro 2: Harnessing Uncertainties from Data Sets, Bayesian Design of Experiments in Chemical Kinetics**

When choosing experimental conditions, Bayesian statistical tools can predict the experimental choices which will yield the highest information gain. Experimental choices could be temperature, pressure, reaction time, number of measurements, reactor volume, etc.. Three example analyses are presented here, each using the software Chemical Kinetics Parameter Estimation and Uncertainty Quantification (CheKiPEUQ). Information gain is a measure of reduction of uncertainty in a model's parameters. The three chemical system examples presented each illustrate Bayesian Design of Experiments using information gain. In the first chemical example, temperature selection impacts the information gain for the free energy of reaction in a two-component equilibrium reaction. In the second example, temperature and pressure are explored for a competitive adsorption Langmuir replacement reaction system. Finally, the third example is a catalytic membrane reactor which is a culmination of the previous examples. The catalytic membrane reactor has a complex and nonlinear response in the observables which is solved by numerical evaluation. In the three examples, the experimental conditions are treated as design variables for maximizing information gain.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2013

Probabilistic assessment of cloud fraction using Bayesian blending of independent datasets: Feasibility study of a new method

We describe and evaluate a novel method to blend two observed cloud fraction (CF) datasets through Bayesian posterior estimation. The research reported here is a feasibility study designed to explore the method. In this proof‐of‐concept study, we illustrate the approach using specific observational datasets from the U. S. Department of Energy Atmospheric Radiation Measurement Program's Southern Great Plains site in the central United States, but the method is quite general and is readily applicable to other datasets. The total sky image (TSI) camera observations are used to determine the prior distribution. A regression model and the active remote sensing of clouds (ARSCL) radar/lidar observations are used to determine the likelihood function. The posterior estimate is a probability density function (pdf) of the CF whose mean is taken to be the optimal blend of the two observations. The data at hourly, daily, 5‐day, monthly, and annual time scales are considered. Some physical and probabilistic properties of the CFs are explored from radar/lidar, camera, and satellite observations and from simulations using the Community Atmosphere Model (CAM5). Our results imply that (a) the Beta distribution is a reasonable model for CF for both short‐ and long‐time means, the 5‐day data are skewed right, and the annual data are almost normally distributed, and (b) the Bayesian method developed successfully yields a pdf of CF, rather than a deterministic CF value, and it is feasible to blend the TSI and ARSCL data with a capability for bias correction.

58 GEOSCIENCES↗
DOE OSTI2016

Using Bayesian variable selection to analyze regular resolution IV two-level fractional factorial designs

Regular two-level fractional factorial designs have complete aliasing in which the associated columns of multiple effects are identical. Here, we show how Bayesian variable selection can be used to analyze experiments that use such designs. In addition to sparsity and hierarchy, Bayesian variable selection naturally incorporates heredity . This prior information is used to identify the most likely combinations of active terms. We also demonstrate the method on simulated and real experiments.

42 ENGINEERING↗
DOE OSTI2021

An initial exploration of Bayesian model calibration for estimating the composition of rocks and soils on Mars

The Mars Curiosity rover carries an instrument, ChemCam, designed to measure the composition of surface rocks and soil using laser-induced breakdown spectroscopy (LIBS). The measured spectra from this instrument must be analyzed to identify the component elements in the target sample, as well as their relative proportions. This process, which we call disaggregation, is complicated by so-called matrix effects, which describe nonlinear changes in the relative heights of emission lines as an unknown function of composition due to atomic interactions within the LIBS plasma. In this work, we explore the use of the plasma physics code ATOMIC, developed at Los Alamos National Laboratory, for the disaggregation task. ATOMIC has recently been used to model LIBS spectra and can robustly reproduce matrix effects from first principles. The ability of ATOMIC to predict LIBS spectra presents an exciting opportunity to perform disaggregation in a manner not yet tried in the LIBS community, namely via Bayesian model calibration. However, using it directly to solve our inverse problem is computationally intractable due to the large parameter space and the computation time required to produce a single output. Therefore, we also explore the use of emulators as a fast solution for this analysis. We discuss a proof of concept Gaussian process emulator for disaggregating two-element compounds of sodium and copper. The training and test datasets were simulated with ATOMIC using a Latin hypercube design. After testing the performance of the emulator, we successfully recover the composition of 25 test spectra with Bayesian model calibration.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2018

Soil moisture estimation using tomographic ground penetrating radar in a MCMC–Bayesian framework

In this paper, we focus on a hydrogeological inverse problem specifically targeting monitoring soil moisture variations using tomographic ground penetrating radar (GPR) travel time data. Technical challenges exist in the inversion of GPR tomographic data for handling non-uniqueness, nonlinearity and high-dimensionality of unknowns. We have developed a new method for estimating soil moisture fields from crosshole GPR data. It uses a pilot-point method to provide a low-dimensional representation of the relative dielectric permittivity field of the soil, which is the primary object of inference: the field can be converted to soil moisture using a petrophysical model. We integrate a multi-chain Markov chain Monte Carlo (MCMC)–Bayesian inversion framework with the pilot point concept, a curved-ray GPR travel time model, and a sequential Gaussian simulation algorithm, for estimating the dielectric permittivity at pilot point locations distributed within the tomogram, as well as the corresponding geostatistical parameters (i.e., spatial correlation range). We infer the dielectric permittivity as a probability density function, thus capturing the uncertainty in the inference. The multi-chain MCMC enables addressing high-dimensional inverse problems as required in the inversion setup. The method is scalable in terms of number of chains and processors, and is useful for computationally demanding Bayesian model calibration in scientific and engineering problems. The proposed inversion approach can successfully approximate the posterior density distributions of the pilot points, and capture the true values. The computational efficiency, accuracy, and convergence behaviors of the inversion approach were also systematically evaluated, by comparing the inversion results obtained with different levels of noises in the observations, increased observational data, as well as increased number of pilot points.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2018

An efficient Bayesian data-worth analysis using a multilevel Monte Carlo method

Improving the understanding of subsurface systems and thus reducing prediction uncertainty requires collection of data. As the collection of subsurface data is costly, it is important that the data collection scheme is cost-effective. Design of a cost-effective data collection scheme, i.e., data-worth analysis, requires quantifying model parameter, prediction, and both current and potential data uncertainties. Assessment of these uncertainties in large-scale stochastic subsurface hydrological model simulations using standard Monte Carlo (MC) sampling or surrogate modeling is extremely computationally intensive, sometimes even infeasible. In this paper, we propose an efficient Bayesian data-worth analysis using a multilevel Monte Carlo (MLMC) method. Compared to the standard MC that requires a significantly large number of high-fidelity model executions to achieve a prescribed accuracy in estimating expectations, the MLMC can substantially reduce computational costs using multifidelity approximations. Since the Bayesian data-worth analysis involves a great deal of expectation estimation, the cost saving of the MLMC in the assessment can be outstanding. While the proposed MLMC-based data-worth analysis is broadly applicable, we use it for a highly heterogeneous two-phase subsurface flow simulation to select an optimal candidate data set that gives the largest uncertainty reduction in predicting mass flow rates at four production wells. Finally, the choices made by the MLMC estimation are validated by the actual measurements of the potential data, and consistent with the standard MC estimation. But compared to the standard MC, the MLMC greatly reduces the computational costs.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2025

Karhunen–Loève deep learning method for surrogate modeling and approximate Bayesian parameter estimation

We evaluate the performance of the Karhunen-Loève Deep Neural Network (KL-DNN) framework for surrogate modeling and approximate Bayesian parameter estimation in partial differential equation models. In the surrogate model, the Karhunen-Loève (KL) expansions are used for the dimensionality reduction of the number of unknown parameters and variables, and a deep neural network is employed to relate the reduced space of parameters to that of the state variables. The KL-DNN surrogate model is used to formulate a maximum-a-posteriori-like least-squares problem, which is randomized to draw samples of the posterior distribution of the parameters. We test the proposed framework for a hypothetical unconfined aquifer via comparison with the forward MODFLOW and inverse PEST++ iterative ensemble smoother (IES) solutions as well as the state-of-the-art Fourier neural operator (FNO) and deep operator networks (DeepONets) operator learning surrogate models. Our results show that the KL-DNN surrogate model outperforms FNO and DeepONet for forward predictions. For solving inverse problems, the randomized algorithm provides the same or more accurate Bayesian predictions of the parameters than IES as evidenced by the higher log-predictive probability of both the estimated parameter field and the forecast hydraulic head. The posterior mean obtained from the randomized algorithm is closer to the reference parameter field than that obtained with FNO as the maximum a posteriori estimate.

Approximate Bayesian inference↗
DOE OSTI2020

Constrained Bayesian optimization of criticality experiments

The design of criticality experiments is typically an iterative process that employs a Monte Carlo transport code. The goal is to find a design that optimizes some variable, like the sensitivity of a response to a cross section, while simultaneously ensuring criticality. The high fidelity of the Monte Carlo code is a great asset, but it makes exploring the design space computationally expensive. Herein, we present how a constrained Bayesian optimization algorithm can be used to efficiently design a criticality experiment. It uses Gaussian processes as a surrogate model to probe the design space and to reduce the number of code executions that are needed to find the optimum. Furthermore, we demonstrate constrained Bayesian optimization with a Pu-239/polyethylene solution system and a TEX experiment that is designed for criticality safety validation of a nuclear waste model at the Hanford Site. For both systems, a global optimum was found within 75 Monte Carlo simulations.

42 ENGINEERING↗
DOE OSTI2019

An anomaly detection framework for dynamic systems using a Bayesian hierarchical framework

Complex systems are susceptible to many types of anomalies, faults, and abnormal behavior caused by a variety of off-nominal conditions that may ultimately result in major failures or catastrophic events. Early and accurate detection of these anomalies using system inputs and outputs collected from sensors and smart devices has become a challenging problem and an active area of research in many application domains. In this article, we present a new Bayesian hierarchical framework that is able to model the relationship between system inputs (sensor measurements) and outputs (response variables) without imposing strong distributional/parametric assumptions while using only a subset of training samples and sensor attributes. Then, an optimal cost-sensitive anomaly detection framework is proposed to determine whether a sample is an anomalous one taking into consideration the trade-off between misclassification errors and detection rates. The model can be used for both supervised and unsupervised settings depending on the availability of data regarding the behavior of the system under anomaly conditions. The unsupervised model is particularly useful when it is prohibitive to identify in advance the anomalies that a system may present and where no data are available regarding the behavior of the system under anomaly conditions. A Bayesian hierarchical setting is used to structure the proposed framework and help with accommodating uncertainty, imposing interpretability, and controlling the sparsity and complexity of the proposed anomaly detection framework. A Markov chain Monte Carlo algorithm is also developed for model training using past data. Furthermore, the numerical experiments conducted using a simulated data set and a wind turbine data set demonstrate the successful application of the proposed work for system response modeling and anomaly detection.

17 WIND ENERGY↗
DOE OSTI2025

Explainable multi-fidelity Bayesian neural network for distribution system state estimation

Distribution System State Estimation (DSSE) is frequently constrained by limited real-time measurements, the uncertainties introduced by distributed energy resources, and the presence of bad data. To address them, this paper proposes an enhanced Multi-Fidelity Bayesian Neural Network (MFBNN) DSSE approach. A low-fidelity layer based on a Deep Neural Network (DNN) is first pre-trained on pseudo-measurement data to learn fundamental state features. Subsequently, a high-fidelity Bayesian Neural Network (BNN) layer leverages limited but high-quality real-time measurements to refine these features, thereby achieving accurate DSSE. Additionally, the deep SHapley Additive exPlanation (SHAP) is developed to quantify the influence of measurement data on DSSE through dual perspectives of global feature importance and local nodal contributions, establishing a hierarchical explainability framework for machine learning-based DSSE. Comparative studies conducted on the IEEE 13-bus system and a real-world 2135-node system from Dominion Energy demonstrate that the proposed method excels in estimation accuracy, even under situations of high noise levels, bad data, and missing data. Further comparisons with Weighted Least Squares (WLS) and other machine learning-based DSSE approaches verify that the proposed framework offers higher accuracy, improved interpretability, and enhanced robustness.

Bad data↗
DOE OSTI2021

Bayesian inference of heterogeneous epidemic models: Application to COVID-19 spread accounting for long-term care facilities

Here we propose a high dimensional Bayesian inference framework for learning heterogeneous dynamics of a COVID-19 model, with a specific application to the dynamics and severity of COVID-19 inside and outside long-term care (LTC) facilities. We develop a heterogeneous compartmental model that accounts for the heterogeneity of the time-varying spread and severity of COVID-19 inside and outside LTC facilities, which is characterized by time-dependent stochastic processes and time-independent parameters in ~ 1500 dimensions after discretization. To infer these parameters, we use reported data on the number of confirmed, hospitalized, and deceased cases with suitable post-processing in both a deterministic inversion approach with appropriate regularization as a first step, followed by Bayesian inversion with proper prior distributions. To address the curse of dimensionality and the ill-posedness of the high-dimensional inference problem, we propose use of a dimension-independent projected Stein variational gradient descent method, and demonstrate the intrinsic low-dimensionality of the inverse problem. We present inference results with quantified uncertainties for both New Jersey and Texas, which experienced different epidemic phases and patterns. Moreover, we also present forecasting and validation results based on the empirical posterior samples of our inference for the future trajectory of COVID-19.

42 ENGINEERING↗
DOE OSTI2021

Bayesian learning of orthogonal embeddings for multi-fidelity Gaussian Processes

Uncertainty propagation in complex engineering systems often poses significant computational challenges related to modeling and quantifying probability distributions of model outputs, as those emerge as the result of various sources of uncertainty that are inherent in the system under investigation. Gaussian Processes regression (GPs) is a robust meta-modeling technique that allows for fast model prediction and exploration of response surfaces. Multi-fidelity variations of GPs further leverage information from cheap and low fidelity model simulations in order to improve their predictive performance on the high fidelity model. In order to cope with the high volume of data required to train GPs in high dimensional design spaces, a common practice is to introduce latent design variables that are typically projections of the original input space to a lower dimensional subspace, and therefore substitute the problem of learning the initial high dimensional mapping, with that of training a GP on a low dimensional space. Here in this paper, we present a Bayesian approach to identify optimal transformations that map the input points to low dimensional latent variables. The \projection" mapping consists of an orthonormal matrix that is considered a priori unknown and needs to be inferred jointly with the GP parameters, conditioned on the available training data. The proposed Bayesian inference scheme relies on a two-step iterative algorithm that samples from the marginal posteriors of the GP parameters and the projection matrix respectively, both using Markov Chain Monte Carlo (MCMC) sampling. In order to take into account the orthogonality constraints imposed on the orthonormal projection matrix, a Geodesic Monte Carlo sampling algorithm is employed, that is suitable for exploiting probability measures on manifolds. We extend the proposed framework to multi-fidelity models using GPs including the scenarios of training multiple outputs together. We validate our framework on three synthetic problems with a known lower-dimensional subspace. The benefits of our proposed framework, are illustrated on the computationally challenging aerodynamic optimization of a last-stage blade for an industrial gas turbine, where we study the effect of an 85-dimensional shape parameterization of a three-dimensional airfoil on two output quantities of interest, specifically on the aerodynamic efficiency and the degree of reaction

42 ENGINEERING↗
DOE OSTI2025

Accelerating Hamiltonian Monte Carlo for Bayesian inference in neural networks and neural operators

Hamiltonian Monte Carlo (HMC) is a powerful and accurate method to sample from the posterior distribution in Bayesian inference. However, HMC techniques are computationally demanding for Bayesian neural networks due to the high dimensionality of the network’s parameter space and the non-convexity of their posterior distributions. Therefore, various approximation techniques, such as variational inference (VI) or stochastic gradient MCMC, are often employed to infer the posterior distribution of the network parameters. Such approximations introduce inaccuracies in the inferred distributions, resulting in unreliable uncertainty estimates. In this work, we propose a hybrid approach that combines inexpensive VI and accurate HMC methods to efficiently and accurately quantify uncertainties in neural networks and neural operators. The proposed approach leverages an initial VI training on the full network. We examine the influence of individual parameters on the prediction uncertainty, which shows that a large proportion of the parameters do not contribute substantially to uncertainty in the network predictions. This information is then used to significantly reduce the dimension of the parameter space, and HMC is performed only for the subset of network parameters that strongly influence prediction uncertainties. This yields a framework for accelerating the full batch HMC for posterior inference in neural networks. We demonstrate the efficiency and accuracy of the proposed framework on deep neural networks and operator networks, showing that inference can be performed for large networks with tens to hundreds of thousands of parameters. Finally, we show that this method can effectively learn surrogates for complex physical systems by modeling the operator that maps from upstream conditions to wall-pressure data on a cone in hypersonic flow.

Bayesian inference↗
DOE OSTI2019

A comparison of material flow strength models using Bayesian cross-validation

Predicting material flow strength over a range of conditions, such as temperature and strain rate, is necessary for many engineering applications. In this paper, we consider how to compare the predictiveness of several different strength models using a statistical technique called Bayesian cross-validation. Given a dataset of flow strength measurements obtained from mechanictesting experiments, the procedure consists of performing a Bayesian calibration of each strength model on a subset of the data and evaluating how well the trained models predict the remaining data. The predictiveness of a calibrated strength model is quantifiable probabilistically, which provides an interpretable metric for comparing the different models. As an illustrative example, we compare the Johnson-Cook (JC), Zerilli-Armstrong (ZA), Preston-Tonks-Wallace (PTW), and Mechanical Threshold Stress (MTS) flow strength models for the tantalum stress-strain curve data from Chen and Gray (1996). We show that prediction intervals for the four strength models cover the held-out data at most experimental conditions, but also that prediction interval coverage and prediction uncertainty varies by model and experimental condition. Finally, the analysis further allows us to identify experimental regimes for which one of the strength models predicts better than the other three.

36 MATERIALS SCIENCE↗
DOE OSTI2018

A numerical Bayesian-calibrated characterization method for multiscale prepreg preforming simulations with tension-shear coupling

We report that carbon fiber reinforced plastics (CFRPs) are attracting growing attention in industry because of their enhanced properties. Preforming of thermoset carbon fiber prepregs is one of the most common production techniques of CFRPs. To simulate preforming, several computational methods have been developed. Most of these methods, however, obtain the material properties directly from experiments such as uniaxial tension and bias-extension where the coupling effect between tension and shear is not considered. Neglecting this coupling effect deteriorates the prediction accuracy of simulations. To address this issue, we develop a Bayesian model calibration and material characterization approach in a multiscale finite element preforming simulation framework that utilizes mesoscopic representative volume element (RVE) to account for the tension-shear coupling. A new geometric modeling technique is first proposed to generate the RVE corresponding to the close-packed uncured prepreg. This RVE model is then calibrated with a modular Bayesian approach to estimate the yarn properties, test its potential biases against the experiments, and fit a stress emulator. Finally, the predictive capability of this multiscale approach is further demonstrated by employing the stress emulator in the macroscale preforming simulation which shows that this approach can provide accurate predictions.

36 MATERIALS SCIENCE↗
DOE OSTI2021

Multimodal Bayesian registration of noisy functions using Hamiltonian Monte Carlo

Functional data registration is a necessary processing step for many applications. The observed data can be inherently noisy, often due to measurement error or natural process uncertainty; which most functional alignment methods cannot handle. A pair of functions can also have multiple optimal alignment solutions, which is not addressed in current literature. In this paper, a flexible Bayesian approach to functional alignment is presented, which appropriately accounts for noise in the data without any pre-smoothing required. Additionally, by running parallel MCMC chains, the method can account for multiple optimal alignments via the multi-modal posterior distribution of the warping functions. To most efficiently sample the warping functions, the approach relies on a modification of the standard Hamiltonian Monte Carlo to be well-defined on the infinite-dimensional Hilbert space. In this work, this flexible Bayesian alignment method is applied to both simulated data and real data sets to show its efficiency in handling noisy functions and successfully accounting for multiple optimal alignments in the posterior; characterizing the uncertainty surrounding the warping functions.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2024

Machine learning assisted bayesian inference of mix and hot-spot conditions in NIF implosions

Experiments on the National Ignition Facility (NIF) have provided clear evidence of ablator material mixing into the Hot-Spot, leading to degraded performance. However, inferring the amount of mix and Hot-Spot conditions from typical experimental observations (e.g. x-ray spectra and images) is highly challenging. Here, we have developed an analysis method that utilizes machine learning assisted Bayesian inference to find the probability distributions of the Hot-Spot and mix conditions. This approach uses a neural network, trained on an idealized 2-dimensional representation of the Hot-Spot and mix distribution, and Bayesian inference to find the statistical distributions of Hot-Spot conditions that provide a match with observations. We have tested this method with synthetic data from simulations.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗