Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Bayesian model calibration”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 235 records · Page 13

A Graphical Approach to Diagnosing the Validity of the Conditional Independence Assumptions of a Bayesian Network Given Data

Bayesian networks have attained widespread use in data analysis and decision making. Well studied topics include: efficient inference, evidence propagation, parameter learning from data for complete and incomplete data scenarios, expert elicitation for calibrating Bayesian network probabilities, and structure learning. It is not uncommon for the researcher to assume the structure of the Bayesian network or to glean the structure from expert elicitation or domain knowledge. In this scenario, the model may be calibrated through learning the parameters from relevant data. There is a lack of work on model diagnostics for fitted Bayesian networks; this is the contribution of this paper. We key on the definition of (conditional) independence to develop a graphical diagnostic method which indicates if the conditional independence assumptions imposed when one assumes the structure of the Bayesian network are supported by the data. We develop the approach theoretically and describe a Monte Carlo method to generate uncertainty measures for the consistency of the data with conditional independence assumptions under the model structure. We describe how this theoretical information and the data are presented in a graphical diagnostic tool. We demonstrate the approach through data simulated from Bayesian networks under different conditional independence assumptions. We also apply the diagnostic to a real world data set. The results indicate that our approach is a reasonable way of visualizing and inspecting the conditional independence assumption of a Bayesian network given data.

Bayesian Networks · Structural Model Diagnostics ·↗

Bayesian calibration of bubble size dynamics applied to CO 2 gas fermenters

To accelerate the scale-up of gaseous CO 2 fermentation reactors, computational models need to predict gas-to-liquid mass transfer which requires capturing the bubble size dynamics, i.e. bubble breakup and coalescence. However, the applicability of existing models beyond air–water mixtures remains to be established. Here, an inverse modeling approach, accelerated with a neural network surrogate, calibrates the breakup and coalescence closure models, that are used in class methods for population balance modeling (PBM). The calibration is performed based on experimental results obtained in a CO 2 -air–water-coflowing bubble column reactor. Bayesian inference is used to account for noise in the experimental dataset and bias in the simulation results. To accurately capture gas holdup and interphase mass transfer, the results show that the breakage rate needs to be increased by one order of magnitude. In conclusion, the inferred model parameters are then used on a separate configuration and shown to also improve bubble size distribution predictions.

09 BIOMASS FUELS↗

A Multi-Objective Bayesian Optimization Approach Using the Weighted Tchebycheff Method

Abstract Bayesian optimization (BO) is a low-cost global optimization tool for expensive black-box objective functions, where we learn from prior evaluated designs, update a posterior surrogate Gaussian process model, and select new designs for future evaluation using an acquisition function. This research focuses upon developing a BO model with multiple black-box objective functions. In the standard multi-objective (MO) optimization problem, the weighted Tchebycheff method is efficiently used to find both convex and non-convex Pareto frontiers. This approach requires knowledge of utopia values before we start optimization. However, in the BO framework, since the functions are expensive to evaluate, it is very expensive to obtain the utopia values as a prior knowledge. Therefore, in this paper, we develop a MO-BO framework where we calibrate with multiple linear regression (MLR) models to estimate the utopia value for each objective as a function of design input variables; the models are updated iteratively with sampled training data from the proposed MO-BO. These iteratively estimated mean utopia values are used to formulate the weighted Tchebycheff MO acquisition function. The proposed approach is implemented in two numerical test examples and one engineering design problem of optimizing thin tube geometries under constant loading of temperature and pressure, with minimizing the risk of creep-fatigue failure and design cost, along with risk-based and manufacturing constraints. Finally, the model accuracy with frequentist, Bayesian and without MLR-based calibration are compared to true Pareto solutions.

Engineering↗

Monte Carlo Control Loops for Cosmic Shear Cosmology with DES Year 1

Weak lensing by large-scale structure is a powerful probe of cosmology and of the dark universe. This cosmic shear technique relies on the accurate measurement of the shapes and redshifts of background galaxies and requires precise control of systematic errors. The Monte Carlo Control Loops (MCCL) is a forward modelling method designed to tackle this problem. It relies on the Ultra Fast Image Generator (UFig) to produce simulated images tuned to match the target data statistically, followed by calibrations and tolerance loops. We present the first end-to-end application of this method, on the Dark Energy Survey (DES) Year 1 wide field imaging data. We simultaneously measure the shear power spectrum $C_{\ell}$ and the redshift distribution $n(z)$ of the background galaxy sample. The method includes maps of the systematic sources, Point Spread Function (PSF), an Approximate Bayesian Computation (ABC) inference of the simulation model parameters, a shear calibration scheme, and the fast estimation of the covariance matrix. We find a close statistical agreement between the simulations and the DES Y1 data using an array of diagnostics. In a non-tomographic setting, we derive a set of $C_\ell$ and $n(z)$ curves that encode the cosmic shear measurement, as well as the systematic uncertainty. Following a blinding scheme, we measure the combination of $\Omega_m$, $\sigma_8$, and intrinsic alignment amplitude $A_{\rm{IA}}$, defined as $S_8D_{\rm{IA}} = \sigma_8(\Omega_m/0.3)^{0.5}D_{\rm{IA}}$, where $D_{\rm{IA}}=1-0.11(A_{\rm{IA}}-1)$. We find $S_8D_{\rm{IA}}=0.895^{+0.054}_{-0.039}$, where systematics are at the level of roughly 60\% of the statistical errors. We discuss these results in the context of earlier cosmic shear analyses of the DES Y1 data. Our findings indicate that this method and its fast runtime offer good prospects for cosmic shear measurements with future wide-field surveys.

79 ASTRONOMY AND ASTROPHYSICS↗

Monte Carlo control loops for cosmic shear cosmology with DES Year 1 data

Weak lensing by large-scale structure is a powerful probe of cosmology and of the dark universe. This cosmic shear technique relies on the accurate measurement of the shapes and redshifts of background galaxies and requires precise control of systematic errors. Monte Carlo control loops (MCCL) is a forward modeling method designed to tackle this problem. It relies on the ultra fast image generator (UFig) to produce simulated images tuned to match the target data statistically, followed by calibrations and tolerance loops. Here, we present the first end-to-end application of this method, on the Dark Energy Survey (DES) Year 1 wide field imaging data. We simultaneously measure the shear power spectrum $C_ℓ$ and the redshift distribution $n(z)$ of the background galaxy sample. The method includes maps of the systematic sources, point spread function (PSF), an approximate Bayesian computation (ABC) inference of the simulation model parameters, a shear calibration scheme, and a fast method to estimate the covariance matrix. We find a close statistical agreement between the simulations and the DES Y1 data using an array of diagnostics. In a nontomographic setting, we derive a set of $C_ℓ$ and $n(z)$ curves that encode the cosmic shear measurement, as well as the systematic uncertainty. Following a blinding scheme, we measure the combination of $Ω_m$, $σ_8$, and intrinsic alignment amplitude $A_{IA}$, defined as $S_8D_{IA}=σ_8(Ω_m/0.3)^{0.5}D_{IA}$, where $D_{IA}=1-0.11(A_{IA}-1)$. We find $S_8D_{IA}=0.8954_{-0.039}^{+0.054}$, where systematics are at the level of roughly 60% of the statistical errors. We discuss these results in the context of earlier cosmic shear analyses of the DES Y1 data. Our findings indicate that this method and its fast runtime offer good prospects for cosmic shear measurements with future wide-field surveys.

79 ASTRONOMY AND ASTROPHYSICS↗

Application of Bayesian Model Selection for Metal Yield Models using ALEGRA and Dakota

This report introduces the concepts of Bayesian model selection, which provides a systematic means of calibrating and selecting an optimal model to represent a phenomenon. This has many potential applications, including for comparing constitutive models. The ideas described herein are applied to a model selection problem between different yield models for hardened steel under extreme loading conditions.

97 MATHEMATICS AND COMPUTING↗

ROSE: A reduced-order scattering emulator for optical models

A new generation of phenomenological optical potentials requires robust calibration and uncertainty quantification, motivating the use of Bayesian statistical methods. These Bayesian methods usually require calculating observables for thousands or even millions of parameter sets, making fast and accurate emulators highly desirable or even essential. Emulating scattering across different energies or with interactions such as optical potentials is challenging because of the nonaffine parameter dependence, meaning the parameters do not all factorize from individual operators. Here we introduce and demonstrate the reduced-order scattering emulator ( ROSE ) framework, a reduced basis emulator that can handle nonaffine problems. ROSE is fully extensible and works within the publicly available band framework software suite for calibration, model mixing, and experimental design. As a demonstration problem, we use ROSE to calibrate a realistic nucleon-target scattering model through the calculation of elastic cross sections. This problem shows the practical value of the ROSE framework for Bayesian uncertainty quantification with controlled trade-offs between emulator speed and accuracy as compared to high-fidelity solvers. Finally, planned extensions of ROSE are discussed.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗

Discrete-Direct Model Calibration and Uncertainty Propagation Method Confirmed on Multi-Parameter Plasticity Model Calibrated to Sparse Random Field Data

A discrete direct (DD) model calibration and uncertainty propagation approach is explained and demonstrated on a 4-parameter Johnson-Cook (J-C) strain-rate dependent material strength model for an aluminum alloy. The methodology's performance is characterized in many trials involving four random realizations of strain-rate dependent material-test data curves per trial, drawn from a large synthetic population. The J-C model is calibrated to particular combinations of the data curves to obtain calibration parameter sets which are then propagated to “Can Crush” structural model predictions to produce samples of predicted response variability. These are processed with appropriate sparse-sample uncertainty quantification (UQ) methods to estimate various statistics of response with an appropriate level of conservatism. This is tested on 16 output quantities (von Mises stresses and equivalent plastic strains) and it is shown that important statistics of the true variabilities of the 16 quantities are bounded with a high success rate that is reasonably predictable and controllable. The DD approach has several advantages over other calibration-UQ approaches like Bayesian inference for capturing and utilizing the information obtained from typically small numbers of replicate experiments in model calibration situations—especially when sparse replicate functional data are involved like force–displacement curves from material tests. The DD methodology is straightforward and efficient for calibration and propagation problems involving aleatory and epistemic uncertainties in calibration experiments, models, and procedures.

42 ENGINEERING↗

Crop physiology calibration in the CLM

Farming is using more of the land surface, as population increases and agriculture is increasingly applied for non-nutritional purposes such as biofuel production. This agricultural expansion exerts an increasing impact on the terrestrial carbon cycle. In order to understand the impact of such processes, the Community Land Model (CLM) has been augmented with a CLM-Crop extension that simulates the development of three crop types: maize, soybean, and spring wheat. The CLM-Crop model is a complex system that relies on a suite of parametric inputs that govern plant growth under a given atmospheric forcing and available resources. CLM-Crop development used measurements of gross primary productivity (GPP) and net ecosystem exchange (NEE) from AmeriFlux sites to choose parameter values that optimize crop productivity in the model. In this paper, we calibrate these parameters for one crop type, soybean, in order to provide a faithful projection in terms of both plant development and net carbon exchange. Calibration is performed in a Bayesian framework by developing a scalable and adaptive scheme based on sequential Monte Carlo (SMC). The model showed significant improvement of crop productivity with the new calibrated parameters. We demonstrate that the calibrated parameters are applicable across alternative years and different sites.

54 ENVIRONMENTAL SCIENCES↗

Bayesian calibration of strength parameters using hydrocode simulations of symmetric impact shock experiments of Al-5083

We report that predictive modeling of materials requires accurately parameterized constitutive models. Parameterizing models that describe dynamic strength and plasticity require experimentally probing materials in a variety of strain rate regimes. Some experimental protocols (e.g., plate impact) probe the constitutive response of a material using indirect measures such as free surface velocimetry. Manual efforts to parameterize constitutive models using indirect experimental measures often lead to non-unique optimizations without quantification of parameter uncertainty. This study uses a Bayesian statistical approach to find model parameters and to quantify the uncertainty of the resulting parameters. The technique is demonstrated by parameterizing the Johnson-Cook strength model for aluminum alloy 5083 by coupling hydrocode simulations and velocimetry measurements of a series of plate impact experiments. Simulation inputs and outputs are used to calibrate an emulator that mimics the outputs of the computationally intensive simulations. Varying the amount of experimental data available for emulator calibration showed clear differences in the degree of uncertainty and uniqueness of the resulting optimized Johnson-Cook parameters for Al-5083. The results of the optimization provided a numerical evaluation of the degree of confidence in model parameters and model performance. Lastly, given an understanding of the physical effects of certain model parameters, individual parameter uncertainty can be leveraged to quickly identify gaps in the physical domains covered by completed experiments.

36 MATERIALS SCIENCE↗

Validation Metrics for Fixed Effects and Mixed-Effects Calibration

The modern scientific process often involves the development of a predictive computational model. To improve its accuracy, a computational model can be calibrated to a set of experimental data. A variety of validation metrics can be used to quantify this process. Some of these metrics have direct physical interpretations and a history of use, while others, especially those for probabilistic data, are more difficult to interpret. In this work, a variety of validation metrics are used to quantify the accuracy of different calibration methods. Frequentist and Bayesian perspectives are used with both fixed effects and mixed-effects statistical models. Through a quantitative comparison of the resulting distributions, the most accurate calibration method can be selected. Two examples are included which compare the results of various validation metrics for different calibration methods. It is quantitatively shown that, in the presence of significant laboratory biases, a fixed effects calibration is significantly less accurate than a mixed-effects calibration. This is because the mixed-effects statistical model better characterizes the underlying parameter distributions than the fixed effects model. The results suggest that validation metrics can be used to select the most accurate calibration model for a particular empirical model with corresponding experimental data.

97 MATHEMATICS AND COMPUTING↗

Bayesian analysis of a future β decay experiment's sensitivity to neutrino mass scale and ordering

Bayesian modeling techniques enable sensitivity analyses that incorporate detailed expectations regarding future experiments. A model-based approach also allows one to evaluate inferences and predicted outcomes, by calibrating (or measuring) the consequences incurred when certain results are reported. In this work, we present procedures for calibrating predictions of an experiment's sensitivity to both continuous and discrete parameters. Using these procedures and a new Bayesian model of the β-decay spectrum, we assess a high-precision β-decay experiment's sensitivity to the neutrino mass scale and ordering for one assumed design scenario. We find that such an experiment could measure the electron-weighted neutrino mass within ~40 meV after 1 year (90% credibility). Neutrino masses > 500 meV could be measured within ≈5 meV. Using only β decay and external reactor neutrino data, we find that next-generation β-decay experiments could potentially constrain the mass ordering using a two-neutrino spectral model analysis. By calibrating mass ordering results, we identify reporting criteria that can be tuned to suppress false ordering claims. In some cases, a two-neutrino analysis can reveal that the mass ordering is inverted, an unobtainable result for the traditional one-neutrino analysis approach.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗

Model Calibration with Censored Data

Here, the purpose of model calibration is to make the model predictions closer to reality. The classical Kennedy-O'Hagan approach is widely used for model calibration, which can account for the inadequacy of the computer model while simultaneously estimating the unknown calibration parameters. In many applications, the phenomenon of censoring occurs when the exact outcome of the physical experiment is not observed, but is only known to fall within a certain region. In such cases, the Kennedy-O'Hagan approach cannot be used directly, and we propose a method to incorporate the censoring information when performing model calibration. The method is applied to study the compression phenomenon of liquid inside a bottle. The results show significant improvement over the traditional calibration methods, especially when the number of censored observations is large.

97 MATHEMATICS AND COMPUTING↗

Investigation of Ethane Dehydrogenation and Hydrogenolysis on Pt(111), Pt(211), and Pt(100): Bayesian Quantification and Correction of DFT-Based Enthalpic and Entropic Uncertainties

Computational investigations of heterogeneously catalyzed reactions using density functional theory (DFT) are often inaccurate, largely due to uncertainties in the choice of DFT functional (enthalpic uncertainty) and approximations for modeling adsorbate movement along the catalyst surface (entropic uncertainty). This work illustrates that both uncertainties are significant in the investigation of ethane dehydrogenation (EDH) and hydrogenolysis on Pt catalysts by considering the complete deconstruction of ethane on Pt(111), Pt(211), and Pt(100) using microkinetic modeling (MKM). Hence, this work uses both noncalibrated and Bayesian-calibrated MKMs to quantify and correct inaccuracies in macroscopic properties due to both uncertainties. A Bayesian approach to the correction of entropic errors was introduced using a “Modified Fermi Function (MFF)” to calibrate between the two bounds of entropy represented by the harmonic oscillator (HO) and free translator (FT) approximations. Regardless of enthalpic and entropic uncertainties, all three surfaces are capable of ethane activation; however, Pt(211) was found to be the most active and is largely responsible for methane production. Next, Pt(111) is largely responsible for acetylene production, and Pt(100) has the highest ethylene selectivity but is most susceptible to coking. By comparison of different calibrated models, the FT entropy approximation was found to better describe EDH under typical experimental conditions. Statistical evidence was found to support Pt(111) as the active site for EDH, assuming that one single site is responsible for the chemistry. On the three surfaces, competing second dehydrogenations to CH 2 CH 2 and CH 3 CH were observed as well as isomerization of CH 3 CH back to CH 2 CH 2 and deeper dehydrogenation of CH 3 CH. In conclusion, C–C cleavage was found to largely proceed via the CH 3 C intermediate on Pt(100) and Pt(111), while on Pt(211), it was via both CHC and CH 3 C.

Bayesian model selection↗

Exploring the contamination of the DES-Y1 cluster sample with SPT-SZ selected clusters

ABSTRACT We perform a cross validation of the cluster catalogue selected by the red-sequence Matched-filter Probabilistic Percolation algorithm (redMaPPer) in Dark Energy Survey year 1 (DES-Y1) data by matching it with the Sunyaev–Zel’dovich effect (SZE) selected cluster catalogue from the South Pole Telescope SPT-SZ survey. Of the 1005 redMaPPer selected clusters with measured richness $\hat{\lambda }\gt 40$ in the joint footprint, 207 are confirmed by SPT-SZ. Using the mass information from the SZE signal, we calibrate the richness–mass relation using a Bayesian cluster population model. We find a mass trend λ ∝ MB consistent with a linear relation (B ∼ 1), no significant redshift evolution and an intrinsic scatter in richness of σλ = 0.22 ± 0.06. By considering two error models, we explore the impact of projection effects on the richness–mass modelling, confirming that such effects are not detectable at the current level of systematic uncertainties. At low richness SPT-SZ confirms fewer redMaPPer clusters than expected. We interpret this richness dependent deficit in confirmed systems as due to the increased presence at low richness of low-mass objects not correctly accounted for by our richness-mass scatter model, which we call contaminants. At a richness $\hat{\lambda }=40$, this population makes up ${\gt}12{{\ \rm per\ cent}}$ (97.5 percentile) of the total population. Extrapolating this to a measured richness $\hat{\lambda }=20$ yields ${\gt}22{{\ \rm per\ cent}}$ (97.5 percentile). With these contamination fractions, the predicted redMaPPer number counts in different plausible cosmologies are compatible with the measured abundance. The presence of such a population is also a plausible explanation for the different mass trends (B ∼ 0.75) obtained from mass calibration using purely optically selected clusters. The mean mass from stacked weak lensing (WL) measurements suggests that these low-mass contaminants are galaxy groups with masses ∼3–5 × 1013 M⊙ which are beyond the sensitivity of current SZE and X-ray surveys but a natural target for SPT-3G and eROSITA.

79 ASTRONOMY AND ASTROPHYSICS↗

Posterior Covariance Matrix Approximations

Here, the Davis equation of state (EOS) is commonly used to model thermodynamic relationships for high explosive (HE) reactants. Typically, the parameters in the EOS are calibrated, with uncertainty, using a Bayesian framework and Markov Chain Monte Carlo (MCMC) methods. However, MCMC methods are computationally expensive, especially for complex models with many parameters. This paper provides a comparison between MCMC and less computationally expensive Variational methods (Variational Bayesian and Hessian Variational Bayesian) for computing the posterior distribution and approximating the posterior covariance matrix based on heterogeneous experimental data. All three methods recover similar posterior distributions and posterior covariance matrices. This study demonstrates that for this EOS parameter calibration application, the assumptions made in the two Variational methods significantly reduce the computational cost but do not substantially change the results compared to MCMC.

97 MATHEMATICS AND COMPUTING↗

Efficient Calibration of Expensive Computational Models

Accounting for uncertainty when calibrating expensive computational models is a common challenge faced by scientists and engineers. Often Bayesian techniques are adopted to estimate a probability density function over the model parameters given noisy empirical data. The methods used to perform this type of probabilistic calibration are computationally prohibitive in that they require a large number of evaluations of the expensive model. In these cases, surrogate modeling -- that is, using a fast-to-evaluate, lower fidelity stand-in for the original computational model -- may be the only option to alleviate this computational burden. However, the upfront cost of generating training data to build a surrogate model can itself be expensive. As such, it is important to be judicious when selecting training points at which the full-fidelity model is evaluated. Here, an active learning approach is proposed that enables efficient selection of training points using approximate samples of the calibrated parameter probability density function. In this way, the training points can be concentrated in regions where the calibration algorithm requires high model accuracy.

active learning↗

Development of an open-source regional data assimilation system in PEcAn v. 1.7.2: application to carbon cycle reanalysis across the contiguous US using SIPNET

Abstract. The ability to monitor, understand, and predict the dynamics of the terrestrial carbon cycle requires the capacity to robustly and coherently synthesize multiple streams of information that each provide partial information about different pools and fluxes. In this study, we introduce a new terrestrial carbon cycle data assimilation system, built on the PEcAn model–data eco-informatics system, and its application for the development of a proof-of-concept carbon “reanalysis” product that harmonizes carbon pools (leaf, wood, soil) and fluxes (GPP, Ra, Rh, NEE) across the contiguous United States from 1986–2019. We first calibrated this system against plant trait and flux tower net ecosystem exchange (NEE) using a novel emulated hierarchical Bayesian approach. Next, we extended the Tobit–Wishart ensemble filter (TWEnF) state data assimilation (SDA) framework, a generalization of the common ensemble Kalman filter which accounts for censored data and provides a fully Bayesian estimate of model process error, to a regional-scale system with a calibrated localization. Combined with additional workflows for propagating parameter, initial condition, and driver uncertainty, this represents the most complete and robust uncertainty accounting available for terrestrial carbon models. Our initial reanalysis was run on an irregular grid of ∼ 500 points selected using a stratified sampling method to efficiently capture environmental heterogeneity. Remotely sensed observations of aboveground biomass (Landsat LandTrendr) and leaf area index (LAI) (MODIS MOD15) were sequentially assimilated into the SIPNET model. Reanalysis soil carbon, which was indirectly constrained based on modeled covariances, showed general agreement with SoilGrids, an independent soil carbon data product. Reanalysis NEE, which was constrained based on posterior ensemble weights, also showed good agreement with eddy flux tower NEE and reduced root mean square error (RMSE) compared to the calibrated forecast. Ultimately, PEcAn's new open-source regional data assimilation framework provides a scalable workflow for harmonizing multiple data constraints and providing a uniform synthetic platform for carbon monitoring, reporting, and verification (MRV) as well as accelerating terrestrial carbon cycle research.

54 ENVIRONMENTAL SCIENCES↗