SEARCH · Engineering Papers

Results for “Bayesian Optimization”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 577 records · Page 32

All sources NASA NTRS DOE OSTI

DOE OSTI2018

Overhead longwave infrared hyperspectral material identification using radiometric models

Material detection algorithms used in hyperspectral data processing are computationally efficient but can produce relatively high numbers of false positives. Material identification performed as a secondary processing step on detected pixels can help mitigate false positives. A material identification processing chain for longwave infrared hyperspectral data of solid materials collected from airborne platforms is presented. The algorithms utilize unwhitened radiance data and Nelder–Meade numerical optimization to estimate the temperature, humidity, and ozone levels of the atmospheric profile. Pixel unmixing is done using constrained linear regression and Bayesian information criteria for model selection. The resulting identification product includes an optimal atmospheric profile and a full radiance material model that includes material temperature, abundance values, and several fit statistics. A logistic regression method utilizing the model parameters to improve identification is also presented. Furthermore, several examples are provided using modeled data at several noise levels.

58 GEOSCIENCES↗

DOE OSTI2019

Estimating Regional Methane Emissions Through Atmospheric Measurements and Inverse Modeling

In this report we describe an enhanced methodology for performing stochastic Bayesian inversions of atmospheric trace gas inversions that allows the time variation of model parameters to be inferred. We use measurements of methane atmospheric mixing ratio made in Livermore, California along with atmospheric transport modeling and published prior estimates of emissions to estimate the regional emissions of methane and the temporal variations in inferred bias parameters. We compute Bayesian model evidence and continuous rank probability score to optimize the model with respect to temporal resolution. Using two different emissions inventories, we perform inversions for a series of models with increasing temporal resolution in the model bias representation. We show that temporal variation in the model bias can improve the model fit and can also increase the likelihood that the parameterization is appropriate, as measured by the Bayesian model evidence.

54 ENVIRONMENTAL SCIENCES↗

DOE OSTI2015

Tau-REx I: A next generation retrieval code for exoplanetary atmospheres

Spectroscopy of exoplanetary atmospheres has become a well established method for the characterization of extrasolar planets. We here present a novel inverse retrieval code for exoplanetary atmospheres. T-REx(Tau Retrieval for Exoplanets) is a line-by-line radiative transfer fully Bayesian retrieval framework. T-REx includes the following features:(1) the optimized use of molecular line lists from the ExoMol project; (2) an unbiased atmospheric composition prior selection, through custom built pattern recognition software; (3) the use of two independent algorithms to fully sample the Bayesian likelihood space: nested sampling as well as a more classical Markov Chain Monte Carlo approach; (4) iterative Bayesian parameter and model selection using the full Bayesian Evidence as well as the Savage–Dickey Ratio for nested models; and (5) the ability to fully map very large parameter spaces through optimal code parallelization and scalability to cluster computing. In this publication we outline the T-REx framework and demonstrate, using a theoretical hot-Jupiter transmission spectrum, the parameter retrieval and model selection. We investigate the impact of signal-to-noise ratio and spectral resolution on the retrievability of individual model parameters, both in terms of error bars on the temperature and molecular mixing ratios as well as its effect on the model’s global Bayesian evidence.

79 ASTRONOMY AND ASTROPHYSICS↗

DOE OSTI2016

Sparse-grid, reduced-basis Bayesian inversion: Nonaffine-parametric nonlinear equations

We extend the reduced basis (RB) accelerated Bayesian inversion methods for affine-parametric, linear operator equations which are considered in [16,17] to non-affine, nonlinear parametric operator equations. We generalize the analysis of sparsity of parametric forward solution maps in [20] and of Bayesian inversion in [48,49] to the fully discrete setting, including Petrov–Galerkin high-fidelity (“HiFi”) discretization of the forward maps. We develop adaptive, stochastic collocation based reduction methods for the efficient computation of reduced bases on the parametric solution manifold. The nonaffinity and nonlinearity with respect to (w.r.t.) the distributed, uncertain parameters and the unknown solution is collocated; specifically, by the so-called Empirical Interpolation Method (EIM). For the corresponding Bayesian inversion problems, computational efficiency is enhanced in two ways: first, expectations w.r.t. the posterior are computed by adaptive quadratures with dimension-independent convergence rates proposed in [49]; the present work generalizes [49] to account for the impact of the PG discretization in the forward maps on the convergence rates of the Quantities of Interest (QoI for short). Second, we propose to perform the Bayesian estimation only w.r.t. a parsimonious, RB approximation of the posterior density. Based on the approximation results in [49], the infinite-dimensional parametric, deterministic forward map and operator admit N-term RB and EIM approximations which converge at rates which depend only on the sparsity of the parametric forward map. In several numerical experiments, the proposed algorithms exhibit dimension-independent convergence rates which equal, at least, the currently known rate estimates for N-term approximation. We propose to accelerate Bayesian estimation by first offline construction of reduced basis surrogates of the Bayesian posterior density. The parsimonious surrogates can then be employed for online data assimilation and for Bayesian estimation. They also open a perspective for optimal experimental design.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

DOE OSTI2018

A Copula-Based Conditional Probabilistic Forecast Model for Wind Power Ramps

Efficient management of wind ramping characteristics can significantly reduce wind integration costs for balancing authorities. By considering the stochastic dependence of wind power ramp (WPR) features, this paper develops a conditional probabilistic wind power ramp forecast (cp-WPRF) model based on Copula theory. The WPRs dataset is constructed by extracting ramps from a large dataset of historical wind power. Each WPR feature (e.g., rate, magnitude, duration, and start-time) is separately forecasted by considering the coupling effects among different ramp features. To accurately model the marginal distributions with a copula, a Gaussian mixture model (GMM) is adopted to characterize the WPR uncertainty and features. The Canonical Maximum Likelihood (CML) method is used to estimate parameters of the multivariable copula. The optimal copula model is chosen based on the Bayesian information criterion (BIC) from each copula family. Finally, the best conditions based cp-WPRF model is determined by predictive interval (PI) based evaluation metrics. Numerical simulations on publicly available wind power data show that the developed copula-based cp-WPRF model can predict WPRs with a high level of reliability and sharpness.

17 WIND ENERGY↗

DOE OSTI2021

Sensitivity Analysis of Particle-In-Cell Modeling Parameters in Settling Bed, Bubbling Fluidized Bed and Circulating Fluidized Bed

The objective of the work presented is to perform a preliminary sensitivity analysis of particle-in-cell (PIC) model parameters when applied to settling bed, bubbling fluidized bed, and circulating fluidized bed simulations. These examples correspond to widely different flow conditions commonly seen in chemical engineering applications. Simulations were performed using the PIC method in the open-source software Multiphase Flow with Interphase eXchanges (MFiX) developed by the National Energy Technology Laboratory (NETL). As part of the non-intrusive uncertainty quantification (UQ) analysis, simulation campaigns were generated using Nodeworks. Sampling locations or settings for PIC model parameters were determined using the Latin Hypercube method. Response surfaces were created using radial basis functions (RBF), and Sobol’ indices were estimated to quantify the influence of model parameters on the quantities of interest (QoI). This study marks a first step towards systematically determining optimal ranges for model parameters used in MFiX-PIC. Based on limited experience, it is expected that these values would depend strongly on flow conditions. Given the complexity of the multiphase flow systems under analysis, a non-intrusive UQ based approach is used to identify the most influential parameters in each case. This prior knowledge will help in proposing an effective design of experiments (DoE) and determine optimal parameters through techniques such as deterministic or Bayesian calibration, which will be pursued in the future.

42 ENGINEERING↗

DOE OSTI2023

Operation Optimization using Reinforcement Learning with Integrated Artificial Reasoning Framework

In large and complex systems, operational decision-making requires a systematic analysis with a vast amount of data from both process parameters and component status monitoring. In this paper, we present an integrated artificial reasoning approach for system state transition models that can help operational decision-making with explainable and traceable reasoning. The integrated artificial reasoning framework is a physics-based approach of defining the system structure in a Bayesian network, so we leveraged it in a Markov decision process (MDP) for finding optimal operational solutions. In our proposed framework, the MDP is implemented on a dynamic Bayesian network (DBN), which represents causalities in a system. The multilevel flow modeling was utilized in order to extract these causalities in a more efficient and objective manner. Since multilevel flow modeling is based on the fundamental energy and mass conservation laws, the target system is decomposed into several mass, energy, and information structures, which serve as the basis for a DBN. The MDP consists of the processes of finding a solution for the Bellman equation, which can be derived from the conditional probability equations of the constructed DBN. System operators can capture stochastic system dynamics as multiple subsystem state transitions based on their physical relations and uncertainties coming from the component degradation process or random failures. We analyzed a simplified example system to illustrate finding an optimal operational policy with this approach.

99 GENERAL AND MISCELLANEOUS↗

DOE OSTI2017

A scalable design of experiments framework for optimal sensor placement

Here, we present a scalable design of an experiments framework for sensor placement in systems described by partial differential equations (PDEs). In particular, we aim to compute optimal sensor locations by minimizing the uncertainty of parameters estimated from Bayesian inverse problems and where the system. The resulting problem is a computationally intractable mixed-integer nonlinear program constrained by PDEs. We approach this problem with two heuristics used in compressed sensing and optimal control literature: a sparsity-inducing approach and a sum-up rounding approach. We also investigate metrics to guide the design of experiments (the total flow variance and the A-optimal design criterion) and analyze the effect of different noise structures (white and colored). Using an application in natural gas pipelines, we conclude that the sum-up rounding approach approach gives the best results and produces shrinking gaps with increasing mesh resolution. We also observe that convergence for the white noise measurement error case is slower than for the colored noise case. For A-optimal design, the solution is close to a uniform distribution of sensors along the pipeline while for the flow variance design the distribution is unstructured.

42 ENGINEERING↗

DOE OSTI2018

Bayesian calibration of strength parameters using hydrocode simulations of symmetric impact shock experiments of Al-5083

We report that predictive modeling of materials requires accurately parameterized constitutive models. Parameterizing models that describe dynamic strength and plasticity require experimentally probing materials in a variety of strain rate regimes. Some experimental protocols (e.g., plate impact) probe the constitutive response of a material using indirect measures such as free surface velocimetry. Manual efforts to parameterize constitutive models using indirect experimental measures often lead to non-unique optimizations without quantification of parameter uncertainty. This study uses a Bayesian statistical approach to find model parameters and to quantify the uncertainty of the resulting parameters. The technique is demonstrated by parameterizing the Johnson-Cook strength model for aluminum alloy 5083 by coupling hydrocode simulations and velocimetry measurements of a series of plate impact experiments. Simulation inputs and outputs are used to calibrate an emulator that mimics the outputs of the computationally intensive simulations. Varying the amount of experimental data available for emulator calibration showed clear differences in the degree of uncertainty and uniqueness of the resulting optimized Johnson-Cook parameters for Al-5083. The results of the optimization provided a numerical evaluation of the degree of confidence in model parameters and model performance. Lastly, given an understanding of the physical effects of certain model parameters, individual parameter uncertainty can be leveraged to quickly identify gaps in the physical domains covered by completed experiments.

36 MATERIALS SCIENCE↗

DOE OSTI2023

Extending the predictive power of perturbative QCD using the principle of maximum conformality and the Bayesian analysis

In addition to the evaluation of high-order loop contributions, the precision and predictive power of perturbative QCD (pQCD) predictions depends on two important issues: (1) how to achieve a reliable, convergent fixed-order series, and (2) how to reliably estimate the contributions of unknown higher-order terms. The recursive use of renormalization group equation, together with the Principle of Maximum Conformality (PMC), eliminates the renormalization scheme-and-scale ambiguities of the conventional pQCD series. The result is a conformal, scale-invariant series of finite order which also satisfies all of the principles of the renormalization group. In this paper we propose a novel Bayesian-based approach to estimate the size of the unknown higher order contributions based on an optimized analysis of probability distributions. We show that by using the PMC conformal series, in combination with the Bayesian analysis, one can consistently achieve high degree of reliability estimates for the unknown high order terms. Thus the predictive power of pQCD can be greatly improved. We illustrate this procedure for two pQCD observables: R e+e– and R τ , which are each known up to four loops in pQCD. Numerical analyses confirm that by using the scale-independent and more convergent PMC conformal series, one can achieve reliable Bayesian probability estimates for the unknown higher-order contributions.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗

DOE OSTI2025

Designing an Optimal Sensor Network via Minimizing Information Loss

Optimal experimental design is a classic topic in statistics, with many well-studied problems, applications, and solutions. The design problem we study is the placement of sensors to monitor spatiotemporal processes, explicitly accounting for the temporal dimension in our modeling and optimization. We observe that recent advancements in computational sciences often yield large datasets based on physics-based simulations, which are rarely leveraged in experimental design. We introduce a novel model-based sensor placement criterion, along with a highly-efficient optimization algorithm, which integrates physics-based simulations and Bayesian experimental design principles to identify sensor networks that “minimize information loss” from simulated data. Our technique relies on sparse variational inference and (separable) Gauss-Markov priors, and thus may adapt many techniques from Bayesian experimental design. We validate our method through a case study monitoring air temperature in Phoenix, Arizona, using state-of-the-art physics-based simulations. Our results show our framework to be superior to random or quasi-random sampling, particularly with a limited number of sensors. We conclude by discussing practical considerations and implications of our framework, including more complex modeling tools and real-world deployments.

54 ENVIRONMENTAL SCIENCES↗

DOE OSTI2024

Identifiability and characterization of transmon qutrits through Bayesian experimental design

Robust control of a quantum system is essential to utilize the current noisy quantum hardware to its full potential, such as quantum algorithms. To achieve such a goal, a systematic search for an optimal control for any given experiment is essential. The design of optimal control pulses requires accurate numerical models and, therefore, accurate characterization of the system parameters. We present an online Bayesian approach for quantum characterization of qutrit systems, which automatically and systematically identifies optimal experiments that provide maximum information on the system parameters, thereby greatly reducing the number of experiments that need to be performed on the quantum testbed. Unlike most characterization protocols that provide point-estimates of the parameters, the proposed approach is able to estimate their probability distribution. The applicability of the Bayesian experimental design technique was demonstrated on test problems, where each experiment was defined by a parameterized control pulse. In addition to this, we also present an approach for iterative pulse extension, which is robust under uncertainties in transition frequencies and coherence times, and shot noise, despite being initialized with wide uninformative priors. Furthermore, we provide a mathematical proof of the theoretical identifiability of the model parameters and present conditions on the quantum state under which the parameters are identifiable. The proof and conditions for identifiability are presented for both closed and open quantum systems using the Schrödinger equation and the Lindblad master equation, respectively.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2021

Towards robust autonomous impedance spectroscopy analysis: A calibrated hierarchical Bayesian approach for electrochemical impedance spectroscopy (EIS) inversion

Distribution-based analyses, such as the distribution of relaxation times (DRT) and the distribution of diffusion times (DDT), present model-free alternatives to equivalent circuit modeling for analysis of electrochemical impedance spectroscopy (EIS) data. However, reconstructing such distributions from noisy impedance data is an ill-posed problem that must be solved with specialized inversion algorithms, requiring careful control and tuning. Furthermore, most inversion algorithms developed to date can only solve problems of limited complexity. Herein, we present a new hierarchical Bayesian method for EIS inversion, leveraging efficient algorithms for optimization and Hamiltonian Monte Carlo (HMC) sampling to solve models of arbitrary complexity. We overcome the challenge of ad-hoc parameter tuning by encoding intrinsic characteristics of the DRT and DDT into flexible prior distributions and “pre-calibrating” the model to simulated data. This approach is versatile, highly robust to noise, and provides quantitative estimates of both the error structure of the data and the uncertainty in the recovered distributions. The model is validated with simulated data to demonstrate accurate recovery of the DRT and the DDT. The method also shows promise for simultaneous recovery of multiple distributions, raising the intriguing possibility of semi-autonomous EIS analysis and ad-hoc model construction. Finally, the practical utility of the method is illustrated with experimental data. Throughout, we draw comparisons to several recently published EIS inversion methodologies.

36 MATERIALS SCIENCE↗

DOE OSTI2022

Atikokan Digital Twin: Machine learning in a biomass energy system

The Atikokan Generating Station, operated by Ontario Power Generation, has a 200 MW, biomass-fired tower boiler that operates on a dispatch schedule with a five-minute cycle. The boiler is generally operated in the range of 40–100 MW using two of five burner levels. In order to optimize boiler performance, we propose the implementation of a unique digital twin. Our digital twin abstraction couples Bayesian inference from science-based models and from observations (machine learning) with decision theory to predict operating-variable set points that optimize the physical asset (the boiler) in the presence of uncertainty (artificial intelligence). We focus this paper on the continuous Bayesian machine learning part of the Atikokan Digital Twin; we discuss decision theory in a companion paper. We identify and learn about 12 operational, model, and measured-output parameters and their uncertainties from high-fidelity, science-based simulations of the Atikokan boiler and from the observed measurements at the power plant. Since the goal of the Atikokan Digital Twin is to implement it online in real time, we require fast function evaluations for the quantities of interest extracted from the simulations in the Bayesian analysis. We use Gaussian process regression/interpolation to create accurate, robust surrogate models. We define the Bayesian priors and likelihood function and solve for the posterior distributions of the 12 parameters. Here we then propagate these distributions (i.e., parameters with uncertainty) into the predicted distributions of 790 quantities of interest to learn about the relative importance of various sources of error including experimental, model, and operating-parameter errors.

09 BIOMASS FUELS↗

DOE OSTI2021

Multimodal Bayesian registration of noisy functions using Hamiltonian Monte Carlo

Functional data registration is a necessary processing step for many applications. The observed data can be inherently noisy, often due to measurement error or natural process uncertainty; which most functional alignment methods cannot handle. A pair of functions can also have multiple optimal alignment solutions, which is not addressed in current literature. In this paper, a flexible Bayesian approach to functional alignment is presented, which appropriately accounts for noise in the data without any pre-smoothing required. Additionally, by running parallel MCMC chains, the method can account for multiple optimal alignments via the multi-modal posterior distribution of the warping functions. To most efficiently sample the warping functions, the approach relies on a modification of the standard Hamiltonian Monte Carlo to be well-defined on the infinite-dimensional Hilbert space. In this work, this flexible Bayesian alignment method is applied to both simulated data and real data sets to show its efficiency in handling noisy functions and successfully accounting for multiple optimal alignments in the posterior; characterizing the uncertainty surrounding the warping functions.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2020

Probabilistic Groundwater Modeling of the RDX Plume at Los Alamos National Laboratory to Support Risk Assessment - 20359

A plume of the contaminant hexahydro-1,3,5-trinitro-1,3,5-triazine (RDX) with concentrations greater than the New Mexico tap water drinking standard (9.66 ppb) is present in the regional aquifer near the southwestern boundary of Los Alamos National Laboratory (LANL). A risk assessment is performed for exposure to regional aquifer groundwater, with long-term predictions of RDX concentrations provided by a calibrated, probabilistic, numerical fate and transport model. The structure of the model is hierarchical, with the RDX Regional Aquifer (RA) groundwater model acting as the primary tool for analysis of downgradient RDX concentrations. The RA model is deeply informed by the conceptual site model (CSM) and is calibrated using site RDX concentration data and hydraulic head measurements, along with other analyses. The model is calibrated using data through December 2019. Where data are scarce other lines of evidence are used to inform inputs, including the multiphase RDX Vadose Zone (VZ) model and Pipe and Disk analytical screening tool. Model inputs are described with informative prior distributions using a robust approach to development that incorporates all available lines of evidence for every parameter used as an input to the RA model. Calibration is performed using a classical nonlinear optimization routine, which is then used to initialize a Bayesian calibration. The Bayesian calibration constrains the uncertainty in the classical calibration, ultimately providing posterior distributions for all model parameters. The challenges of the calibration include high-dimensional parameter space, including spatially heterogeneous hydraulic conductivities, comparatively sparse data, and low RDX concentrations. Posterior parameter distributions developed in the calibration process are then used for stochastic predictive model runs into the future. The result of the forward runs is spatially and temporally explicit estimates of head and concentration with uncertainty at all points in space and time. The probabilistic modeling approach presented here includes innovative computational and statistical methods that leverage high-performance computing (HPC) resources. It makes use of prior modeling work performed at the chromium plume site in the central LANL area, with extensive updates. The risk assessment will ultimately be used to support decision-making at the site, using multiple appropriately weighted sources of information, as well as uncertainty, sensitivity, and value-of-information analyses. (authors)

12 MANAGEMENT OF RADIOACTIVE AND NON-RADIOACTIVE W↗

DOE OSTI2020

Fast strength model characterization using Bayesian statistics

A variety of flow stress models exist with new models constantly being developed. These models aim to approximate the strength of materials in a variety of regimes from quasistatic loading through shock scenarios. All models contain an array of parameters which need to be tuned to the material under study. Some models perform well under limited conditions, requiring adjustment of the parameters when venturing outside of those predefined ranges. Other models perform well over a wide range of conditions with a set of parameters, but may be outperformed by other models optimized on a tighter range of conditions. Recent research by Los Alamos demonstrated the ability to optimize the Johnson Cook (JC) model using a set of 3 plate-impact experiments on Aluminum. They utilized Bayesian statistics and emulation to determine optimal parameters for the model with a quantification of parameter uncertainty. We present an extension of this capability to incorporate velocimetry from plate-impact tests, stress-strain data from split Hopkinson pressure bar and quasistatic compression tests, plus profiles from Taylor cylinders in a unified fashion. Statistically robust comparisons of the performance and uncertainty of different realizations of the JC flow stress model were carried out based on calibration to several possible combinations of these three different experiment types.

75 CONDENSED MATTER PHYSICS, SUPERCONDUCTIVITY AND↗

DOE OSTI2021

Sequential Bayesian inference of transition rates in the hidden Markov model for multi-state system degradation

The more easily available system performance data and advances in data analytics have provided us with opportunities to optimize maintenance programs for engineered systems, for example nuclear power plants. One key task in maintenance optimization is to obtain an accurate model for system degradation. Here, in this research, we propose a Bayesian method to address this problem. Noting that systems usually exhibit multiple states and that the actual state of a system usually is not directly observable, in the method we first model the system degradation process and the observation process based on a hidden Markov model. Then we develop a sequential Bayesian inference algorithm based on importance sampling and the forward algorithm to infer the posterior distributions of the transition rates in the hidden Markov model based on available observations. The proposed Bayesian method allows us to take advantage of evidence from multiple sources, and also allows us to perform Bayesian inference sequentially, without the need to use the entire history of observations every time new observations are collected. We demonstrate the proposed method using both synthetic data for a nuclear power plant feedwater pump and realistic data for a nuclear power plant chemistry analytical device.

42 ENGINEERING↗