Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “bayesian optimization”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 541 records · Page 30

Final Report, DOE Early Career Award: Predictive modeling of complex physical systems: new tools for statistical inference, uncertainty quantification, and experimental design

Predictive simulation of complex physical systems increasingly rests on the interplay of experimental observations with computational models. Key inputs, parameters, or structural aspects of models may be incomplete or unknown, and must be developed from indirect and limited observations. At the same time, quantified uncertainties are needed to qualify computational predictions in the support of design and decision-making. In this context, Bayesian statistics provides a foundation for inference from noisy and limited data, but at prohibitive computional expense. This project intends to make rigorous predictive modeling *feasible* in complex physical systems, via accelerated and scalable tools for uncertainty quantification, Bayesian inference, and experimental design. Specific objectives are as follows: 1. Develop adaptive posterior approximations and dimensionality reduction approaches for Bayesian inference in high-dimensional nonlinear systems. 2. Extend accelerated Bayesian methodologies to large-scale {\em sequential} data assimilation, fully treating nonlinear models and non-Gaussian state and parameter distributions. 3. Devise efficient surrogate-based methods for Bayesian model selection and the learning of model structure. 4. Develop scalable simulation/optimization approaches to nonlinear Bayesian experimental design, for both parameter inference and model selection. 5. Demonstrate these inferential tools on chemical kinetic models in reacting flow, constructing and refining thermochemical and electrochemical models from limited data. Demonstrate Bayesian filtering on canonical stochastic PDEs and in the dynamic estimation of inhomogeneous subsurface properties and flow fields.

97 MATHEMATICS AND COMPUTING↗

Efficient high-dimensional variational data assimilation with machine-learned reduced-order models

Abstract. Data assimilation (DA) in geophysical sciences remains the cornerstone of robust forecasts from numerical models. Indeed, DA plays a crucial role in the quality of numerical weather prediction and is a crucial building block that has allowed dramatic improvements in weather forecasting over the past few decades. DA is commonly framed in a variational setting, where one solves an optimization problem within a Bayesian formulation using raw model forecasts as a prior and observations as likelihood. This leads to a DA objective function that needs to be minimized, where the decision variables are the initial conditions specified to the model. In traditional DA, the forward model is numerically and computationally expensive. Here we replace the forward model with a low-dimensional, data-driven, and differentiable emulator. Consequently, gradients of our DA objective function with respect to the decision variables are obtained rapidly via automatic differentiation. We demonstrate our approach by performing an emulator-assisted DA forecast of geopotential height. Our results indicate that emulator-assisted DA is faster than traditional equation-based DA forecasts by 4 orders of magnitude, allowing computations to be performed on a workstation rather than a dedicated high-performance computer. In addition, we describe accuracy benefits of emulator-assisted DA when compared to simply using the emulator for forecasting (i.e., without DA). Our overall formulation is denoted AIEADA (Artificial Intelligence Emulator-Assisted Data Assimilation).

58 GEOSCIENCES↗

Trust from the past: Bayesian Personalized Ranking based Link Prediction in Knowledge Graphs

Estimating the confidence for a link is a critical task for Knowledge Graph construction. Link prediction, or predicting the likelihood of a link in a knowledge graph based on prior state is a key research direction within this area. We propose a Latent Feature Embedding based link recommendation model for prediction task and utilize Bayesian Personalized Ranking based optimization technique for learning models for each predicate. Experimental results on large-scale knowledge bases such as YAGO2 show that our approach achieves substantially higher performance than several state-of-art approaches. Furthermore, we also study the performance of the link prediction algorithm in terms of topological properties of the Knowledge Graph and present a linear regression model to reason about its expected level of accuracy.

knowledge graphs↗

Optimization of artificial viscosity in production codes based on Gaussian Regression surrogate models

To accurately model flows with shock waves using staggered-grid Lagrangian hydrodynamics, artificial viscosity has to be introduced to convert kinetic energy into internal energy, thereby increasing the entropy across shocks. Determining the appropriate strength of the artificial viscosity is an art and strongly depends on the particular problem and experience of the researcher. The objective of this study is to pose the problem of finding the appropriate strength of artificial viscosity as an optimization problem and solve this problem using machine learning (ML) tools, specifically using surrogate models based on Gaussian Process regression and Bayesian analysis. We describe the optimization method and discuss various practical details of its implementation. The shock-containing problems for which we apply this method all have been implemented in the LANL code FLAG. First, we apply ML to find optimal values to isolated shock problems of different strengths. Second, we apply ML to optimize viscosity for a 1D propagating detonation problem based on Zel’dovich-von Neumann-Doring (ZND) detonation theory using a reactive burn model. We compare results for default (currently used values in FLAG) and optimized values of artificial viscosity for these problems demonstrating the potential for significant improvement in the accuracy of computations.

42 ENGINEERING↗

Particle transport constraints via Bayesian spectral fitting of multiple atomic lines

Optimized operation of fusion devices demands detailed understanding of plasma transport, a problem that must be addressed with advances in both measurement and data analysis techniques. In this work, we adopt Bayesian inference methods to determine experimental particle transport, leveraging opportunities from high-resolution He-like ion spectra in a tokamak plasma. The Bayesian spectral fitting code is used to analyze resonance (w), forbidden (z), intercombination (x, y), and satellite (k, j) lines of He-like Ca following laser blow-off injections on Alcator C-Mod. This offers powerful transport constraints since these lines depend differently on electron temperature and density, but also differ in their relation to Li-like, He-like, and H-like ion densities, often the dominant Ca charge states over most of the C-Mod plasma radius. Using synthetic diagnostics based on the AURORA package, we demonstrate improved effectiveness of impurity transport inferences when spectroscopic data from a progressively larger number of lines are included.

47 OTHER INSTRUMENTATION↗

Particle transport constraints via Bayesian spectral fitting of multiple atomic lines

Optimized operation of fusion devices demands detailed understanding of plasma transport, a problem that must be addressed with advances in both measurement and data analysis techniques. In this work, we adopt Bayesian inference methods to determine experimental particle transport, leveraging opportunities from high-resolution He-like ion spectra in a tokamak plasma. The Bayesian spectral fitting code is used to analyze resonance (w), forbidden (z), intercombination (x, y), and satellite (k, j) lines of He-like Ca following laser blow-off injections on Alcator C-Mod. This offers powerful transport constraints since these lines depend differently on electron temperature and density, but also differ in their relation to Li-like, He-like, and H-like ion densities, often the dominant Ca charge states over most of the C-Mod plasma radius. Using synthetic diagnostics based on the AURORA package, we demonstrate improved effectiveness of impurity transport inferences when spectroscopic data from a progressively larger number of lines are included.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗

Two-photon exchange effect studied with neural networks

An approach to the extraction of the two-photon exchange (TPE) correction from elastic ep scattering data is presented. The cross-section, polarization transfer (PT), and charge asymmetry data are considered. It is assumed that the TPE correction to the PT data is negligible. The form factors and TPE correcting term are given by one multidimensional function approximated by the feedforward neural network (NN). To find a model-independent approximation, the Bayesian framework for the NNs is adapted. A large number of different parametrizations is considered. The most optimal model is indicated by the Bayesian algorithm. The obtained fit of the TPE correction behaves linearly in {epsilon} but it has a nontrivial Q{sup 2} dependence. A strong dependence of the TPE fit on the choice of parametrization is observed.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗

Joint Data Filtering and Labeling Using Gaussian Processes and Alternating Direction Method of Multipliers

Here, sequence labeling aims at assigning a label to every sample of a signal (or pixel of an image) while considering the sequentiality (or vicinity) of the samples. To perform this task, many works in the literature first filter and then label the data. Unfortunately, the filtering, which is performed independently from the labeling, is far from optimal and frequently makes the latter task harder. In this paper, a novel approach that trains a Gaussian process classifier and estimates the coefficients of an optimal filter jointly is presented. The new approach, based on Bayesian modeling and alternating direction method of multipliers (ADMMs) optimization, performs both tasks simultaneously. All unknowns are treated as stochastic variables, which are estimated using variational inference and filtering and labeling are linked with the use of ADMM. In the experimental section, synthetic and real experiments are presented to compare the proposed method with other existing approaches.

97 MATHEMATICS AND COMPUTING↗

A Bayesian experimental design approach to structural health monitoring

Optimal system design for SHM involves two primarily challenges. The first is the derivation of a proper performance function for a given system design. The second is the development of an efficient optimization algorithm for choosing a design that maximizes, or nearly maximizes the performance function. In this paper we will outline how an SHM practitioner can construct the proper performance function by casting the entire design problem into a framework of Bayesian experimental design. The approach demonstrates how the design problem necessarily ties together all steps of the SHM process.

42 ENGINEERING↗

MADLens, a python package for fast and differentiable non-Gaussian lensing simulations

Here, we present MADLens a python package for producing non-Gaussian lensing convergence maps at arbitrary source redshifts with unprecedented precision. MADLens is designed to achieve high accuracy while keeping computational costs as low as possible. A MADLens simulation with only particles produces convergence maps whose power agrees with theoretical lensing power spectra up to within the accuracy limits of HaloFit. This is made possible by a combination of a highly parallelizable particle-mesh algorithm, a sub-evolution scheme in the lensing projection, and a machine-learning inspired sharpening step. Further, MADLens is fully differentiable with respect to the initial conditions of the underlying particle-mesh simulations and a number of cosmological parameters. These properties allow MADLens to be used as a forward model in Bayesian inference algorithms that require optimization or derivative-aided sampling. Another use case for MADLens is the production of large, high resolution simulation sets as they are required for training novel deep-learning-based lensing analysis tools. We make the MADLens package publicly available under a Creative Commons License

79 ASTRONOMY AND ASTROPHYSICS↗

Robust Estimation of Electron Density From Anatomic Magnetic Resonance Imaging of the Brain Using a Unifying Multi-Atlas Approach

Purpose: To develop a reliable method to estimate electron density based on anatomic magnetic resonance imaging (MRI) of the brain. Methods and Materials: We proposed a unifying multi-atlas approach for electron density estimation based on standard T1- and T2-weighted MRI. First, a composite atlas was constructed through a voxelwise matching process using multiple atlases, with the goal of mitigating effects of inherent anatomic variations between patients. Next we computed for each voxel 2 kinds of conditional probabilities: (1) electron density given its image intensity on T1- and T2-weighted MR images; and (2) electron density given its spatial location in a reference anatomy, obtained by deformable image registration. These were combined into a unifying posterior probability density function using the Bayesian formalism, which provided the optimal estimates for electron density. We evaluated the method on 10 patients using leave-one-patient-out cross-validation. Receiver operating characteristic analyses for detecting different tissue types were performed. Results: The proposed method significantly reduced the errors in electron density estimation, with a mean absolute Hounsfield unit error of 119, compared with 140 and 144 (P<.0001) using conventional T1-weighted intensity and geometry-based approaches, respectively. For detection of bony anatomy, the proposed method achieved an 89% area under the curve, 86% sensitivity, 88% specificity, and 90% accuracy, which improved upon intensity and geometry-based approaches (area under the curve: 79% and 80%, respectively). Conclusion: The proposed multi-atlas approach provides robust electron density estimation and bone detection based on anatomic MRI. If validated on a larger population, our work could enable the use of MRI as a primary modality for radiation treatment planning.

62 RADIOLOGY AND NUCLEAR MEDICINE↗

A Multi-Route Model of Nicotine-Cotinine Pharmacokinetics, Pharmacodynamics and Brain Nicotinic Acetylcholine Receptor Binding in Humans

The pharmacokinetics of nicotine, the pharmacologically active alkaloid in tobacco responsible for addiction, are well characterized in humans. We developed a physiologically based pharmacokinetic/pharmacodynamic model of nicotine pharmacokinetics, brain dosimetry and brain nicotinic acetylcholine receptor (nAChRs) occupancy. A Bayesian framework was applied to optimize model parameters against multiple human data sets. The resulting model was consistent with both calibration and test data sets, but in general underestimated variability. A pharmacodynamic model relating nicotine levels to increases in heart rate as a proxy for the pharmacological effects of nicotine accurately described the nicotine related changes in heart rate and the development and decay of tolerance to nicotine. The PBPK model was utilized to quantitatively capture the combined impact of variation in physiological and metabolic parameters, nicotine availability and smoking compensation on the change in number of cigarettes smoked and toxicant exposure in a population of 10,000 people presented with a reduced toxicant (50%), reduced nicotine (50%) cigarette Across the population, toxicant exposure is reduced in some but not all smokers. Reductions are not in proportion to reductions in toxicant yields, largely due to partial compensation in response to reduced nicotine yields. This framework can be used as a key element of a dosimetry-driven risk assessment strategy for cigarette smoke constituents.

Teeguarden, Justin G↗

Bayesian Model Selection as a Continuous-Variable Helmholtz Machine

We show that Bayesian model selection is equivalent to optimization of a suitably defined free energy. The entropy term in the free energy is defined over the space of possible model selections and model parameters for a given model. Bayesian model selection thus follows a minimum description length principle.

97 MATHEMATICS AND COMPUTING↗

Inferring Type II-P Supernova Progenitor Masses from Plateau Luminosities

Abstract Connecting observations of core-collapse supernova explosions to the properties of their massive star progenitors is a long-sought, and challenging, goal of supernova science. Recently, Barker et al. presented bolometric light curves for a landscape of progenitors from spherically symmetric neutrino-driven core-collapse supernova (CCSN) simulations using an effective model. They find a tight relationship between the plateau luminosity of the Type II-P CCSN light curve and the terminal iron-core mass of the progenitor. Remarkably, this allows us to constrain progenitor properties with photometry alone. We analyze a large observational sample of Type II-P CCSN light curves and estimate a distribution of iron-core masses using the relationship of Barker et al. The inferred distribution matches extremely well with the distribution of iron-core masses from stellar evolutionary models and namely, contains high-mass iron cores that suggest contributions from very massive progenitors in the observational data. We use this distribution of iron-core masses to infer minimum and maximum masses of progenitors in the observational data. Using Bayesian inference methods to locate optimal initial mass function parameters, we find M min = 9.8 − 0.27 + 0.37 and M max = 24.0 − 1.9 + 3.9 solar masses for the observational data.

79 ASTRONOMY AND ASTROPHYSICS↗

Data-free inference of the joint distribution of uncertain model parameters.

It is known that, in general, the correlation structure in the joint distribution of model parameters is critical to the uncertainty analysis of that model. Very often, however, studies in the literature only report nominal values for parameters inferred from data, along with confidence intervals for these parameters, but no details on the correlation or full joint distribution of these parameters. When neither posterior nor data are available, but only summary statistics such as nominal values and confidence intervals, a joint PDF must be chosen. Given the summary statistics it may not be reasonable nor necessary to assume the parameters are independent random variables. We demonstrate, using a Bayesian inference procedure, how to construct a posterior density for the parameters exhibiting self consistent correlations, in the absence of data, given (1) the fit-model, (2) nominal parameter values, (3) bounds on the parameters, and (4) a postulated statistical model, around the fit-model, for the missing data. Our approach ensures external Bayesian updating while marginalizing over possible data realizations. We then address the matching of given parameter bounds through the choice of hyperparameters, which are introduced in postulating the statistical model, but are not given nominal values. We discuss some possible approaches, including (1) inferring them in a separate Bayesian inference loop and (2) optimization. We also perform an empirical evaluation of the algorithm showing the posterior obtained with this data free inference compares well with the true posterior obtained from inference against the full data set.

99 GENERAL AND MISCELLANEOUS↗

Data-free inference of uncertain model parameters.

It is known that, in general, the correlation structure in the joint distribution of model parameters is critical to the uncertainty analysis of that model. Very often, however, studies in the literature only report nominal values for parameters inferred from data, along with confidence intervals for these parameters, but no details on the correlation or full joint distribution of these parameters. When neither posterior nor data are available, but only summary statistics such as nominal values and confidence intervals, a joint PDF must be chosen. Given the summary statistics it may not be reasonable nor necessary to assume the parameters are independent random variables. We demonstrate, using a Bayesian inference procedure, how to construct a posterior density for the parameters exhibiting self consistent correlations, in the absence of data, given (1) the fit-model, (2) nominal parameter values, (3) bounds on the parameters, and (4) a postulated statistical model, around the fit-model, for the missing data. Our approach ensures external Bayesian updating while marginalizing over possible data realizations. We then address the matching of given parameter bounds through the choice of hyperparameters, which are introduced in postulating the statistical model, but are not given nominal values. We discuss some possible approaches, including (1) inferring them in a separate Bayesian inference loop and (2) optimization. We also perform an empirical evaluation of the algorithm showing the posterior obtained with this data free inference compares well with the true posterior obtained from inference against the full data set.

99 GENERAL AND MISCELLANEOUS↗

Uncertainty quantification for nuclear density functional theory and information content of new measurements

Statistical tools of uncertainty quantification can be used to assess the information content of measured observables with respect to present-day theoretical models, to estimate model errors and thereby improve predictive capability, to extrapolate beyond the regions reached by experiment, and to provide meaningful input to applications and planned measurements. To showcase new opportunities offered by such tools, we make a rigorous analysis of theoretical statistical uncertainties in nuclear density functional theory using Bayesian inference methods. By considering the recent mass measurements from the Canadian Penning Trap at Argonne National Laboratory, we demonstrate how the Bayesian analysis and a direct least-squares optimization, combined with high-performance computing, can be used to assess the information content of the new data with respect to a model based on the Skyrme energy density functional approach. Employing the posterior probability distribution computed with a Gaussian process emulator, we apply the Bayesian framework to propagate theoretical statistical uncertainties in predictions of nuclear masses, two-neutron dripline, and fission barriers. Overall, we find that the new mass measurements do not impose a constraint that is strong enough to lead to significant changes in the model parameters. In addition, the example discussed in this study sets the stage for quantifying and maximizing the impact of new measurements with respect to current modeling and guiding future experimental efforts, thus enhancing the experiment-theory cycle in the scientific method.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

An optimal merging technique for high-resolution precipitation products: OPTIMAL MERGING OF PRECIPITATION METHOD

Precipitation products are currently available from various sources at higher spatial and temporal resolution than any time in the past. Each of the precipitation products has its strengths and weaknesses in availability, accuracy, resolution, retrieval techniques and quality control. By merging the precipitation data obtained from multiple sources, one can improve its information content by minimizing these issues. However, precipitation data merging poses challenges of scale-mismatch, and accurate error and bias assessment. In this paper we present Optimal Merging of Precipitation (OMP), a new method to merge precipitation data from multiple sources that are of different spatial and temporal resolutions and accuracies. This method is a combination of scale conversion and merging weight optimization, involving performance-tracing based on Bayesian statistics and trend-analysis, which yields merging weights for each precipitation data source. The weights are optimized at multiple scales to facilitate multiscale merging and better precipitation downscaling. Precipitation data used in the experiment include products from the 12-km resolution North American Land Data Assimilation (NLDAS) system, the 8-km resolution CMORPH and the 4-km resolution National Stage-IV QPE. The test cases demonstrate that the OMP method is capable of identifying a better data source and allocating a higher priority for them in the merging procedure, dynamically over the region and time period. This method is also effective in filtering out poor quality data introduced into the merging process.

54 ENVIRONMENTAL SCIENCES↗