Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Bayesian Gaussian process”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 217 records · Page 12
All sourcesNASA NTRSDOE OSTI
DOE OSTI2022

Random field optimization

Herein we present a new modeling paradigm for optimization that we call random field optimization. Random fields are a powerful modeling abstraction that aims to capture the behavior of random variables that live on infinite-dimensional spaces (e.g., space and time) such as stochastic processes (e.g., time series, Gaussian processes, and Markov processes), random matrices, and random spatial fields. This paradigm involves sophisticated mathematical objects (e.g., stochastic differential equations and space-time kernel functions) and has been widely used in neuroscience, geoscience, physics, civil engineering, and computer graphics. Despite of this, however, random fields have seen limited use in optimization; specifically, existing optimization paradigms that involve uncertainty (e.g., stochastic programming and robust optimization) mostly focus on the use of finite random variables. This trend is rapidly changing with the advent of statistical optimization (e.g., Bayesian optimization) and multi-scale optimization (e.g., integration of molecular sciences and process engineering). Our work extends a recently-proposed abstraction for infinite-dimensional optimization problems by capturing more general uncertainty representations. Moreover, we discuss solution paradigms for this new class of problems based on finite transformations and sampling, and identify open questions and challenges.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2022

Nonlinear gyrokinetic predictions of SPARC burning plasma profiles enabled by surrogate modeling

Multi-channel, nonlinear predictions of core temperature and density profiles are performed for the SPARC tokamak accounting for both kinetic neoclassical and fully nonlinear gyro-kinetic turbulent fluxes. A series of flux-tube, nonlinear, electromagnetic simulations using the CGYRO code with six gyrokinetic species are coupled to a nonlinear optimizer using Gaussian process regression techniques. The simultaneous evolution of energy sources, including alpha heat, radiation, and energy exchange, coupled with these high fidelity models and techniques, leads to a converged solution in electron temperature, ion temperature and electron density channels with a minimal number of expensive gyrokinetic simulations without compromising accuracy.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗
DOE OSTI2022

A Differentiable Hydrology Approach for Modeling With Time-Varying Parameters

In the analysis of observational data with mathematical models, it is often desirable to let one or more model parameters vary with time to account for changing environmental conditions or to offer more flexibility. A standard approach for dynamic models is to allow for the number of parameters to grow as a function of time, leading to a high-dimensional inference problem for long data records. We propose using Hamiltonian Monte Carlo, a gradient-based Markov chain Monte Carlo (MCMC) method, to perform Bayesian inference for time-varying parameters (TVP) in hydrology models. As derivatives of model error functions with regard to parameters are not available in closed form, we implemented the GR4J rainfall-runoff models in Theano/PyMC3, and Jax/NumPyro, allowing for the application of automatic differentiation techniques. A simulation experiment assessing the viability of these methods for recovering underlying temporal variation in parameters indicates that a discrete Gaussian random walk-based prior appears to be best suited across a range of scenarios while the continuous-time Gaussian process compares relatively poorly. In an analysis of 20 years of daily real-world streamflow records from the Model Parameter Estimation Experiment (MOPEX) hydrology data set with increasing data sparsity and at multiple temporal resolutions, we find that TVP inference with gradient-based MCMC is a flexible and reliable approach for analyzing hydrology models with dynamic parameter sets.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2021

srMO-BO-3GP: A sequential regularized multi-objective Bayesian optimization for constrained design applications using an uncertain Pareto classifier

Bayesian optimization (BO) is an efficient and flexible global optimization framework that is applicable to a very wide range of engineering applications. To leverage the capability of the classical BO, many extensions, including multi-objective, multi-fidelity, parallelization, and latent-variable modeling, have been proposed to address the limitations of the classical BO framework. In this work, we propose a novel multi-objective BO formalism, called srMO-BO-3GP, to solve multi-objective optimization problems in a sequential setting. Three different Gaussian processes (GPs) are stacked together, where each of the GPs is assigned with a different task. The first GP is used to approximate a single-objective computed from the multi-objective definition, the second GP is used to learn the unknown constraints, and the third one is used to learn the uncertain Pareto frontier. At each iteration, a multi-objective augmented Tchebycheff function is adopted to convert multi-objective to single-objective, where the regularization with a regularized ridge term is also introduced to smooth the single-objective function. Finally, we couple the third GP along with the classical BO framework to explore the convergence and diversity of the Pareto frontier by the acquisition function for exploitation and exploration. The proposed framework is demonstrated using several numerical benchmark functions, as well as a thermomechanical finite element model for flip-chip package design optimization.

42 ENGINEERING↗
DOE OSTI2024

Forward variable selection enables fast and accurate dynamic system identification with Karhunen-Loève decomposed Gaussian processes

A promising approach for scalable Gaussian processes (GPs) is the Karhunen-Loève (KL) decomposition, in which the GP kernel is represented by a set of basis functions which are the eigenfunctions of the kernel operator. Such decomposed kernels have the potential to be very fast, and do not depend on the selection of a reduced set of inducing points. However KL decompositions lead to high dimensionality, and variable selection thus becomes paramount. This paper reports a new method of forward variable selection, enabled by the ordered nature of the basis functions in the KL expansion of the Bayesian Smoothing Spline ANOVA kernel (BSS-ANOVA), coupled with fast Gibbs sampling in a fully Bayesian approach. It quickly and effectively limits the number of terms, yielding a method with competitive accuracies, training and inference times for tabular datasets of low feature set dimensionality. Theoretical computational complexities are O ( N P 2 ) in training and O ( P ) per point in inference, where N is the number of instances and P the number of expansion terms. The inference speed and accuracy makes the method especially useful for dynamic systems identification, by modeling the dynamics in the tangent space as a static problem, then integrating the learned dynamics using a high-order scheme. The methods are demonstrated on two dynamic datasets: a ‘Susceptible, Infected, Recovered’ (SIR) toy problem, along with the experimental ‘Cascaded Tanks’ benchmark dataset. Comparisons on the static prediction of time derivatives are made with a random forest (RF), a residual neural network (ResNet), and the Orthogonal Additive Kernel (OAK) inducing points scalable GP, while for the timeseries prediction comparisons are made with LSTM and GRU recurrent neural networks (RNNs) along with the SINDy package.

Hayes, Kyle↗
DOE OSTI2020

A Bayesian analysis of nuclear deformation properties with Skyrme energy functionals

In spite of numerous scientific and practical applications, there is still no comprehensive theoretical description of the nuclear fission process based solely on protons, neutrons and their interactions. Furthermore, the most advanced simulations of fission are currently carried out within nuclear density functional theory (DFT). In spite of being fully quantum-mechanical and rooted in the theory of nuclear forces, DFT still depends on a dozen or so parameters characterizing the energy functional. Calibrating these parameters on experimental data results in uncertainties that must be quantified for applications. This task is very challenging because of the high computational cost of DFT calculations for fission. In this paper, we use Gaussian processes to build emulators of DFT models in order to quantify and propagate statistical uncertainties of theoretical predictions for a range of nuclear deformations relevant to describing the fission process.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗
DOE OSTI2021

Photometric redshifts for the S-PLUS Survey: Is machine learning up to the task?

The Southern Photometric Local Universe Survey (S-PLUS) is a novel project that aims to map the Southern Hemisphere using a twelve filter system, comprising five broad-band SDSS-like filters and seven narrow-band filters optimized for important stellar features in the local universe. In this paper we use the photometry and morphological information from the first S-PLUS data release (S-PLUS DR1) cross-matched to unWISE data and spectroscopic redshifts from Sloan Digital Sky Survey DR15. We explore three different machine learning methods (Gaussian Processes with GPz and two Deep Learning models made with TensorFlow) and compare them with the currently used template-fitting method in the S-PLUS DR1 to address whether machine learning methods can take advantage of the twelve filter system for photometric redshift prediction. Using tests for accuracy for both single-point estimates such as the calculation of the scatter, bias, and outlier fraction, and probability distribution functions (PDFs) such as the Probability Integral Transform (PIT), the Continuous Ranked Probability Score (CRPS) and the Odds distribution, we conclude that a deep-learning method using a combination of a Bayesian Neural Network and a Mixture Density Network offers the most accurate photometric redshifts for the current test sample. In conclusion, it achieves single-point photometric redshifts with scatter (σ NMAD ) of 0.023, normalized bias of -0.001, and outlier fraction of 0.64% for galaxies with r_auto magnitudes between 16 and 21.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2023

Coupling multi-fidelity xRAGE with machine learning for graded inner shell design optimization in double shell capsules

Bayesian optimization has shown promise for the design optimization of inertial confinement fusion targets. Specifically, in Vazirani et al. [Phys. Plasmas 28 , 122709 (2021)], optimal designs for double shell capsules with graded inner shells were identified using one-dimensional xRAGE simulation yield calculations. While the machine learning models were able to accurately learn and predict one-dimensional simulation target performance, using simulations with higher fidelity would improve design optimization and better match with the expected experimental performance. However, higher fidelity physics modeling, i.e., two-dimensional xRAGE simulations, requires significantly larger computational time/cost, usually at least an order of magnitude, in comparison with one-dimensional simulations. This study presents a multi-fidelity Bayesian optimization, in which the machine learning model leverages low-fidelity (one-dimensional xRAGE) and high-fidelity (two-dimensional xRAGE) simulations to more accurately predict “pre-shot” target performance with respect to the expected experimental performance. By building a multi-fidelity Bayesian optimization framework coupled with xRAGE, the low-fidelity and high-fidelity simulations are able to inform one another, such that we have: (1) improved physics modeling in comparison with using low-fidelity simulations alone, (2) reduced computational time/cost in comparison with using high-fidelity simulations alone, and (3) more confidence in the expected performance of optimized targets during real-world experiments. In the future, we plan to use this robust multi-fidelity Bayesian optimization methodology to expedite the design of graded inner shells further and eventually full capsules as a part of the current double shell campaign at the National Ignition Facility.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗
DOE OSTI2025

Designing an Optimal Sensor Network via Minimizing Information Loss

Optimal experimental design is a classic topic in statistics, with many well-studied problems, applications, and solutions. The design problem we study is the placement of sensors to monitor spatiotemporal processes, explicitly accounting for the temporal dimension in our modeling and optimization. We observe that recent advancements in computational sciences often yield large datasets based on physics-based simulations, which are rarely leveraged in experimental design. We introduce a novel model-based sensor placement criterion, along with a highly-efficient optimization algorithm, which integrates physics-based simulations and Bayesian experimental design principles to identify sensor networks that “minimize information loss” from simulated data. Our technique relies on sparse variational inference and (separable) Gauss-Markov priors, and thus may adapt many techniques from Bayesian experimental design. We validate our method through a case study monitoring air temperature in Phoenix, Arizona, using state-of-the-art physics-based simulations. Our results show our framework to be superior to random or quasi-random sampling, particularly with a limited number of sensors. We conclude by discussing practical considerations and implications of our framework, including more complex modeling tools and real-world deployments.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2024

A machine learning approach for determining temperature-dependent bandgap of metal oxides utilizing Allen–Heine–Cardona theory and O’Donnell model parameterization

To evaluate the high temperature sensing properties of metal oxide and perovskite materials suitable for use in combustion environments, it is necessary to understand the temperature dependence of their bandgaps. Although such temperature-driven changes can be calculated via the Allen–Heine–Cardona (AHC) theory, which assesses electron–phonon coupling for the bandgap correction at given temperatures, this approach is computationally demanding. Another approach to predict bandgap temperature-dependence is the O’Donnell model, which uses analytical expressions with multiple fitting parameters that require bandgap information at 0 K. This work employs data-driven Gaussian process regression (GPR) to predict the parameters employed in the O’Donnell model from a set of physical features. We use a sample of 54 metal oxides for which density functional theory has been performed to calculate the bandgap at 0 K, and the AHC calculations have been carried out to determine the shift in the bandgap at non-zero temperatures. As the AHC calculations are impractical for high-throughput screening of materials, the developed GPR model attempts to alleviate this issue by predicting the O'Donnell parameters purely from physical features. To mitigate the reliability issues arising from the very small size of the dataset, we apply a Bayesian technique to improve the generalizability of the data-driven models as well as quantify the uncertainty associated with the predictions. The method captures well the overall trend of the O’Donnell parameters with respect to a reduced feature set obtained by transforming the available physical features. Quantifying the associated uncertainty helps us understand the reliability of the predictions of the O’Donnell parameters and, therefore, the bandgap as a function of temperature for any novel material.

36 MATERIALS SCIENCE↗
DOE OSTI2025

Enhancing approximate modular Bayesian inference by emulating the conditional posterior

In modular Bayesian analyses, complex models are composed of distinct modules, each representing different aspects of the data or prior information. In this context, fully Bayesian approaches can sometimes lead to undesirable feedback between modules, compromising the integrity of the inference. The “cut-distribution” prevents unwanted influence between modules by “cutting” feedback. The direct sampling (DS) algorithm is standard practice for approximating the cut-distribution, but it can be computationally intensive, especially when the number of imputations required is large. An enhanced method is proposed, the Emulating the Conditional Posterior (ECP) algorithm, which leverages emulation to increase the number of imputations. Through numerical experiment it is demonstrated that the ECP algorithm outperforms the traditional DS approach in terms of accuracy and computational efficiency, particularly when resources are constrained. Here, it is also shown how the DS algorithm can be improved using ideas from design of experiments. Some practical recommendations are given for algorithm choice in modular Bayesian analyses.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2022

Bayesian operator inference for data-driven reduced-order modeling

This work proposes a Bayesian inference method for the reduced-order modeling of time-dependent systems. Informed by the structure of the governing equations, the task of learning a reduced-order model from data is posed as a Bayesian inverse problem with Gaussian prior and likelihood. The resulting posterior distribution characterizes the operators defining the reduced-order model, hence the predictions subsequently issued by the reduced-order model are endowed with uncertainty. The statistical moments of these predictions are estimated via a Monte Carlo sampling of the posterior distribution. Since the reduced models are fast to solve, this sampling is computationally efficient. Furthermore, the proposed Bayesian framework provides a statistical interpretation of the regularization term that is present in the deterministic operator inference problem, and the empirical Bayes approach of maximum marginal likelihood suggests a selection algorithm for the regularization hyperparameters. The proposed method is demonstrated on two examples: the compressible Euler equations with noise-corrupted observations, and a single-injector combustion process.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2024

Physics-based reward driven image analysis in microscopy

The rise of electron microscopy has expanded our ability to acquire nanometer and atomically resolved images of complex materials. The resulting vast datasets are typically analyzed by human operators, an intrinsically challenging process due to the multiple possible analysis steps and the corresponding need to build and optimize complex analysis workflows. We present a methodology based on the concept of a Reward Function coupled with Bayesian Optimization, to optimize image analysis workflows dynamically. The Reward Function is engineered to closely align with the experimental objectives and broader context and is quantifiable upon completion of the analysis. Here, cross-section, high-angle annular dark field (HAADF) images of ion-irradiated (Y, Dy)Ba 2 Cu 3 O 7–δ thin-films were used as a model system. The reward functions were formed based on the expected materials density and atomic spacings and used to drive multi-objective optimization of the classical Laplacian-of-Gaussian (LoG) method. These results can be benchmarked against the DCNN segmentation. This optimized LoG* compares favorably against DCNN in the presence of the additional noise. We further extend the reward function approach towards the identification of partially-disordered regions, creating a physics-driven reward function and action space of high-dimensional clustering. We pose that with correct definition, the reward function approach allows real-time optimization of complex analysis workflows at much higher speeds and lower computational costs than classical DCNN-based inference, ensuring the attainment of results that are both precise and aligned with the human-defined objectives.

47 OTHER INSTRUMENTATION↗
DOE OSTI2022

An improved framework for the dynamic likelihood filtering approach to data assimilation

Here, we propose improvements to the Dynamic Likelihood Filter (DLF), a Bayesian data assimilation filtering approach, specifically tailored to wave problems. The DLF approach was developed to address the common challenge in the application of data assimilation to hyperbolic problems in the geosciences and in engineering, where observation systems are sparse in space and time. When these observations have low uncertainties, as compared to model uncertainties, the DLF exploits the inherent nature of information and uncertainties to propagate along characteristics to produce estimates that are phase aware as well as amplitude aware, as would be the case in the traditional data assimilation approach. Along characteristics, the stochastic partial differential equations underlying the linear or nonlinear stochastic dynamics are differential equations. This study focuses on developing the explicit challenges of relating dynamics and uncertainties in the Eulerian and Lagrangian frames via dynamic Gaussian processes. It also implements the approach using the ensemble Kalman filter (EnKF) and compares the DLF approach to the conventional one with respect to wave amplitude and phase estimates in linear and nonlinear wave problems. Numerical comparisons show that the DLF/EnKF outperforms the EnKF estimates, when applied to linear and nonlinear wave problems. This advantage is particularly noticeable when sparse, low uncertainty observations are used.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2026

Bayesian optimization of laser wakefield acceleration via spectral pulse shaping

In this paper, we investigate the effect of spectral pulse shaping of the laser driver on the performance of channel-guided, laser–plasma accelerators. The study was carried out with the assistance of Bayesian optimization using particle-in-cell simulations. We used a realistic plasma profile based on a novel optical-field-ionized channel technique with ionization injection and low, on-axis plasma densities to maximize the energy gain of the electron bunch trailing the laser. Spectral shaping allows us to modify the temporal profile of the laser driver while keeping the laser energy constant, affecting the acceleration and injection processes. In addition, we consider how modifying the plasma channel parameters may affect the target outputs. Given the complexity and breadth of the parameter space in question, we used numerical optimization to identify high-performing configurations. In particular, we found laser profiles with additional spectral content that, when used with optimal plasma channel parameters, result in charge content an order of magnitude higher than the baseline Gaussian case while also increasing the mean energy of the electron bunch.

Physics - Plasma physics↗
DOE OSTI2025

Order-by-order uncertainties of nucleon-nucleon Wolfenstein amplitudes in chiral effective field theory

Quantum mechanical invariance principles dictate the most general operator structure that can be present in the nucleon-nucleon (NN) interaction. Five independent operators appear in the on-shell NN amplitude together with five corresponding coefficient functions. The usual choice for these coefficient functions is known as the NN Wolfenstein amplitudes. We analyze the order-by-order convergence of each of the five NN Wolfenstein amplitudes predicted by a semilocal coordinate space potential implementation of chiral effective field theory (𝜒⁢EFT). We do this at laboratory kinetic energies between 25 and 200 MeV for both neutron-proton and proton-proton scattering. Our analysis uses the Gaussian-process methods developed by the BUQEYE Collaboration to describe the contributions of each 𝜒⁢EFT order, and so yields truncation uncertainties for each Wolfenstein amplitude that are correlated across scattering angles. We combine information on the size of different orders in the EFT to infer the 𝜒⁢EFT breakdown scale for each amplitude, finding, on average, Λ 𝑏 between 750 and 800 MeV. Furthermore, with this choice of Λ 𝑏 , the EFT truncation uncertainties cover both higher-order results and empirical Wolfenstein amplitudes well for all orders other than the leading order.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗
DOE OSTI2023

Sequential Bayesian Experimental Design for Calibration of Expensive Simulation Models

Simulation models of critical systems often have parameters that need to be calibrated using observed data. For expensive simulation models, calibration is done using an emulator of the simulation model built on simulation output at different parameter settings. Using intelligent and adaptive selection of parameters to build the emulator can drastically improve the efficiency of the calibration process. The article proposes a sequential framework with a novel criterion for parameter selection that targets learning the posterior density of the parameters. The emergent behavior from this criterion is that exploration happens by selecting parameters in uncertain posterior regions while simultaneously exploitation happens by selecting parameters in regions of high posterior density. Furthermore, the advantages of the proposed method are illustrated using several simulation experiments and a nuclear physics reaction model.

97 MATHEMATICS AND COMPUTING↗