Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Bayesian Gaussian process”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 37 records · Page 2
All sourcesNASA NTRSDOE OSTI
DOE OSTI2022

Gaussian processes meet NeuralODEs: a Bayesian framework for learning the dynamics of partially observed systems from scarce and noisy data

We present a machine learning framework (GP-NODE) for Bayesian model discovery from partial, noisy and irregular observations of nonlinear dynamical systems. The proposed method takes advantage of differentiable programming to propagate gradient information through ordinary differential equation solvers and perform Bayesian inference with respect to unknown model parameters using Hamiltonian Monte Carlo sampling and Gaussian Process priors over the observed system states. This allows us to exploit temporal correlations in the observed data, and efficiently infer posterior distributions over plausible models with quantified uncertainty. The use of the Finnish Horseshoe as a sparsity-promoting prior for free model parameters also enables the discovery of parsimonious representations for the latent dynamics. A series of numerical studies is presented to demonstrate the effectiveness of the proposed GP-NODE method including predator–prey systems, systems biology and a 50-dimensional human motion dynamical system. This article is part of the theme issue ‘Data-driven prediction in dynamical systems’.

Science & Technology - Other Topics↗
DOE OSTI2024

A Bayesian Learning Approach to Wireless Outdoor Heatmap Construction using Deep Gaussian Process

We present a novel Bayesian learning approach to outdoor radio heatmap construction utilizing deep Gaussian process (GP). The proposed approach employs a two-layer hierarchy which consists of two cascaded Gaussian processes that are capable of modeling more complex input-output relations than standard single-layer Gaussian processes. Since deriving the exact model likelihood is challenging, a lower bound is optimized instead so that gradient descent-based methods can be performed to find out the optimal model parameters. Typically, inducing points are used in GPs to facilitate low-rank approximation of covariance (kernel) matrices for computation speedup. However, the inaccuracy induced by inducing points can accumulate when stacking multiple layers of GP which may hinder the performance of deep GP. Moreover, since inducing points need to be learned, having them at all layers of deep GP also incurs computational burden. To overcome the above challenges, in contrast to the canonical deep GP model, we use a modified architecture where a full standard GP resides in the first layer and inducing points are only introduced for the second layer. This modified architecture strikes a balance between model accuracy and training complexity. In the proposed model, the noise parameter of the first GP layer is also eliminated to improve the training efficiency as the noise parameter at the output of the second layer suffices to model the uncertainty in the output. The proposed approach is evaluated on real-world datasets, in the form of location-Received Signal Strength (RSS) pairs, collected from the Platform for Open Wireless Data-driven Experimental Research (POWDER) located at the campus of the University of Utah. Experiment results show that the proposed approach can achieve smaller prediction errors on various training and testing data configurations than DNN-based and GP-based methods.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗
DOE OSTI2025

Bayesian D‐Optimal Designs for Gaussian Process Surrogate Models

Computer experiments often employ space-filling strategies to create surrogate models with strong predictive performance. The impact of model parameter estimation for Gaussian process surrogates, however, is often overlooked. Obtaining a better initial estimate of the covariance lengthscale parameter, θ, can greatly improve the resulting Gaussian process fit through more effective sequential acquisitions during active learning. In this work, we propose a novel initial design maximizing the Bayesian D-optimality criterion of the Gaussian process lengthscale parameter. Previously published results have shown the emphasis on lengthscale estimation to be promising, but relied on an empirically driven design creation process. Our Bayesian D-optimal designs are rooted in information theory and lead to more informative sequential acquisitions by improving lengthscale estimation. In many cases, these gains eventually result in better surrogates than those seeded with space-filling initial designs. Furthermore, Bayesian D-optimal designs can be tailored to either isotropic or anisotropic covariance structures, and the Bayesian framework enables the inclusion of prior knowledge in the design process, offering greater flexibility and adaptability. Through several simulation studies, we demonstrate the advantages of Bayesian D-optimal designs in terms of both lengthscale estimation accuracy and predictive performance during active learning.

Bayesian experimental design↗
DOE OSTI2026

Gaussian processes for inferring parton distributions

The extraction of parton distribution functions (PDFs) from experimental or lattice QCD data is an ill-posed inverse problem, where regularization strongly impacts both systematic uncertainties and the reliability of the results. We study a framework based on Gaussian Process Regression (GPR) to reconstruct PDFs from lattice QCD matrix elements. Within a Bayesian framework, Gaussian processes serve as flexible priors that encode uncertainties, correlations, and constraints without imposing rigid functional forms. We investigate a wide range of kernel choices, mean functions, and hyperparameter treatments. We quantify information gained from the data using the Kullback-Leibler divergence. Synthetic data tests demonstrate the consistency and robustness of the method. Our study establishes GPR as a systematic and non-parametric approach to PDF reconstruction, offering controlled uncertainty estimates and reduced model bias in lattice QCD analyses.

hadronic spectroscopy↗
DOE OSTI2023

Uncertainty quantification in machine learning for engineering design and health prognostics: A tutorial

On top of machine learning (ML) models, uncertainty quantification (UQ) functions as an essential layer of safety assurance that could lead to more principled decision making by enabling sound risk assessment and management. The safety and reliability improvement of ML models empowered by UQ has the potential to significantly facilitate the broad adoption of ML solutions in high-stakes decision settings, such as healthcare, manufacturing, and aviation, to name a few. In this tutorial, we aim to provide a holistic lens on emerging UQ methods for ML models with a particular focus on neural networks and the applications of these UQ methods in tackling engineering design as well as prognostics and health management problems. Towards this goal, we start with a comprehensive classification of uncertainty types, sources, and causes pertaining to UQ of ML models. Next, we provide a tutorial-style description of several state-of-the-art UQ methods: Gaussian process regression, Bayesian neural network, neural network ensemble, and deterministic UQ methods focusing on spectral-normalized neural Gaussian process. Established upon the mathematical formulations, we subsequently examine the soundness of these UQ methods quantitatively and qualitatively (by a toy regression example) to examine their strengths and shortcomings from different dimensions. Then, we review quantitative metrics commonly used to assess the quality of predictive uncertainty in classification and regression problems. Afterward, we discuss the increasingly important role of UQ of ML models in solving challenging problems in engineering design and health prognostics. In conclusion, two case studies with source codes available on GitHub are used to demonstrate these UQ methods and compare their performance in the life prediction of lithium-ion batteries at the early stage (case study 1) and the remaining useful life prediction of turbofan engines (case study 2).

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2021

A comparison of Gaussian processes and neural networks for computer model emulation and calibration

The Department of Energy relies on complex physics simulations for prediction in domains like cosmology, nuclear theory, and materials science. These simulations are often extremely computationally intensive, with some requiring days or weeks for a single simulation. In order to assure their accuracy, these models are calibrated against observational data in order to estimate inputs and systematic biases. Because of their great computational complexity, this process typically requires the construction of an emulator, a fast approximation to the simulation. In this paper, two emulator approaches are compared: Gaussian process regression and neural networks. Their emulation accuracy and calibration performance on three real problems of Department of Energy interest is considered. On these problems, the Gaussian process emulator tends to be more accurate with narrower, but still well-calibrated uncertainty estimates. The neural network emulator is accurate, but tends to have large uncertainty on its predictions. Finally, as a result, calibration with the Gaussian process emulator produces more constrained posteriors that still perform well in prediction.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2025

Active oversight and quality control in standard Bayesian optimization for autonomous experiments

The fusion of experimental automation and machine learning has catalyzed a new era in materials research, prominently featuring Gaussian Process (GP) Bayesian Optimization (BO) driven autonomous experiments. Here we introduce a Dual-GP approach that enhances traditional GPBO by adding a secondary surrogate model to dynamically constrain the experimental space based on real-time assessments of the raw experimental data. This Dual-GP approach enhances the optimization efficiency of traditional GPBO by isolating more promising space for BO sampling and more valuable experimental data for primary GP training. We also incorporate a flexible, human-in-the-loop intervention method in the Dual-GP workflow to adjust for unanticipated results. We demonstrate the effectiveness of the Dual-GP model with synthetic model data and implement this approach in autonomous pulsed laser deposition experimental data. This Dual-GP approach has broad applicability in diverse GPBO-driven experimental settings, providing a more adaptable and precise framework for refining autonomous experimentation for more efficient optimization.

36 MATERIALS SCIENCE↗
DOE OSTI2026

Deep Gaussian process-based cost-aware batch Bayesian optimization for complex materials design campaigns

The accelerating pace and expanding scope of materials discovery demand optimization frameworks that efficiently navigate vast design spaces with complex response surfaces while judiciously allocating limited evaluation resources. We present a cost-aware, batch Bayesian optimization scheme powered by deep Gaussian process (DGP) surrogates and a heterotopic querying strategy. Our DGP surrogate, formed by stacking GP layers, models complex hierarchical relationships among high-dimensional compositional features and captures correlations across multiple target properties, propagating uncertainty through successive layers. We integrate evaluation cost into an upper-confidence-bound acquisition extension, which, together with heterotopic querying, proposes small batches of candidates in parallel, balancing exploration of under-characterized regions with exploitation of high-mean, low-variance predictions across correlated properties. Applied to refractory high-entropy alloys for high-temperature applications, our framework converges to optimal formulations in fewer iterations with cost-aware queries than conventional GP-based BO, highlighting the value of deep, uncertainty-aware, cost-sensitive strategies in materials campaigns.

36 MATERIALS SCIENCE↗
DOE OSTI2022

Uncertainty-aware mixed-variable machine learning for materials design

Abstract Data-driven design shows the promise of accelerating materials discovery but is challenging due to the prohibitive cost of searching the vast design space of chemistry, structure, and synthesis methods. Bayesian optimization (BO) employs uncertainty-aware machine learning models to select promising designs to evaluate, hence reducing the cost. However, BO with mixed numerical and categorical variables, which is of particular interest in materials design, has not been well studied. In this work, we survey frequentist and Bayesian approaches to uncertainty quantification of machine learning with mixed variables. We then conduct a systematic comparative study of their performances in BO using a popular representative model from each group, the random forest-based Lolo model (frequentist) and the latent variable Gaussian process model (Bayesian). We examine the efficacy of the two models in the optimization of mathematical functions, as well as properties of structural and functional materials, where we observe performance differences as related to problem dimensionality and complexity. By investigating the machine learning models’ predictive and uncertainty estimation capabilities, we provide interpretations of the observed performance differences. Our results provide practical guidance on choosing between frequentist and Bayesian uncertainty-aware machine learning models for mixed-variable BO in materials design.

36 MATERIALS SCIENCE↗
DOE OSTI2022

Machine learning in nuclear materials research

Nuclear materials are often demanded to function for extended time in extreme environments, including high radiation fluxes with associated transmutations, high temperature and temperature gradients, mechanical stresses, and corrosive coolants. They also have a wide range of microstructural and chemical makeups, resulting in multifaceted and often out-of-equilibrium interactions. Machine learning (ML) is increasingly being used to tackle these complex time-dependent interactions and aid researchers in developing models and making predictions, sometimes with better accuracy than traditional modeling that focuses on one or two parameters at a time. Conventional practices of acquiring new experimental data in nuclear materials research are often slow and expensive, limiting the opportunity for data-centric ML, but new methods are changing that paradigm. Here we review high-throughput computational and experimental data approaches, especially robotic experimentation and active learning that is based on Gaussian process and Bayesian optimization. We show ML examples in structural materials (e.g., reactor pressure vessel (RPV) alloys and radiation detecting scintillating materials) and highlight new techniques of high-throughput sample preparation and characterizations, and automated radiation/environmental exposures and real-time online diagnostics. Herein, this review suggests that ML models of material constitutive relations in plasticity, damage, and even electronic and optical responses to radiation are likely to become powerful tools as they develop. Finally, we speculate on how the recent trends of using natural language processing (NLP) to aid the collection and analysis of literature data, interpretable artificial intelligence (AI), and the use of streamlined scripting, database, workflow management, and cloud computing platforms that will soon make the utilization of ML techniques as commonplace as the spreadsheet curve-fitting practices of today.

36 MATERIALS SCIENCE↗
DOE OSTI2020

Quantified uncertainties in fission yields from machine learning

As machine learning methods gain traction in the nuclear physics community, especially those methods that aim to propagate uncertainties to unmeasured quantities, it is important to understand how the uncertainty in the training data coming either from theory or experiment propagates to the uncertainty in the predicted values. Gaussian Processes and Bayesian Neural Networks are being more and more widely used, in particular to extrapolate beyond measured data. However, studies are typically not performed on the impact of the experimental errors on these extrapolated values. In this work, we focus on understanding how uncertainties propagate from input to prediction when using machine learning methods. We use a Mixture Density Network (MDN) to incorporate experimental error into the training of the network and construct uncertainties for the associated predicted quantities. Systematically, we study the effect of the size of the experimental error, both on the reproduced training data and extrapolated predictions for fission yields of actinides.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗
DOE OSTI2026

Automated scanning probe microscopy of combinatorial ferroelectric libraries: Gaussian-process-guided exploration and noise-aware experiment planning

Combinatorial materials libraries provide an efficient route for mapping composition–property relationships, but their broader impact depends on rapid, quantitative, and functionally relevant characterization. Scanning Probe Microscopy (SPM), including piezoresponse force microscopy (PFM), offers significant potential for quantitative, functionally relevant combi-library readouts. Here, we implement a fully automated SPM workflow for ferroelectric combinatorial libraries and benchmark Gaussian-process-based Bayesian optimization strategies for autonomous experiment planning. The workflow integrates automated probe motion, contact optimization, imaging, and dual amplitude resonance tracking-PFM spectroscopy, and uses scalarized spectroscopic observables to guide subsequent measurements. Stage motion, probe engagement, in-contact tuning, imaging, spectroscopy, and the choice of the next measurement location all proceed without human input. We demonstrate the approach on Sm-doped BiFeO 3 and Zn x Mg 1−x O libraries. By comparing vanilla Bayesian optimization with a measured-noise variant, we show that explicit treatment of local reproducibility can improve modeling of composition-dependent response when the measured variance is physically meaningful, but can also reduce robustness when variability is dominated by outliers or topographic artifacts. Furthermore, these results establish automated SPM as a bridge between combinatorial synthesis and quantitative functional characterization.

Liu, Yu [University of Tennessee, Knoxville, TN (U↗
DOE OSTI2022

Nuclear masses learned from a probabilistic neural network

Machine learning methods and uncertainty quantification have been gaining interest throughout the last several years in low-energy nuclear physics. In particular, Gaussian processes and Bayesian neural networks have increasingly been applied to improve mass model predictions while providing well-quantified uncertainties. In this work, we use the probabilistic Mixture Density Network (MDN) to directly predict the mass excess of the 2016 Atomic Mass Evaluation within the range of measured data, and we extrapolate the inferred models beyond available experimental data. The MDN provides not only mean values but also full posterior distributions both within the training set and extrapolated testing set. We show that the addition of physical information to the feature space increases the accuracy of the match to the training data as well as provides for more physically meaningful extrapolations beyond the the limits of experimental data.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗
DOE OSTI2022

Industrial battery operation and utilization in the presence of electrical load uncertainty using Bayesian decision theory

Behind the meter battery storage is becoming increasing popular in all sectors, though enthusiasm has recently lagged in the industrial sector. Even though there may be many factors contributing to this including lack of innovation, prohibitive costs, and undesirable rate structures, a difficulty arises in accounting for uncertainty of electrical load in industrial facilities while still attempting to utilize battery storage as much as possible all while trying to achieve fiscal profitability. Here this study utilizes Gaussian process regression and Bayesian decision theory to organize load data and quantify electrical load uncertainty to properly and effectively discharge industrial battery storage. The study employs a simulation model to set battery load setpoints for the span of the utility billing period according to the degree of risk aversion. This combination of economic analysis according to utility billing period and utilization of degree of risk aversion to make decisions on the uncertainty of the data has not before been applied to battery storage. The method resulted in an annual average reduction of peak demand by 3.8 % at the lowest amount of savings and lowest risk aversion. The highest risk aversion resulted in an annual average reduction of peak demand of 7.5 %. The maximum reduction of peak load in any month was 13.8 % in the month of December with a relatively high risk aversion. With a the highest amount risk aversion tested, the model reduced demand ten of the twelve months of the year.

25 ENERGY STORAGE↗
DOE OSTI2022

Physics makes the difference: Bayesian optimization and active learning via augmented Gaussian process

Abstract Both experimental and computational methods for the exploration of structure, functionality, and properties of materials often necessitate the search across broad parameter spaces to discover optimal experimental conditions and regions of interest in the image space or parameter space of computational models. The direct grid search of the parameter space tends to be extremely time-consuming, leading to the development of strategies balancing exploration of unknown parameter spaces and exploitation towards required performance metrics. However, classical Bayesian optimization (BO) strategies based on the Gaussian process (GP) do not readily allow for the incorporation of the known physical behaviors or past knowledge. Here we explore a hybrid optimization/exploration algorithm created by augmenting the standard GP with a structured probabilistic model of the expected system’s behavior. This approach balances the flexibility of the non-parametric GP approach with a rigid structure of physical knowledge encoded into the parametric model. The fully Bayesian treatment of the latter allows additional control over the optimization via the selection of priors for the model parameters. The method is demonstrated for a noisy version of a standard univariate test function used to evaluate optimization algorithms and further extended to physical lattice models. This methodology is expected to be universally suitable for injecting prior knowledge in the form of physical models and past data in the BO framework.

42 ENGINEERING↗
DOE OSTI2024

A general Bayesian algorithm for the autonomous alignment of beamlines

Autonomous methods to align beamlines can decrease the amount of time spent on diagnostics, and also uncover better global optima leading to better beam quality. The alignment of these beamlines is a high-dimensional expensive-to-sample optimization problem involving the simultaneous treatment of many optical elements with correlated and nonlinear dynamics. Bayesian optimization is a strategy of efficient global optimization that has proved successful in similar regimes in a wide variety of beamline alignment applications, though it has typically been implemented for particular beamlines and optimization tasks. In this paper, we present a basic formulation of Bayesian inference and Gaussian process models as they relate to multi-objective Bayesian optimization, as well as the practical challenges presented by beamline alignment. We show that the same general implementation of Bayesian optimization with special consideration for beamline alignment can quickly learn the dynamics of particular beamlines in an online fashion through hyperparameter fitting with no prior information. We present the implementation of a concise software framework for beamline alignment and test it on four different optimization problems for experiments on X-ray beamlines at the National Synchrotron Light Source II and the Advanced Light Source, and an electron beam at the Accelerator Test Facility, along with benchmarking on a simulated digital twin. We discuss new applications of the framework, and the potential for a unified approach to beamline alignment at synchrotron facilities.

47 OTHER INSTRUMENTATION↗
DOE OSTI2022

Bayesian Active Learning for Scanning Probe Microscopy: From Gaussian Processes to Hypothesis Learning

Recent progress in machine learning methods and the emerging availability of programmable interfaces for scanning probe microscopes (SPMs) have propelled automated and autonomous microscopies to the forefront of attention of the scientific community. However, enabling automated microscopy requires the development of task-specific machine learning methods, understanding the interplay between physics discovery and machine learning, and fully defined discovery workflows. This, in turn, requires balancing the physical intuition and prior knowledge of the domain scientist with rewards that define experimental goals and machine learning algorithms that can translate these to specific experimental protocols. Here, we discuss the basic principles of Bayesian active learning and illustrate its applications for SPM. We progress from the Gaussian process as a simple data-driven method and Bayesian inference for physical models as an extension of physics-based functional fits to more complex deep kernel learning methods, structured Gaussian processes, and hypothesis learning. These frameworks allow for the use of prior data, the discovery of specific functionalities as encoded in spectral data, and exploration of physical laws manifesting during the experiment. Here, the discussed framework can be universally applied to all techniques combining imaging and spectroscopy, SPM methods, nanoindentation, electron microscopy and spectroscopy, and chemical imaging methods and can be particularly impactful for destructive or irreversible measurements.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗