Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Bayesian Gaussian process”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 145 records · Page 8
All sourcesNASA NTRSDOE OSTI
DOE OSTI2025

Fully Bayesian Analysis With Model Inadequacy Correction For Nuclear Graphite Property Models With Hierarchical Variance Structure

Nuclear-grade graphites are extensively utilized in the core designs of various advanced nuclear reactors. Within the reactor environment, graphite is subjected to prolonged exposure to extreme conditions, including high temperatures, radiation, and potentially molten salt and oxygen. Such exposure can induce several degradation mechanisms in graphite, such as nonuniform volumetric strains caused by irradiation and thermal expansion, leading to stresses that may compromise the performance of graphite components. Assessing component integrity, forecasting component performance over the reactor's lifespan, and developing design standards necessitate robust tools for predicting fracture initiation and propagation in graphite structural components within nuclear reactors. This code enables the Bayesian calibration of properties for nuclear-grade graphites. Using a hierarchical Bayesian approach, multiple experimental data sources are combined to develop Gaussian process models for the properties. Using the Kennedy O'Hagan framework, the uncertainties due inadequacies in the model and the inherent spread in the experimental data are quantified.

Dhulipala, Som Lakshmi NarasimhaLakshmi Narasimha ↗
DOE OSTI2025

MOOSE ProbML: Parallelizable Probabilistic Machine Learning and Uncertainty Quantification Capabilities

The Multiphysics Object Oriented Simulation Environment (MOOSE) is a widely used open- source finite element software for performing multiphysics multiscale simulations in a massively parallel fashion. Recently, the computational team at Idaho National Laboratory (INL) has implemented Probabilistic Machine Learning (ProbML) capabilities in MOOSE—in a parallelized fashion—and enable active learning with large-scale computational models for tasks such as surrogate model development, scale bridging, forward/inverse uncertainty quantification (UQ), Bayesian optimization, etc. This presentation summarizes these developments in MOOSE along with demonstrations on several real applications relevant to nuclear energy. At the fundamental level, samplers like Monte Carlo/Latin Hypercube, variance reduction, parallelized Markov Chain Monte Carlo (MCMC) support uncertainty propagation in both forward and inverse settings. These samplers can be integrated with the Gaussian processes (GP) suite in MOOSE, which offer several variants like scalar GPs, multi-output GPs, and deep GPs, to enable active learning. These GPs can be tuned using gradient-based optimization methods like Adam and its variants or gradient-free methods like the elliptical slice sampler (a variant of MCMC adept under Gaussian settings) for more complex covariance kernels or likelihoods whose gradient computations can be cumbersome. A variety of batch acquisition functions permit parallelized evaluation of the computational model and support different learning objectives with high efficiency like Bayesian inference, global surrogate development, optimization, etc. Furthermore, libtorch integration supports training, evaluation, and re-training of neural networks and other complex machine learning models in active learning settings. The impacts of these developments are shown on several real applications: (1) nuclear fuel inverse UQ and model inadequacy assessment using the Kennedy O’Hagan framework; (2) uncertainty aware surrogate modeling for additive manufacturing to predict field quantities; (3) nuclear reactor rare events analysis; and (4) complex fluid flow prediction using a global surrogate with quantified prediction uncertainty. Finally, the outlook of MOOSE ProbML is discussed for both outer-loop and inner-loop computations in the broad view to accelerate fuels and materials qualification, address gaps in knowledge and data, and assess new reactor/fuel systems.

11 - NUCLEAR FUEL CYCLE AND FUEL MATERIALS↗
DOE OSTI2020

Featureless adaptive optimization accelerates functional electronic materials design

Electronic materials that exhibit phase transitions between metastable states (e.g., metal-insulator transition materials with abrupt electrical resistivity transformations) are challenging to decode. For these materials, conventional machine learning methods display limited predictive capability due to data scarcity and the absence of features that impede model training. In this article, we demonstrate a discovery strategy based on multi-objective Bayesian optimization to directly circumvent these bottlenecks by utilizing latent variable Gaussian processes combined with high-fidelity electronic structure calculations for validation in the chalcogenide lacunar spinel family. We directly and simultaneously learn phase stability and bandgap tunability from chemical composition alone to efficiently discover all superior compositions on the design Pareto front. Previously unidentified electronic transitions also emerge from our featureless adaptive optimization engine. Our methodology readily generalizes to optimization of multiple properties, enabling co-design of complex multifunctional materials, especially where prior data is sparse.

36 MATERIALS SCIENCE↗
DOE OSTI2021

Efficient Active Learning for Gaussian Process Classification by Error Reduction

Active learning sequentially selects the best instance for labeling by optimizing an acquisition function to enhance data/label efficiency. The selection can be either from a discrete instance set (pool-based scenario) or a continuous instance space (query synthesis scenario). In this work, we study both active learning scenarios for Gaussian Process Classification (GPC). The existing active learning strategies that maximize the Estimated Error Reduction (EER) aim at reducing the classification error after training with the new acquired instance in a onestep-look-ahead manner. The computation of EER-based acquisition functions is typically prohibitive as it requires retraining the GPC with every new query. Moreover, as the EER is not smooth, it can not be combined with gradient-based optimization techniques to efficiently explore the continuous instance space for query synthesis. To overcome these critical limitations, we develop computationally efficient algorithms for EER-based active learning with GPC. Further, we derive the joint predictive distribution of label pairs as a one-dimensional integral, as a result of which the computation of the acquisition function avoids retraining the GPC for each query, remarkably reducing the computational overhead. We also derive the gradient chain rule to efficiently calculate the gradient of the acquisition function, which leads to the first query synthesis active learning algorithm implementing EER-based strategies. Our experiments clearly demonstrate the computational efficiency of the proposed algorithms. We also benchmark our algorithms on both synthetic and real-world datasets, which show superior performance in terms of sampling efficiency compared to the existing state-of-the-art algorithms.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2024

Hybrid Parameter Search and Dynamic Model Selection for Mixed-Variable Bayesian Optimization

Herein this article presents a new type of hybrid model for Bayesian optimization (BO) adept at managing mixed variables, encompassing both quantitative (continuous and integer) and qualitative (categorical) types. Our proposed new hybrid models (named hybridM) merge the Monte Carlo Tree Search structure (MCTS) for categorical variables with Gaussian Processes (GP) for continuous ones. hybridM leverages the upper confidence bound tree search (UCTS) for MCTS strategy, showcasing the tree architecture’s integration into Bayesian optimization. Our innovations, including dynamic online kernel selection in the surrogate modeling phase and a unique UCTS search strategy, position our hybrid models as an advancement in mixed-variable surrogate models. Numerical experiments underscore the superiority of hybrid models, highlighting their potential in Bayesian optimization.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2021

Coupling 1D xRAGE simulations with machine learning for graded inner shell design optimization in double shell capsules

Advances in machine learning provide the ability to leverage data from expensive simulations of high-energy-density experiments to significantly cut down on computational time and costs associated with the search for optimal target designs. This study presents an application of cutting-edge Bayesian optimization methods to the one-dimensional (1D) design optimization of double shell graded layer targets for inertial confinement fusion experiments. This investigation attempts to reduce hydrodynamic instabilities while retaining high yields for future NIF experiments. Machine learning methods can use predictive physics simulations to identify graded layer designs from within the vast design space that demonstrate high predicted performance, including novel designs with high uncertainty in performance that may hold unexpected promise. By applying machine learning tools to the simulation design, we map the trade-off between 1D yield and instability, specifically isolating parameter ranges, which maintain high performance while showing significantly improved Rayleigh–Taylor stability over the point design. Furthermore, the groundwork laid in this study will be a useful design tool for future NIF experiments with graded layer targets.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗
DOE OSTI2024

Neural network emulation of flow in heavy-ion collisions at intermediate energies

Applications of new techniques in machine learning are speeding up progress in research in various fields. In this work, we construct and evaluate a deep neural network (DNN) to be used within a Bayesian statistical framework as a faster and more reliable alternative to the Gaussian process (GP) emulator of an isospin-dependent Boltzmann-Uehling-Uhlenbeck (IBUU) transport model simulator of heavy-ion reactions at intermediate beam energies. We found strong evidence of the DNN being able to emulate the IBUU simulator's prediction on the strengths of protons' directed and elliptical flow very efficiently even with small training datasets and with accuracy about ten times higher than the GP. Here, limitations of our present work and future improvements are also discussed.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗
DOE OSTI2026

A Machine Learning based Approach of Estimating Equivalent Circuit Model Parameters at Different SoCs of Li-ion Batteries from Voltage Relaxation

Abstract: In this study, an approach of estimating the equivalent circuit model (ECM) parameters for Li-ion batteries (LIBs) is proposed based on the voltage value at different intervals while relaxing the LIB after discharge. The typical approach for estimating ECM parameters of a LIB is to conduct electrochemical impedance spectroscopy (EIS) measurements at different frequencies and fit them to a predefined circuit model, which requires additional measuring arrangements and specialized devices. The proposed methodology utilizes four different voltages at 0s, 60s, 360s, and 1800s alongside the specific state of charge (SoC) value for a specific constant discharge current value of ~1C until the relaxation stage to train and evaluate three regression-based machine learning models— Support Vector Regression (SVR), Extreme Gradient Boosting (XGBoost), and Gaussian Process Regression (GPR)—for estimating the ECM parameters of the selected model. Bayesian optimization is employed for hyperparameter tuning to achieve optimal performance for all the regressor models, among which, the GPR provided the best performance with the root-mean-squared error (RMSE) of less than 4x10-4 on average for the resistive components and less than 0.27 for capacitive components with excellent R2 scores. The simplicity of the approach enables it to eliminate the need for sophisticated measuring equipment and computation power.

Sagar, Md. Samiul [The University of Alabama (UA)]↗
DOE OSTI2023

Uncertainty Quantification for Multiphase Computational Fluid Dynamics Closure Relations with a Physics-Informed Bayesian Approach

Multiphase Computational Fluid Dynamics (MCFD) based on the two-fluid model is considered a promising tool to model complex two-phase flow systems. MCFD simulation can predict local flow features without resolving interfacial information. As a result, the MCFD solver relies on closure relations to describe the interaction between the two phases. Those empirical or semi-mechanistic closure relations constitute a major source of uncertainty for MCFD predictions. In this paper, we leverage a physics-informed uncertainty quantification (UQ) approach to inversely quantify the closure relations’ model form uncertainty in a physically consistent manner. This proposed approach considers the model form uncertainty terms as stochastic fields that are additive to the closure relation outputs. Combining dimensionality reduction and Gaussian processes, the posterior distribution of the stochastic fields can be effectively quantified within the Bayesian framework with the support of experimental measurements. As this UQ approach is fully integrated into the MCFD solving process, the physical constraints of the system can be naturally preserved in the UQ results. Here, in a case study of adiabatic bubbly flow, we demonstrate that this UQ approach can quantify the model form uncertainty of the MCFD interfacial force closure relations, thus effectively improving the simulation results with relatively sparse data support.

42 ENGINEERING↗
DOE OSTI2022

Machine learning-assisted ultrafast flash sintering of high-performance and flexible silver–selenide thermoelectric devices

Flexible thermoelectric generators (TEGs) have shown immense potential for serving as a power source for wearable electronics and the Internet of Things. A key challenge preventing large-scale application of TEGs lies in the lack of a high-throughput processing method, which can sinter thermoelectric (TE) materials rapidly while maintaining their high thermoelectric properties. Herein, we integrate high-throughput experimentation and Bayesian optimization (BO) to accelerate the discovery of the optimum sintering conditions of silver–selenide TE films using an ultrafast intense pulsed light (flash) sintering technique. Due to the nature of the high-dimensional optimization problem of flash sintering processes, a Gaussian process regression (GPR) machine learning model is established to rapidly recommend the optimum flash sintering variables based on Bayesian expected improvement. For the first time, an ultrahigh-power factor flexible TE film (a power factor of 2205 μW m -1 K -2 with a zT of 1.1 at 300 K) is demonstrated with a sintering time less than 1.0 second, which is several orders of magnitude shorter than that of conventional thermal sintering techniques. Further, the films also show excellent flexibility with 92% retention of the power factor (PF) after 10 3 bending cycles with a 5 mm bending radius. In addition, a wearable thermoelectric generator based on the flash-sintered films generates a very competitive power density of 0.5 mW cm -2 at a temperature difference of 10 K. This work not only shows the tremendous potential of high-performance and flexible silver–selenide TEGs but also demonstrates a machine learning-assisted flash sintering strategy that could be used for ultrafast, high-throughput and scalable processing of functional materials for a broad range of energy and electronic applications.

25 ENERGY STORAGE↗
DOE OSTI2024

Toward Accelerating Discovery via Physics-Driven and Interactive Multifidelity Bayesian Optimization

Both computational and experimental material discovery bring forth the challenge of exploring multidimensional and often nondifferentiable parameter spaces, such as phase diagrams of Hamiltonians with multiple interactions, composition spaces of combinatorial libraries, processing spaces, and molecular embedding spaces. Often these systems are expensive or time consuming to evaluate a single instance, and hence classical approaches based on exhaustive grid or random search are too data intensive. This resulted in strong interest toward active learning methods such as Bayesian optimization (BO) where the adaptive exploration occurs based on human learning (discovery) objective. However, classical BO is based on a predefined optimization target, and policies balancing exploration and exploitation are purely data driven. In practical settings, the domain expert can pose prior knowledge of the system in the form of partially known physics laws and exploration policies often vary during the experiment. Here, we propose an interactive workflow building on multifidelity BO (MFBO), starting with classical (data-driven) MFBO, then expand to a proposed structured (physics-driven) structured MFBO (sMFBO), and finally extend it to allow human-in-the-loop interactive interactive MFBO (iMFBO) workflows for adaptive and domain expert aligned exploration. These approaches are demonstrated over highly nonsmooth multifidelity simulation data generated from an Ising model, considering spin–spin interaction as parameter space, lattice sizes as fidelity spaces, and the objective as maximizing heat capacity. Detailed analysis and comparison show the impact of physics knowledge injection and real-time human decisions for improved exploration with increased alignment to ground truth. Here, the associated notebooks allow to reproduce the reported analyses and apply them to other systems.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2022

AI and extreme scale computing to learn and infer the physics of higher order gravitational wave modes of quasi-circular, spinning, non-precessing black hole mergers

We use artificial intelligence (AI) to learn and infer the physics of higher order gravitational wave modes of quasi-circular, spinning, non precessing binary black hole mergers. We trained AI models using 14 million waveforms, produced with the surrogate model NRHybSur3dq8, that include modes up to $\ell$ ≤ 4 and (5,5), except for (4,0) and (4,1), that describe binaries with mass-ratios $\textit{q}$ ≤ 8, individual spins $s^z_{\{1,2\}} \in$[–0.8,0.8], and inclination angle $θ \in$ [0,π]. Our probabilistic AI surrogates can accurately constrain the mass-ratio, individual spins, effective spin, and inclination angle of numerical relativity waveforms that describe such signal manifold. We compared the predictions of our AI models with Gaussian process regression, random forest, k-nearest neighbors, and linear regression, and with traditional Bayesian inference methods through the PyCBC Inference toolkit, finding that AI outperforms all these approaches in terms of accuracy, and are between three to four orders of magnitude faster than traditional Bayesian inference methods. Our AI surrogates were trained within 3.4 hours using distributed training on 1,536 NVIDIA V100 GPUs in the Summit supercomputer.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2022

Descriptor Aided Bayesian Optimization for Many-Level Qualitative Variables With Materials Design Applications

Abstract Engineering design often involves qualitative and quantitative design variables, which requires systematic methods for the exploration of these mixed-variable design spaces. Expensive simulation techniques, such as those required to evaluate optimization objectives in materials design applications, constitute the main portion of the cost of the design process and underline the need for efficient search strategies—Bayesian optimization (BO) being one of the most widely adopted. Although recent developments in mixed-variable Bayesian optimization have shown promise, the effects of dimensionality of qualitative variables have not been well studied. High-dimensional qualitative variables, i.e., with many levels, impose a large design cost as they typically require a larger dataset to quantify the effect of each level on the optimization objective. We address this challenge by leveraging domain knowledge about underlying physical descriptors, which embody the physics of the underlying physical phenomena, to infer the effect of unobserved levels that have not been sampled yet. We show that physical descriptors can be intuitively embedded into the latent variable Gaussian process approach—a mixed-variable GP modeling technique—and used to selectively explore levels of qualitative variables in the Bayesian optimization framework. This physics-informed approach is particularly useful when one or more qualitative variables are high dimensional (many-level) and the modeling dataset is small, containing observations for only a subset of levels. Through a combination of mathematical test functions and materials design applications, our method is shown to be robust to certain types of incomplete domain knowledge and significantly reduces the design cost for problems with high-dimensional qualitative variables.

Engineering↗
DOE OSTI2023

Bayesian sequential optimal experimental design for nonlinear models using policy gradient reinforcement learning

We present a mathematical framework and computational methods for optimally designing a finite sequence of experiments. This sequential optimal experimental design (sOED) problem is formulated as a finite-horizon partially observable Markov decision process (POMDP) under a Bayesian setting and with information-theoretic utilities. The formulation is general and may accommodate continuous random variables, non-Gaussian posteriors, and nonlinear forward models. The sOED design policy incorporates elements of feedback and lookahead simultaneously, and we show it to generalize the commonly-used batch and greedy design strategies. We solve for the sOED policy using the policy gradient (PG) method from reinforcement learning, and provide a derivation for the PG expression in the sOED context. Adopting an actor-critic approach, the policy and value functions are parameterized using deep neural networks and improved via PG estimates produced from simulated episodes of designs and observations. The new PG-sOED algorithm is first validated on a linear-Gaussian benchmark, and then compared against other design baselines on a sensor movement problem for contaminant source inversion in a convection-diffusion field. As a result, we provide explanation for the policy behaviors using knowledge of the underlying physical process.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2023

MACHINE LEARNING-ENABLED PREDICTION OF TRANSIENT INJECTION MAP IN AUTOMOTIVE INJECTORS WITH UNCERTAINTY QUANTIFICATION

Accurate prediction of injection profiles is a critical aspect of linking injector operation with engine performance and emissions. However, highly resolved injector simulations can take one to two weeks of wall-clock time, which is incompatible with engine design cycles with desired turnaround times of less than a day. Hence, it is important to reduce the time-to-solution of the internal flow simulations by several orders of magnitude to make it compatible with engine simulations. This work demonstrates a data-driven approach for tackling the computational overhead of injector simulations, whereby the transient injection profiles are emulated for a side-oriented, single-hole diesel injector using a Bayesian machine-learning framework. First, an interpretable Bayesian learning strategy was employed to understand the effect of design parameters on the total void fraction field. Then, autoencoders are utilized for efficient dimensionality reduction of the flowfields. Gaussian process models are finally used to predict the spatiotemporal void fraction field at the injector exit for unknown operating conditions. The Gaussian process models produce principled uncertainty estimates associated with the emulated flowfields, which provide the engine designer with valuable information of where the data-driven predictions can be trusted in the design space. The Bayesian flowfield predictions are compared with the corresponding predictions from a deep neural network, which has been transfer-learned from static needle simulations from a previous work by the authors. The emulation framework can predict the void fraction field at the exit of the orifice within a few seconds, thus achieving a speed-up factor of up to 38 x 10(6) over the traditional simulation-based approach of generating transient injection maps.

machine learning↗
DOE OSTI2026

GP Cosmology Surrogate v1.0

GP Cosmology Surrogate is a Python library for building and training a generalized multi-output Gaussian process (GP) framework of @takhtaganov2021cosmic. In this approach, the surrogate is constructed sequentially, guided by a Bayesian optimization acquisition function that targets reduction of emulation error in the regions most consistent with the observational data. This adaptive design concentrates computational resources where they have the greatest impact on inference accuracy. The library supports efficient training for separable GP kernels, which allows the use of Kronecker algebra to handle high-dimensional input spaces and large numbers of correlated outputs. This makes it well suited for applications such as modeling cosmological power spectra, large-scale physical simulations, and multi-output hyperparameter tuning. By combining scalable multi-output GP modeling with data-driven adaptive sampling, GPsurrogate enables parameter inference and optimization with substantially fewer simulations than conventional space-filling designs.

Lukic, Zarija [Lawrence Berkeley National Laborato↗