Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Bayesian modeling”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 919 records · Page 51
All sourcesNASA NTRSDOE OSTI
DOE OSTI2022

Descriptor Aided Bayesian Optimization for Many-Level Qualitative Variables With Materials Design Applications

Abstract Engineering design often involves qualitative and quantitative design variables, which requires systematic methods for the exploration of these mixed-variable design spaces. Expensive simulation techniques, such as those required to evaluate optimization objectives in materials design applications, constitute the main portion of the cost of the design process and underline the need for efficient search strategies—Bayesian optimization (BO) being one of the most widely adopted. Although recent developments in mixed-variable Bayesian optimization have shown promise, the effects of dimensionality of qualitative variables have not been well studied. High-dimensional qualitative variables, i.e., with many levels, impose a large design cost as they typically require a larger dataset to quantify the effect of each level on the optimization objective. We address this challenge by leveraging domain knowledge about underlying physical descriptors, which embody the physics of the underlying physical phenomena, to infer the effect of unobserved levels that have not been sampled yet. We show that physical descriptors can be intuitively embedded into the latent variable Gaussian process approach—a mixed-variable GP modeling technique—and used to selectively explore levels of qualitative variables in the Bayesian optimization framework. This physics-informed approach is particularly useful when one or more qualitative variables are high dimensional (many-level) and the modeling dataset is small, containing observations for only a subset of levels. Through a combination of mathematical test functions and materials design applications, our method is shown to be robust to certain types of incomplete domain knowledge and significantly reduces the design cost for problems with high-dimensional qualitative variables.

Engineering↗
DOE OSTI2020

Constraining physical models at gigabar pressures

High-energy-density (HED) experiments in convergent geometry are able to test physical models at pressures beyond hundreds of millions of atmospheres. The measurements from these experiments are generally highly integrated and require unique analysis techniques to procure quantitative infor mation. This work describes a methodology to constrain the physics in convergent HED experiments by adapting the methods common to many other fields of physics. As an example, a mechanical model of an imploding shell is constrained by data from a thin-shelled direct-drive exploding-pusher experiment on the OMEGA Laser System using Bayesian inference, resulting in the reconstruction of the shell dynamics and energy transfer during the implosion. The model is tested by analyzing synthetic data from a 1-D hydrodynamics code and is sampled using a Markov chain Monte Carlo to generate the posterior distributions of the model parameters. As a result, the goal of this work is to demonstrate a general methodology that can be used to draw conclusions from a wide variety of HED experiments.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗
DOE OSTI2021

Constraining Physical Models at Gigabar Pressures

High-energy-density (HED) experiments in convergent geometry are able to test physical models at pressures beyond hundreds of millions of atmospheres. The measurements from these experiments are generally highly integrated and require unique analysis techniques to procure quantitative information. This work describes a methodology to constrain the physics in convergent HED experiments by adapting the methods common to many other fields of physics. As an example, a mechanical model of an imploding shell is constrained by data from a thin-shelled direct-drive exploding-pusher experiment on the OMEGA Laser System using Bayesian inference, resulting in the reconstruction of the shell dynamics and energy transfer during the implosion. The model is tested by analyzing synthetic data from a 1-D hydrodynamics code and is sampled using a Markov chain Monte Carlo to generate the posterior distributions of the model parameters. The goal of this work is to demonstrate a general methodology that can be used to draw conclusions from a wide variety of HED experiments.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗
DOE OSTI2019

A copula-based Bayesian method for probabilistic solar power forecasting

With increased penetration of solar energy sources, solar power forecasting has become more crucial and challenging. This paper proposes a copula-based Bayesian approach to improve probabilistic solar power forecasting by capturing the joint distribution between solar power and ambient temperature. A prior forecast distribution is first obtained using different underlying point forecasting models. Parametric and empirical copulas of solar power and temperature are then developed to update the prior distribution to the posterior forecast distribution. A public solar power database is used to demonstrate effectiveness of the proposed method. Numerical results show that the copula-based Bayesian method outperforms the forecasting method that directly uses temperature as a feature. The Bayesian method is also compared with persistent models and show improved performance. Furthermore, this article includes supplementary material (data and code) for reproducibility.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2021

Improved naive Bayesian probability classifier in predictions of nuclear mass

Recently, novel statistical methods such as neural networks and Bayesian learning methods are implemented to describe the nuclear masses. Based on previous studies, an improved naive Bayesian probability (iNBP) classifier is proposed to study the nuclear masses by refining the results of sophisticated nuclear models. In the iNBP method, the prediction for nuclear masses is treated as a classification problem. The residuals are classified into several groups to generate prior and conditional probabilities, and the posterior probabilities are further determined by the Bayesian formula. We choose the expectation with maximum probability as the final prediction. Reliability of the iNBP method is assessed by analyzing the global optimizations and the extrapolating capabilities. Here, the iNBP method exhibits impressive improvements on global descriptions for different mass models. Moreover, the method shows robust extrapolating capabilities. Results demonstrate the iNBP method can be applied to predict the nuclear masses of unknown regions. Considering the local mass relations, the iNBP method can offer considerable fine-tuning of the mass descriptions from nuclear models. The methodology proposed in this paper can also be applied to other model-based extrapolations of nuclear observables.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗
DOE OSTI2023

Quantifying uncertainty for deep learning based forecasting and flow-reconstruction using neural architecture search ensembles

Classical problems in computational physics such as data-driven forecasting and signal reconstruction from sparse sensors have recently seen an explosion in deep neural network (DNN) based algorithmic approaches. However, most DNN models do not provide uncertainty estimates, which are crucial for establishing the trustworthiness of these techniques in downstream decision making tasks and scenarios. In recent years, ensemble-based methods have achieved significant success for the uncertainty quantification in DNNs on a number of benchmark problems. However, their performance on real-world applications remains under-explored. In this work, we present an automated approach to DNN discovery and demonstrate how this may also be utilized for ensemble-based uncertainty quantification. Specifically, we propose the use of a scalable neural and hyperparameter architecture search for discovering an ensemble of DNN models for complex dynamical systems. We highlight how the proposed method not only discovers high-performing neural network ensembles for our tasks, but also quantifies uncertainty seamlessly. This is achieved by using genetic algorithms and Bayesian optimization for sampling the search space of neural network architectures and hyperparameters. Subsequently, a model selection approach is used to identify candidate models for an ensemble set construction. Afterwards, a variance decomposition approach is used to estimate the uncertainty of the predictions from the ensemble. We demonstrate the feasibility of this framework for two tasks — forecasting from historical data and flow reconstruction from sparse sensors for the sea-surface temperature. In conclusion, we demonstrate superior performance from the ensemble in contrast with individual high-performing models and other benchmarks.

Deep ensembles↗
DOE OSTI2024

High-Dimensional Bayesian Optimization via Semi-Supervised Learning with Optimized Unlabeled Data Sampling

We introduce a novel semi-supervised learning approach, named Teacher-Student Bayesian Optimization (TSBO ), integrating the teacher-student paradigm into BO to minimize expensive labeled data queries for the first time. TSBO incorporates a teacher model, an unlabeled data sampler, and a student model. The student is trained on unlabeled data locations generated by the sampler, with pseudo labels predicted by the teacher. The interplay between these three components implements a unique selective regularization to the teacher in the form of student feedback. This scheme enables the teacher to predict high-quality pseudo labels, enhancing the generalization of the GP surrogate model in the search space. To fully exploit TSBO , we propose two optimized unlabeled data samplers to construct effective student feedback that well aligns with the objective of Bayesian optimization. Furthermore, we quantify and leverage the uncertainty of the teacher-student model for the provision of reliable feedback to the teacher in the presence of risky pseudo-label predictions. TSBO demonstrates significantly improved sample-efficiency in several global optimization tasks under tight labeled data budgets. The implementation is available at https://github.com/reminiscenty/TSBO-Official.

Yin, Yuxuan↗
DOE OSTI2022

General-Purpose Bayesian Tensor Learning With Automatic Rank Determination and Uncertainty Quantification

A major challenge in many machine learning tasks is that the model expressive power depends on model size. Low-rank tensor methods are an efficient tool for handling the curse of dimensionality in many large-scale machine learning models. The major challenges in training a tensor learning model include how to process the high-volume data, how to determine the tensor rank automatically, and how to estimate the uncertainty of the results. While existing tensor learning focuses on a specific task, this paper proposes a generic Bayesian framework that can be employed to solve a broad class of tensor learning problems such as tensor completion, tensor regression, and tensorized neural networks. We develop a low-rank tensor prior for automatic rank determination in nonlinear problems. Our method is implemented with both stochastic gradient Hamiltonian Monte Carlo (SGHMC) and Stein Variational Gradient Descent (SVGD). We compare the automatic rank determination and uncertainty quantification of these two solvers. We demonstrate that our proposed method can determine the tensor rank automatically and can quantify the uncertainty of the obtained results. We validate our framework on tensor completion tasks and tensorized neural network training tasks.

Bayesian inference↗
DOE OSTI2023

Prehydrodynamic evolution in large and small systems

We extend our previous investigation of the effects of prehydrodynamic evolution on final-state observables in heavy-ion collisions [38] to smaller systems. We use a state-of-the-art hybrid model for the numerical simulations with optimal parameters obtained from a previous Bayesian study. By studying p–Pb collisions, we find that the effects due to the assumption of a conformal evolution in the prehydrodynamical stage are even more important in small systems. We also show that this effect depends on the time duration of the pre-equilibrium stage, which is further enhanced in small systems. Finally, we show that the recent proposal of a free-streaming with subluminal velocity for the pre-equilibrium stage, thus effectively breaking conformal invariance, can alleviate the contamination of final-state observables. Furthermore, our study further reinforces the need for moving beyond conformal approaches in pre-equilibrium dynamics modeling, especially when extracting transport coefficients from hybrid models in the high-precision era of heavy-ion collisions.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗
DOE OSTI2023

Adaptive Discovery and Mixed-Variable Optimization of Next Generation Synthesizable Microelectronic Materials

Design of new microelectronic materials is characterized by several challenges such as high-dimensionality of the atomic structure-composition variable space, formidable cost of directly using high-fidelity simulations for design optimization, dispersity in literature-reported similar materials and synthesis methods, complex physical mechanisms, and mixed qualitative and quantitative design variables that lead to a disjointed design space. Even though machine learning (ML) techniques have been employed to expedite materials innovation, existing methods treat ML and design optimization as two separate processes, failing to resolve the fundamental challenges associated with high dimensionality and mixed-variable complexity. We have developed a ML enhanced mixed-variable material design optimization framework to efficiently extract useful information from existing data in literature and physics-based simulations to guide the autonomous search for optimal materials. Our proposed framework is composed of four computational modules: (1) a natural language processing (NLP) based virtual screening module, (2) classification based concept exploration module, (3) a density functional theory (DFT)-based high-fidelity evaluation model, and (4) a novel latent-variable Gaussian process (LVGP) ML model for mixed-variable problems with uncertainty quantification, which seamlessly integrates with Bayesian Optimization (BO) and achieves superb efficiency through embedded physics-based dimension reduction. Our approach is demonstrated and validated using the testbed of functional materials exhibiting metal-insulation transitions (MITs), with the targeted reversible resistivity changes (∼10^5) near room temperature. At the end of the 30-month project, we have developed a series of new ML techniques using NLP, conditional variational autoencoders, active learning, latent-variable Gaussian processes, integrated with Bayesian optimization. Our project has resulted in new predicted MITs compounds and improved understanding of MITs microscopic mechanisms, which in turn will revolutionize microelectronics science to provide energy-saving solutions. Our research has improved both creativity and efficiency in transforming rare-event discoveries of new functional materials to persistent innovations. In addition to open-sourcing the online MIT database and the classification model, the LVGP open source code has been downloaded more than 15,000 times within two years. More than 40 MIT compounds have been identified and many have been pursued experimentally via collaborators. The research results are published in close to 20 collaborative papers in high-impact journals, such as Chem. Mater., Appl. Phys. Rev., Sci. Rep., among others of design space.

36 MATERIALS SCIENCE↗
DOE OSTI2021

Validation Metrics for Fixed Effects and Mixed-Effects Calibration

The modern scientific process often involves the development of a predictive computational model. To improve its accuracy, a computational model can be calibrated to a set of experimental data. A variety of validation metrics can be used to quantify this process. Some of these metrics have direct physical interpretations and a history of use, while others, especially those for probabilistic data, are more difficult to interpret. In this work, a variety of validation metrics are used to quantify the accuracy of different calibration methods. Frequentist and Bayesian perspectives are used with both fixed effects and mixed-effects statistical models. Through a quantitative comparison of the resulting distributions, the most accurate calibration method can be selected. Two examples are included which compare the results of various validation metrics for different calibration methods. It is quantitatively shown that, in the presence of significant laboratory biases, a fixed effects calibration is significantly less accurate than a mixed-effects calibration. This is because the mixed-effects statistical model better characterizes the underlying parameter distributions than the fixed effects model. The results suggest that validation metrics can be used to select the most accurate calibration model for a particular empirical model with corresponding experimental data.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2024

Assessing and benchmarking the fidelity of posterior inference methods for astrophysics data analysis

In this era of large and complex astronomical survey data, interpreting, validating, and comparing inference techniques becomes increasingly difficult. This is particularly critical for emerging inference methods like Simulation-Based Inference (SBI), which offer significant speedup potential and posterior modeling flexibility, especially when deep learning is incorporated. We present a study to assess and compare the performance and uncertainty prediction capability of Bayesian inference algorithms – from traditional MCMC sampling of analytic functions to deep learning-enabled SBI. We focus on testing the capacity of hierarchical inference modeling in those scenarios. Before we extend this study to cosmology, we first use astrophysical simulation data to ensure interpretability. We demonstrate a probabilistic programming implementation of hierarchical and non-hierarchical Bayesian inference using simulations derived from the DeepBench software library, a benchmarking tool developed by our group that generates simple and controllable astrophysical objects from first principles. This study will enable astronomers and physicists to harness the inference potential of these methods with confidence.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗
DOE OSTI2025

Knowledge graph-aided Bayesian active learning for top- K genetic interaction discovery

In silico methods for predicting the effects of multi-gene perturbations hold great promise for advancing functional genomics, computational drug discovery, and disease modeling. However, the development of these predictive algorithms for mammalian systems has been hampered by limited datasets and high experimental costs. In this study, we present a Bayesian active learning framework designed to discover pairwise host gene knockdowns that effectively inhibit viral proliferation in an in vitro HIV-1 infection model. Our method leverages a biological knowledge graph as side information and employs a computationally efficient batch diversification approach. We evaluated this framework using a dataset of viral load measurements obtained from multi-day dual-gene depletion experiments, encompassing all possible pairwise knockdowns of over 350 host genes associated with HIV infection. We demonstrate that our framework rapidly identifies the most effective gene knockdown pairs for reducing viral load. Furthermore, we show that incorporating side information enhances performance during the early stages of active learning (low data regime), while our batch diversification strategy significantly boosts performance in later stages (high data regime). This framework is general and can be adapted to explore gene interactions in other contexts, such as synthetic lethality prediction and mapping epistatic effects across quantitative trait loci.

Computational biology and bioinformatics↗
DOE OSTI2025

Machine Learning-Guided Optimization of SABRE Hyperpolarization for α-Ketoglutarate in Acetone–Water

Signal amplification by reversible exchange (SABRE) is a hyperpolarization method that polarizes target nuclei of metabolites quickly and efficiently. Recent SABRE advances, including Ace-SABRE, yield biocompatible, aqueous solutions of hyperpolarized markers for metabolic monitoring. Building on recent advancements, expanding the substrate scope of Ace-SABRE is desirable. However, SABRE polarization is sensitive to many different parameters; therefore, traditional optimization approaches are experimentally time-consuming. In this proof-of-concept application of machine learning (ML), Bayesian optimization (BO) is used for four important input parameters to model the complex SABRE dynamics while saving experimental time. The presented ML model also provides chemical insights that enable predictions of sample compositions for increased polarization levels. In this paper, we transition from an original average free polarization of p = ∼0.90% to a maximum observed free polarization of p = ∼6.6% for 1- 13 C alpha-ketoglutarate (AKG) with 13 C at natural abundance, utilizing both direct outputs as well as chemical insights revealed by the ML model.

Catalysts↗
DOE OSTI2020

Embedding physics domain knowledge into a Bayesian network enables layer-by-layer process innovation for photovoltaics

Process optimization of photovoltaic devices is a time-intensive, trial-and-error endeavor, which lacks full transparency of the underlying physics and relies on user-imposed constraints that may or may not lead to a global optimum. Herein, we demonstrate that embedding physics domain knowledge into a Bayesian network enables an optimization approach for gallium arsenide (GaAs) solar cells that identifies the root cause(s) of underperformance with layer-by-layer resolution and reveals alternative optimal process windows beyond traditional black-box optimization. Our Bayesian network approach links a key GaAs process variable (growth temperature) to material descriptors (bulk and interface properties, e.g., bulk lifetime, doping, and surface recombination) and device performance parameters (e.g., cell efficiency). For this purpose, we combine a Bayesian inference framework with a neural network surrogate device-physics model that is 100× faster than numerical solvers. With the trained surrogate model and only a small number of experimental samples, our approach reduces significantly the time-consuming intervention and characterization required by the experimentalist. As a demonstration of our method, in only five metal organic chemical vapor depositions, we identify a superior growth temperature profile for the window, bulk, and back surface field layer of a GaAs solar cell, without any secondary measurements, and demonstrate a 6.5% relative AM1.5G efficiency improvement above traditional grid search methods.

14 SOLAR ENERGY↗
DOE OSTI2020

Evaluation of a pair-based, joint-likelihood association approach for regional infrasound event identification

SUMMARY A Bayesian framework for the association of infrasonic detections is presented and evaluated for analysis at regional propagation scales. A pair-based, joint-likelihood association approach is developed that identifies events by computing the probability that individual detection pairs are attributable to a hypothetical common source and applying hierarchical clustering to identify events from the pair-based analysis. The framework is based on a Bayesian formulation introduced for infrasonic source localization and utilizes the propagation models developed for that application with modifications to improve the numerical efficiency of the analysis. Clustering analysis is completed using hierarchical analysis via weighted linkage for a non-Euclidean distance matrix defined by the negative log-joint-likelihood values. The method is evaluated using regional synthetic data with propagation distances of hundreds of kilometres in order to study the sensitivity of the method to uncertainties and errors in backazimuth and time of arrival. The method is found to be robust and stable for typical uncertainties, able to effectively distinguish noise detections within the data set from those in events, and can be made numerically efficient due to its ease of parallelization.

58 GEOSCIENCES↗
DOE OSTI2025

Predicting fusion ignition at the National Ignition Facility with physics-informed deep learning

Here, an inertial confinement fusion experiment, carried out at the National Ignition Facility, has achieved ignition by generating fusion energy exceeding the laser energy that drove the experiment. Prior to the experiment, a generative machine learning model that combines radiation hydrodynamics simulations, deep learning, experimental data, and Bayesian statistics was used to predict, with a probability greater than 70%, that ignition was the most likely outcome for this shot.

Spears, Brian K. [Lawrence Livermore National Labo↗
DOE OSTI2026

BASS

SAND2026-17001O BASS implements Bayesian Adaptive Spline Surfaces in MATLAB and serves as a surrogate model for regression applications. Sandia National Laboratories is a multimission laboratory managed and operated by National Technology & Engineering Solutions of Sandia, LLC, a wholly owned subsidiary of Honeywell International Inc., for the U.S. Department of Energy's National Nuclear Security Administration under contract DE-NA0003525.

Tucker, J. Derek [Sandia National Lab. (SNL-CA), L↗