Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Bayesian modeling”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,171 records · Page 65
All sourcesNASA NTRSDOE OSTI
DOE OSTI2019

A copula-based Bayesian method for probabilistic solar power forecasting

With increased penetration of solar energy sources, solar power forecasting has become more crucial and challenging. This paper proposes a copula-based Bayesian approach to improve probabilistic solar power forecasting by capturing the joint distribution between solar power and ambient temperature. A prior forecast distribution is first obtained using different underlying point forecasting models. Parametric and empirical copulas of solar power and temperature are then developed to update the prior distribution to the posterior forecast distribution. A public solar power database is used to demonstrate effectiveness of the proposed method. Numerical results show that the copula-based Bayesian method outperforms the forecasting method that directly uses temperature as a feature. The Bayesian method is also compared with persistent models and show improved performance. Furthermore, this article includes supplementary material (data and code) for reproducibility.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2021

Improved naive Bayesian probability classifier in predictions of nuclear mass

Recently, novel statistical methods such as neural networks and Bayesian learning methods are implemented to describe the nuclear masses. Based on previous studies, an improved naive Bayesian probability (iNBP) classifier is proposed to study the nuclear masses by refining the results of sophisticated nuclear models. In the iNBP method, the prediction for nuclear masses is treated as a classification problem. The residuals are classified into several groups to generate prior and conditional probabilities, and the posterior probabilities are further determined by the Bayesian formula. We choose the expectation with maximum probability as the final prediction. Reliability of the iNBP method is assessed by analyzing the global optimizations and the extrapolating capabilities. Here, the iNBP method exhibits impressive improvements on global descriptions for different mass models. Moreover, the method shows robust extrapolating capabilities. Results demonstrate the iNBP method can be applied to predict the nuclear masses of unknown regions. Considering the local mass relations, the iNBP method can offer considerable fine-tuning of the mass descriptions from nuclear models. The methodology proposed in this paper can also be applied to other model-based extrapolations of nuclear observables.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗
DOE OSTI2023

Quantifying uncertainty for deep learning based forecasting and flow-reconstruction using neural architecture search ensembles

Classical problems in computational physics such as data-driven forecasting and signal reconstruction from sparse sensors have recently seen an explosion in deep neural network (DNN) based algorithmic approaches. However, most DNN models do not provide uncertainty estimates, which are crucial for establishing the trustworthiness of these techniques in downstream decision making tasks and scenarios. In recent years, ensemble-based methods have achieved significant success for the uncertainty quantification in DNNs on a number of benchmark problems. However, their performance on real-world applications remains under-explored. In this work, we present an automated approach to DNN discovery and demonstrate how this may also be utilized for ensemble-based uncertainty quantification. Specifically, we propose the use of a scalable neural and hyperparameter architecture search for discovering an ensemble of DNN models for complex dynamical systems. We highlight how the proposed method not only discovers high-performing neural network ensembles for our tasks, but also quantifies uncertainty seamlessly. This is achieved by using genetic algorithms and Bayesian optimization for sampling the search space of neural network architectures and hyperparameters. Subsequently, a model selection approach is used to identify candidate models for an ensemble set construction. Afterwards, a variance decomposition approach is used to estimate the uncertainty of the predictions from the ensemble. We demonstrate the feasibility of this framework for two tasks — forecasting from historical data and flow reconstruction from sparse sensors for the sea-surface temperature. In conclusion, we demonstrate superior performance from the ensemble in contrast with individual high-performing models and other benchmarks.

Deep ensembles↗
NASA NTRS2014

An Uncertainty Quantification Framework for Prognostics and Condition-Based Monitoring

This paper presents a computational framework for uncertainty quantification in prognostics in the context of condition-based monitoring of aerospace systems. The different sources of uncertainty and the various uncertainty quantification activities in condition-based prognostics are outlined in detail, and it is demonstrated that the Bayesian subjective approach is suitable for interpreting uncertainty in online monitoring. A state-space model-based framework for prognostics, that can rigorously account for the various sources of uncertainty, is presented. Prognostics consists of two important steps. First, the state of the system is estimated using Bayesian tracking, and then, the future states of the system are predicted until failure, thereby computing the remaining useful life of the system. The proposed framework is illustrated using the power system of a planetary rover test-bed, which is being developed and studied at NASA Ames Research Center.

Health Monitoring↗
DOE OSTI2024

High-Dimensional Bayesian Optimization via Semi-Supervised Learning with Optimized Unlabeled Data Sampling

We introduce a novel semi-supervised learning approach, named Teacher-Student Bayesian Optimization (TSBO ), integrating the teacher-student paradigm into BO to minimize expensive labeled data queries for the first time. TSBO incorporates a teacher model, an unlabeled data sampler, and a student model. The student is trained on unlabeled data locations generated by the sampler, with pseudo labels predicted by the teacher. The interplay between these three components implements a unique selective regularization to the teacher in the form of student feedback. This scheme enables the teacher to predict high-quality pseudo labels, enhancing the generalization of the GP surrogate model in the search space. To fully exploit TSBO , we propose two optimized unlabeled data samplers to construct effective student feedback that well aligns with the objective of Bayesian optimization. Furthermore, we quantify and leverage the uncertainty of the teacher-student model for the provision of reliable feedback to the teacher in the presence of risky pseudo-label predictions. TSBO demonstrates significantly improved sample-efficiency in several global optimization tasks under tight labeled data budgets. The implementation is available at https://github.com/reminiscenty/TSBO-Official.

Yin, Yuxuan↗
DOE OSTI2022

General-Purpose Bayesian Tensor Learning With Automatic Rank Determination and Uncertainty Quantification

A major challenge in many machine learning tasks is that the model expressive power depends on model size. Low-rank tensor methods are an efficient tool for handling the curse of dimensionality in many large-scale machine learning models. The major challenges in training a tensor learning model include how to process the high-volume data, how to determine the tensor rank automatically, and how to estimate the uncertainty of the results. While existing tensor learning focuses on a specific task, this paper proposes a generic Bayesian framework that can be employed to solve a broad class of tensor learning problems such as tensor completion, tensor regression, and tensorized neural networks. We develop a low-rank tensor prior for automatic rank determination in nonlinear problems. Our method is implemented with both stochastic gradient Hamiltonian Monte Carlo (SGHMC) and Stein Variational Gradient Descent (SVGD). We compare the automatic rank determination and uncertainty quantification of these two solvers. We demonstrate that our proposed method can determine the tensor rank automatically and can quantify the uncertainty of the obtained results. We validate our framework on tensor completion tasks and tensorized neural network training tasks.

Bayesian inference↗
DOE OSTI2023

Prehydrodynamic evolution in large and small systems

We extend our previous investigation of the effects of prehydrodynamic evolution on final-state observables in heavy-ion collisions [38] to smaller systems. We use a state-of-the-art hybrid model for the numerical simulations with optimal parameters obtained from a previous Bayesian study. By studying p–Pb collisions, we find that the effects due to the assumption of a conformal evolution in the prehydrodynamical stage are even more important in small systems. We also show that this effect depends on the time duration of the pre-equilibrium stage, which is further enhanced in small systems. Finally, we show that the recent proposal of a free-streaming with subluminal velocity for the pre-equilibrium stage, thus effectively breaking conformal invariance, can alleviate the contamination of final-state observables. Furthermore, our study further reinforces the need for moving beyond conformal approaches in pre-equilibrium dynamics modeling, especially when extracting transport coefficients from hybrid models in the high-precision era of heavy-ion collisions.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗
DOE OSTI2023

Adaptive Discovery and Mixed-Variable Optimization of Next Generation Synthesizable Microelectronic Materials

Design of new microelectronic materials is characterized by several challenges such as high-dimensionality of the atomic structure-composition variable space, formidable cost of directly using high-fidelity simulations for design optimization, dispersity in literature-reported similar materials and synthesis methods, complex physical mechanisms, and mixed qualitative and quantitative design variables that lead to a disjointed design space. Even though machine learning (ML) techniques have been employed to expedite materials innovation, existing methods treat ML and design optimization as two separate processes, failing to resolve the fundamental challenges associated with high dimensionality and mixed-variable complexity. We have developed a ML enhanced mixed-variable material design optimization framework to efficiently extract useful information from existing data in literature and physics-based simulations to guide the autonomous search for optimal materials. Our proposed framework is composed of four computational modules: (1) a natural language processing (NLP) based virtual screening module, (2) classification based concept exploration module, (3) a density functional theory (DFT)-based high-fidelity evaluation model, and (4) a novel latent-variable Gaussian process (LVGP) ML model for mixed-variable problems with uncertainty quantification, which seamlessly integrates with Bayesian Optimization (BO) and achieves superb efficiency through embedded physics-based dimension reduction. Our approach is demonstrated and validated using the testbed of functional materials exhibiting metal-insulation transitions (MITs), with the targeted reversible resistivity changes (∼10^5) near room temperature. At the end of the 30-month project, we have developed a series of new ML techniques using NLP, conditional variational autoencoders, active learning, latent-variable Gaussian processes, integrated with Bayesian optimization. Our project has resulted in new predicted MITs compounds and improved understanding of MITs microscopic mechanisms, which in turn will revolutionize microelectronics science to provide energy-saving solutions. Our research has improved both creativity and efficiency in transforming rare-event discoveries of new functional materials to persistent innovations. In addition to open-sourcing the online MIT database and the classification model, the LVGP open source code has been downloaded more than 15,000 times within two years. More than 40 MIT compounds have been identified and many have been pursued experimentally via collaborators. The research results are published in close to 20 collaborative papers in high-impact journals, such as Chem. Mater., Appl. Phys. Rev., Sci. Rep., among others of design space.

36 MATERIALS SCIENCE↗
DOE OSTI2021

Validation Metrics for Fixed Effects and Mixed-Effects Calibration

The modern scientific process often involves the development of a predictive computational model. To improve its accuracy, a computational model can be calibrated to a set of experimental data. A variety of validation metrics can be used to quantify this process. Some of these metrics have direct physical interpretations and a history of use, while others, especially those for probabilistic data, are more difficult to interpret. In this work, a variety of validation metrics are used to quantify the accuracy of different calibration methods. Frequentist and Bayesian perspectives are used with both fixed effects and mixed-effects statistical models. Through a quantitative comparison of the resulting distributions, the most accurate calibration method can be selected. Two examples are included which compare the results of various validation metrics for different calibration methods. It is quantitatively shown that, in the presence of significant laboratory biases, a fixed effects calibration is significantly less accurate than a mixed-effects calibration. This is because the mixed-effects statistical model better characterizes the underlying parameter distributions than the fixed effects model. The results suggest that validation metrics can be used to select the most accurate calibration model for a particular empirical model with corresponding experimental data.

97 MATHEMATICS AND COMPUTING↗
NASA NTRS2022

Gaussian Process Regression Method for Costing SmallSat Bus Capabilities

NASA is responding to the growing interest in, andcapabilities of, small satellites for science applications with an increasingnumber and frequency of Announcements of Opportunityfor small satellite space missions. Estimating the probabilitythat these mission concepts will fit within the small cost capsof these opportunities is largely driven by the probability thatone of the burgeoning number of small satellite providers will beable to meet the payload’s accommodation requirements withinthe budget for the spacecraft. JPL has collected a databasecontaining technical specifications and cost of commerciallyavailable Smallsat buses across various vendors. The primarypurpose of the database is for use in JPL’s Team X architecturestudies to inform cost estimates of a spacecraft bus which fitsthe customer’s technical requirements for their payload andmission. Customer needs are often unique and don’t alignperfectly with an off-the-shelf commercial spacecraft bus, whichmotivates the need to develop a cost model across the continuoustechnical parameter space.Al’s Bus Cost Distribution Estimator (ABCDE) uses Gaussianprocess regression (GPR) to predict commercial Smallsat spacecraftbus cost based on a subset of a customer’s technicalrequirements (payload mass, payload power, delta V, pointingcontrol, and downlink rate). GPR is implemented in ABCDE asa Bayesian method which fits an implied multivariate regressionon the technical parameters and uses kriging to intentionally“overfit” the residuals. Overfitting the residuals allows costestimates to collapse in uncertainty closer to the data pointswhile maintaining larger uncertainty intervals in regions of parameterspace with fewer data records. The data used to fit thismodel is sensitive and represents cost estimates for off-the-shelfcommercial buses. GPR simultaneously protects the sensitivityof the database and uses the sparse nature of the database toaccount for uncertainty in cost in a useful way. For a givenset of customer technical requirements, the tool provides a costestimate distribution, the percentiles of which can be interpretedas a confidence level of finding a commercial bus under a specifiedcost cap. ABCDE dramatically pushes the boundaries ofspacecraft cost estimation models due to its Bayesian methodology(accounting for the maximum uncertainty in the underlyingregression), the mathematically advanced kriging methodology,and the novelty of its application in Team X architecture tradestudies.

Austin, Alex↗
DOE OSTI2024

Assessing and benchmarking the fidelity of posterior inference methods for astrophysics data analysis

In this era of large and complex astronomical survey data, interpreting, validating, and comparing inference techniques becomes increasingly difficult. This is particularly critical for emerging inference methods like Simulation-Based Inference (SBI), which offer significant speedup potential and posterior modeling flexibility, especially when deep learning is incorporated. We present a study to assess and compare the performance and uncertainty prediction capability of Bayesian inference algorithms – from traditional MCMC sampling of analytic functions to deep learning-enabled SBI. We focus on testing the capacity of hierarchical inference modeling in those scenarios. Before we extend this study to cosmology, we first use astrophysical simulation data to ensure interpretability. We demonstrate a probabilistic programming implementation of hierarchical and non-hierarchical Bayesian inference using simulations derived from the DeepBench software library, a benchmarking tool developed by our group that generates simple and controllable astrophysical objects from first principles. This study will enable astronomers and physicists to harness the inference potential of these methods with confidence.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗
DOE OSTI2025

Knowledge graph-aided Bayesian active learning for top- K genetic interaction discovery

In silico methods for predicting the effects of multi-gene perturbations hold great promise for advancing functional genomics, computational drug discovery, and disease modeling. However, the development of these predictive algorithms for mammalian systems has been hampered by limited datasets and high experimental costs. In this study, we present a Bayesian active learning framework designed to discover pairwise host gene knockdowns that effectively inhibit viral proliferation in an in vitro HIV-1 infection model. Our method leverages a biological knowledge graph as side information and employs a computationally efficient batch diversification approach. We evaluated this framework using a dataset of viral load measurements obtained from multi-day dual-gene depletion experiments, encompassing all possible pairwise knockdowns of over 350 host genes associated with HIV infection. We demonstrate that our framework rapidly identifies the most effective gene knockdown pairs for reducing viral load. Furthermore, we show that incorporating side information enhances performance during the early stages of active learning (low data regime), while our batch diversification strategy significantly boosts performance in later stages (high data regime). This framework is general and can be adapted to explore gene interactions in other contexts, such as synthetic lethality prediction and mapping epistatic effects across quantitative trait loci.

Computational biology and bioinformatics↗
NASA NTRS1996

Land Covering Classifications of Boreas Modeling Grid Using AIRSAR Images

Mapping forest types in the boreal ecosystem in an integrated part of any modeling excercise of biogeophysical processes characterizing the interaction of forest with the atmosphere. In this paper, we report the results of the land cover classification of the SAR data acquired during the BOREAS (BOReal Ecosystem Atmospheric Study) intensive field campaigns over the modeling sub-grid of the southern study area in Saskatchewan , Canada. A Bayesian-maximum-a-posteriori classifier has been applied on the NASA/JPL AIRSAR images covering the region during the peak of the growing season in July, 1994.

Atmospheric Study↗
DOE OSTI2025

Machine Learning-Guided Optimization of SABRE Hyperpolarization for α-Ketoglutarate in Acetone–Water

Signal amplification by reversible exchange (SABRE) is a hyperpolarization method that polarizes target nuclei of metabolites quickly and efficiently. Recent SABRE advances, including Ace-SABRE, yield biocompatible, aqueous solutions of hyperpolarized markers for metabolic monitoring. Building on recent advancements, expanding the substrate scope of Ace-SABRE is desirable. However, SABRE polarization is sensitive to many different parameters; therefore, traditional optimization approaches are experimentally time-consuming. In this proof-of-concept application of machine learning (ML), Bayesian optimization (BO) is used for four important input parameters to model the complex SABRE dynamics while saving experimental time. The presented ML model also provides chemical insights that enable predictions of sample compositions for increased polarization levels. In this paper, we transition from an original average free polarization of p = ∼0.90% to a maximum observed free polarization of p = ∼6.6% for 1- 13 C alpha-ketoglutarate (AKG) with 13 C at natural abundance, utilizing both direct outputs as well as chemical insights revealed by the ML model.

Catalysts↗
DOE OSTI2020

Embedding physics domain knowledge into a Bayesian network enables layer-by-layer process innovation for photovoltaics

Process optimization of photovoltaic devices is a time-intensive, trial-and-error endeavor, which lacks full transparency of the underlying physics and relies on user-imposed constraints that may or may not lead to a global optimum. Herein, we demonstrate that embedding physics domain knowledge into a Bayesian network enables an optimization approach for gallium arsenide (GaAs) solar cells that identifies the root cause(s) of underperformance with layer-by-layer resolution and reveals alternative optimal process windows beyond traditional black-box optimization. Our Bayesian network approach links a key GaAs process variable (growth temperature) to material descriptors (bulk and interface properties, e.g., bulk lifetime, doping, and surface recombination) and device performance parameters (e.g., cell efficiency). For this purpose, we combine a Bayesian inference framework with a neural network surrogate device-physics model that is 100× faster than numerical solvers. With the trained surrogate model and only a small number of experimental samples, our approach reduces significantly the time-consuming intervention and characterization required by the experimentalist. As a demonstration of our method, in only five metal organic chemical vapor depositions, we identify a superior growth temperature profile for the window, bulk, and back surface field layer of a GaAs solar cell, without any secondary measurements, and demonstrate a 6.5% relative AM1.5G efficiency improvement above traditional grid search methods.

14 SOLAR ENERGY↗
DOE OSTI2020

Evaluation of a pair-based, joint-likelihood association approach for regional infrasound event identification

SUMMARY A Bayesian framework for the association of infrasonic detections is presented and evaluated for analysis at regional propagation scales. A pair-based, joint-likelihood association approach is developed that identifies events by computing the probability that individual detection pairs are attributable to a hypothetical common source and applying hierarchical clustering to identify events from the pair-based analysis. The framework is based on a Bayesian formulation introduced for infrasonic source localization and utilizes the propagation models developed for that application with modifications to improve the numerical efficiency of the analysis. Clustering analysis is completed using hierarchical analysis via weighted linkage for a non-Euclidean distance matrix defined by the negative log-joint-likelihood values. The method is evaluated using regional synthetic data with propagation distances of hundreds of kilometres in order to study the sensitivity of the method to uncertainties and errors in backazimuth and time of arrival. The method is found to be robust and stable for typical uncertainties, able to effectively distinguish noise detections within the data set from those in events, and can be made numerically efficient due to its ease of parallelization.

58 GEOSCIENCES↗
DOE OSTI2025

Predicting fusion ignition at the National Ignition Facility with physics-informed deep learning

Here, an inertial confinement fusion experiment, carried out at the National Ignition Facility, has achieved ignition by generating fusion energy exceeding the laser energy that drove the experiment. Prior to the experiment, a generative machine learning model that combines radiation hydrodynamics simulations, deep learning, experimental data, and Bayesian statistics was used to predict, with a probability greater than 70%, that ignition was the most likely outcome for this shot.

Spears, Brian K. [Lawrence Livermore National Labo↗
DOE OSTI2026

BASS

SAND2026-17001O BASS implements Bayesian Adaptive Spline Surfaces in MATLAB and serves as a surrogate model for regression applications. Sandia National Laboratories is a multimission laboratory managed and operated by National Technology & Engineering Solutions of Sandia, LLC, a wholly owned subsidiary of Honeywell International Inc., for the U.S. Department of Energy's National Nuclear Security Administration under contract DE-NA0003525.

Tucker, J. Derek [Sandia National Lab. (SNL-CA), L↗