Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Bayesian framework”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 559 records · Page 31

Cosmological parameter estimation with a joint-likelihood analysis of the cosmic microwave background and big bang nucleosynthesis

Here, we present a joint-likelihood analysis of big bang nucleosynthesis (BBN) and cosmic microwave background (CMB) data, consistently combining likelihoods and taking into account uncertainties in nuclear reaction rates for the first time. Bayesian inference is performed on the baryon abundance and the effective number of neutrino species, 𝑁 eff , using a CMB Boltzmann solver in combination with LINX , a new flexible and efficient BBN code. We marginalize over Planck nuisance parameters and nuclear rates to find 𝑁 eff =3.0⁢8$^{+0.15}_{−0.14}$, 2.9⁢4$^{+0.16}_{−0.15}$, or 2.96$^{+0.13}_{−0.14}$, for three separate reaction networks. This framework enables robust testing of the lambda cold dark matter paradigm and its variants with CMB and BBN data.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗

Improving Subsurface Stress Characterization for Carbon Dioxide Storage Projects by Incorporating Machine Learning Techniques

The overall objective of this project is to develop a framework for reliable characterization and prediction of the state of stress in the overburden and underburden (including the basement) in CO 2 storage reservoirs using machine learning and integrated geomechanics and geophysical methods. Specifically, we propose to develop workflow encompassing of technologies and/or methods to predict stress and pressure changes due to CO 2 injection in an active tertiary recovery site and their impacts on subtle fault activation, fractures and occurrence of microseismic events and compare responses to field observations. In this project, we anticipate using dataset from the Farnsworth field Unit (FWU) which is operated by Purdure Petroleum. A novel elastic-waveform VSP inversion technique will be used to estimate high-resolution spatial and temporal changes of elastic moduli in CO 2 storage reservoirs, which will be combined with velocity-stress relationship derived from laboratory tests to obtain subsurface pressure and stress. Clustered microseismic data will be jointly inverted for improved focal mechanisms. Least-squares reverse-time migration of microseismic waveform data will be performed to directly image fracture/fault zones. Additionally, a deep neural network machine learning technique with convolutional and recurrent layers will be used for learning the spectro-temporal structures in microseismic waveforms. The results of this geotechnical data analysis will be integrated to develop a high-resolution 3D mechanical earth model extending from the overburden sealing formations to the underburden including the basement. Mechanical properties will be derived through integration of mechanical logs, tests, available results from chemo-mechanical laboratory tests, and elastic inversion of seismic data using a combination of Bayesian and stochastic methods as well as machine learning technique. Failure features (faults/fractures) will be represented and/or modeled based on seismic and core data analysis. A transient hydrodynamic-geomechanical model will be developed through coupling with the calibrated FWU reservoir simulation model. The full physics coupled model will be used to train a reduced order proxy model using machine learning algorithm for estimating stress which will then be used with appropriate constitutive relationships and forward seismological models to simulate pressure changes and induced microseismicity. An advanced optimization framework will be developed to perform a history match to minimize error between field observations and simulated. The history matched proxy model will be verified against the full-physics equivalent. The field observations that will be used in the coupled model calibration process include pressure/stress inverted from VSP, moment magnitude from microseismic analysis, real time downhole pressure measurements, production and injection data. Parameter sensitivity and uncertainty analysis will be performed to characterize the impact of model parameter uncertainty on stress estimates. The proposed project will have significant impact on future field implementation of the proposed technology. Because the project field site is an ongoing CO 2 EOR development, the value of the new technology will be demonstrated in an operational context and evaluated as a viable risk mitigation strategy. Cost/benefit will be evaluated together with the various commercial incentives for CO 2 sequestration available to oil and gas operators. The extensive available dataset and ongoing data acquisition under the SWP Phase III work plan provides flexibility for investigation of multiple approaches and reduces technical risk.

58 GEOSCIENCES↗

Crustal Scale Travel Time Prediction with the SALSA3D Framework and Machine Learning

The SALSA3D project aims to improve our models used in travel time prediction. The current version uses tomographic modeling for propagation through the Earth’s mantle because of the large number (order of millions) of observations of seismic phases which primarily traverse the Earth’s mantle and the ability to pose the travel time problem as a set of linear equations. However, all seismic rays traverse the crust to reach receivers at the surface and therefore models of propagation through the crust are required. Therefore, the primary motivation for this study is to explore how to increase the scope of the SALSA3D project to phases which travel primarily through the crust. In this report, we evaluate new, machine learning based and physics-based methods to model these travel times for integration into the SALSA3D framework. Our results suggest that using our existing physics-based travel time tomography method is a viable approach for the regional to global scale, but better predictive capabilities can be achieved through a neural network trained on the region of interest for near-regional offsets. We suggest future iterations of SALSA3D should incorporate machine learning tools such as Physics-Informed Neural Networks or Bayesian Neural Networks.

58 GEOSCIENCES↗

Solving Stochastic Inverse Problems for Property–Structure Linkages Using Data-Consistent Inversion and Machine Learning

Determining process–structure–property linkages is one of the key objectives in material science, and uncertainty quantification plays a critical role in understanding both process–structure and structure–property linkages. In this work, we seek to learn a distribution of microstructure parameters that are consistent in the sense that the forward propagation of this distribution through a crystal plasticity finite element model matches a target distribution on materials properties. This stochastic inversion formulation infers a distribution of acceptable/consistent microstructures, as opposed to a deterministic solution, which expands the range of feasible designs in a probabilistic manner. Furthermore, to solve this stochastic inverse problem, we employ a recently developed uncertainty quantification framework based on push-forward probability measures, which combines techniques from measure theory and Bayes’ rule to define a unique and numerically stable solution. This approach requires making an initial prediction using an initial guess for the distribution on model inputs and solving a stochastic forward problem. To reduce the computational burden in solving both stochastic forward and stochastic inverse problems, we combine this approach with a machine learning Bayesian regression model based on Gaussian processes and demonstrate the proposed methodology on two representative case studies in structure–property linkages.

36 MATERIALS SCIENCE↗

DeepONet-grid-UQ: A trustworthy deep operator framework for predicting the power grid’s post-fault trajectories

This paper proposes a novel data-driven method for the reliable prediction of the power grid’s post-fault trajectories, i.e., the power grid’s dynamic response after a disturbance or fault. Here, the proposed method is based on the recently proposed concept of Deep Operator Networks (DeepONets). Unlike traditional neural networks that learn to approximate functions, DeepONets are designed to approximate nonlinear operators, i.e., mappings between infinite-dimensional spaces. Under this operator framework, we design a novel and efficient DeepONet that (i) takes as inputs the trajectories collected before and during the fault and (ii) outputs the predicted post-fault trajectories. In addition, we endow our method with the much-needed ability to balance efficiency with reliable/trustworthy predictions via uncertainty quantification. To this end, we propose and compare two novel methods that enable quantifying the predictive uncertainty. First, we propose a Bayesian DeepONet (B-DeepONet) that uses stochastic gradient Hamiltonian Monte-Carlo to sample from the posterior distribution of the DeepONet trainable parameters. Then, we design a Probabilistic DeepONet (Prob-DeepONet) that uses a probabilistic training strategy to enable quantifying uncertainty at virtually no extra computational cost. Finally, we validate the proposed methods’ predictive power and uncertainty quantification capability using the New York-New England power grid model.

24 POWER TRANSMISSION AND DISTRIBUTION↗

Exploration with Scalable Gaussian Process Reinforcement Learning

Exploration is a challenging problem in reinforcement learning (RL), especially in environments with sparse rewards. Quantifying and utilizing the parametric uncertainty has been shown to be paramount for successful exploration [Osband et al., 2018]. Bayesian, or approximately Bayesian, methods present a principled means of estimating the parametric uncertainty in RL problems. Gaussian processes, nonparametric Bayesian models, are often impractical due to poor scalability and computational bottlenecks. We introduce a scalable Gaussian process RL (GPRL) method which directly induces sparsity in the covariance matrix to facilitate faster computation. This is a departure from previous GPRL methods which instead rely on data reduction and subsampling. We compare various covariance-based exploration techniques (Thompson sampling, upper confidence bound, and probabilistic maximum variance) which leverage our scalable GP framework in sparse reward environments. Finally, we show favorable comparison against the bootstrapped deep Q-Network.

97 MATHEMATICS AND COMPUTING↗

Earthquake Phase Association Using a Bayesian Gaussian Mixture Model

Earthquake phase association algorithms aggregate picked seismic phases from a network of seismometers into individual seismic events and play an important role in earthquake monitoring and research. Dense seismic networks and improved phase picking methods produce massive seismic phase datasets, particularly for earthquake swarms and aftershocks occurring closely in time and space, making phase association a challenging problem. Here, we present a new association method, the Gaussian Mixture Model Association (GaMMA), that combines the Gaussian mixture model with earthquake location, origin time, and magnitude estimation. We treat earthquake phase association as an unsupervised clustering problem in a probabilistic framework, where each earthquake corresponds to a cluster of P and S phases with a hyperbolic moveout of arrival times and a decay of amplitude with distance. We use the multivariate Gaussian distribution to model the collection of phase picks of an event; and the mean of the multivariate Gaussian distribution is given by the predicted arrival time and amplitude from the causative event. We carry out the pick assignment to each earthquake and determine earthquake source parameters (i.e., earthquake location, origin time, and magnitude) under the maximum likelihood criterion using the Expectation-Maximization algorithm. The GaMMA method does not require typical association steps of other algorithms, such as grid-search or supervised training. The results for both synthetic tests and for the 2019 Ridgecrest earthquake sequence show that GaMMA effectively associates phases from a temporally and spatially dense earthquake sequence while producing useful estimates of earthquake location and magnitude.

58 GEOSCIENCES↗

Jensen–Shannon divergence based novel loss functions for Bayesian neural networks

Bayesian neural networks (BNNs) are state-of-the-art machine learning methods that can naturally regularize and systematically quantify uncertainties using their stochastic parameters. Kullback–Leibler (KL) divergence-based variational inference used in BNNs suffer from unstable optimization and challenges in approximating light-tailed posteriors due to the unbounded nature of the KL divergence. To resolve these issues, we formulate a novel loss function for BNNs based on a new modification to the generalized Jensen–Shannon (JS) divergence, which is bounded. In addition, we propose a Geometric JS divergence-based loss, which is computationally efficient since it can be evaluated analytically. We found that the JS divergence-based variational inference is intractable, and hence employed a constrained optimization framework to formulate these losses. Our theoretical analysis and empirical experiments on multiple regression and classification data sets suggest that the proposed losses perform better than the KL divergence-based loss, especially when the data sets are noisy or biased. Specifically, there are approximately 5% and 8% improvements in accuracy for a noise-added CIFAR-10 dataset and a regression dataset, respectively. There is about 13% reduction in false negative predictions of a biased histopathology dataset. Additionally, we quantify and compare the uncertainty metrics for the regression and classification tasks.

97 MATHEMATICS AND COMPUTING↗

A novel machine learning based identification of potential adopter of rooftop solar photovoltaics

With the proliferation of rooftop solar photovoltaic installations, there is a need to proactively predict consumer potential for solar photovoltaic adoption, for improved electric utility planning and operation. Traditional analytical modeling approaches are limited to a few survey features and a larger part of the survey would remain untouched by the decision model. This article presents a novel, data-driven modeling approach that strategically prunes a large set of consumer profile features using a machine learning framework to train a model for predicting potential solar adoption. The approach utilizes the Gradient Boosting Decision Tree model through a Light Gradient Boosting framework that improves significantly over the poor prediction accuracy of the existing approaches. Model training using focal-loss based supervision is used to overcome the difficulty in identifying the potential adopters that is inherent in conventional data-driven models. In addition, to overcome possible data sparsity in a limited survey sample, a Generative Adversarial Network is presented to create synthetic user samples and its effectiveness on model performance is assessed. A Bayesian optimization approach is used to systematically arrive at the hyperparameters of the proposed model. Validation of the presented approach on a survey data collected by the National Rural Electric Cooperative Association in Virginia in 2018 demonstrates the excellent predictive capability of the machine learning based approach to modeling solar adoption reliably.

14 SOLAR ENERGY↗

Bayesian Gaussian process inference for neutron spin echo measurement

Neutron spin echo (NSE) spectroscopy provides unique access to microscopic dynamics, but its application is often constrained by low neutron flux, long acquisition times, and significant noise. Here, we present a Bayesian inference approach based on Gaussian process regression (GPR) to reconstruct high-quality spin echo signals from sparse and noisy data by exploiting correlations in reciprocal space. Benchmarks on synthetic datasets and validation with experimental NSE measurements of dendrimers show that GPR suppresses noise, interpolates missing intensity values, and accommodates irregular observations. The method improves accuracy, shortens acquisition times, and enables high-throughput and real-time studies. Beyond NSE, the framework is broadly applicable to other low signal-to-noise ratio scattering techniques, thereby extending the scope of neutron spectroscopy.

Tung, Chi-Huan [Oak Ridge National Laboratory (ORN↗

Reduced-dimension Bayesian optimization for model calibration of transient vapor compression cycles

Development and calibration of first-principles dynamic models of vapor compression cycles (VCCs) is of critical importance for applications that include control design and fault detection and diagnostics. Nevertheless, the inherent complexity of models that are represented by large systems of differential–algebraic equations leads to significant challenges for model calibration processes that utilize classical gradient-based methods. Bayesian optimization (BO) is a sample-efficient and gradient-free approach using a probabilistic surrogate model and optimal search over a feasible parameter space. Despite the benefits of BO in reducing computational costs, challenges remain in dealing with a high-dimensional calibration task resulting from a large set of parameters that have significant impacts on system behavior and need to be calibrated simultaneously. This paper presents a reduced-dimension BO framework for calibrating transient VCCs models where the calibration space is projected to a low-dimensional subspace for accelerating convergence of the solution algorithm and consequently reducing the number of transient simulations. The proposed approach was demonstrated via two case studies associated with different VCC applications where 10 parameters were calibrated in each case using laboratory measurements. The reduced-dimension BO framework only required 1 / 8 th of the iterations associated with a standard BO method that deals with high-dimensional calibration parameters for converged solutions and yielded comparable accuracy. Furthermore, both calibrated models revealed significant accuracy improvements compared to uncalibrated models.

Ma, Jiacheng↗

BMINN: Learning chemical potentials and parameters from voltage data for multi-phase battery modeling

Free-energy landscapes and chemical potentials govern the dynamics of phase transitions, transport, and stability in functional materials, yet they remain experimentally inaccessible under realistic operating conditions. Here we introduce a Bayesian model-integrated neural network (BMINN) that embeds physics-based formulations of non-autonomous partial differential-algebraic equations into probabilistic learning. This approach reconstructs hidden thermodynamics directly from macroscopic current-voltage data, providing quantitative access to metastable states, staging transitions, and energy barriers without synchrotron probes. Demonstrated on lithium-graphite electrodes, BMINN recovers full Gibbs free-energy landscapes with fidelity validated against operando X-ray diffraction. The framework generalizes across dynamical regimes, enabling accurate voltage prediction, internal state estimation, and inference of governing parameters. Beyond batteries, BMINN exemplifies a broadly applicable strategy for learning missing physics in multiphase, non-equilibrium systems, offering a new pathway to uncover hidden thermodynamic functions across condensed matter and materials physics.

25 ENERGY STORAGE↗

Accelerating Hamiltonian Monte Carlo for Bayesian inference in neural networks and neural operators

Hamiltonian Monte Carlo (HMC) is a powerful and accurate method to sample from the posterior distribution in Bayesian inference. However, HMC techniques are computationally demanding for Bayesian neural networks due to the high dimensionality of the network’s parameter space and the non-convexity of their posterior distributions. Therefore, various approximation techniques, such as variational inference (VI) or stochastic gradient MCMC, are often employed to infer the posterior distribution of the network parameters. Such approximations introduce inaccuracies in the inferred distributions, resulting in unreliable uncertainty estimates. In this work, we propose a hybrid approach that combines inexpensive VI and accurate HMC methods to efficiently and accurately quantify uncertainties in neural networks and neural operators. The proposed approach leverages an initial VI training on the full network. We examine the influence of individual parameters on the prediction uncertainty, which shows that a large proportion of the parameters do not contribute substantially to uncertainty in the network predictions. This information is then used to significantly reduce the dimension of the parameter space, and HMC is performed only for the subset of network parameters that strongly influence prediction uncertainties. This yields a framework for accelerating the full batch HMC for posterior inference in neural networks. We demonstrate the efficiency and accuracy of the proposed framework on deep neural networks and operator networks, showing that inference can be performed for large networks with tens to hundreds of thousands of parameters. Finally, we show that this method can effectively learn surrogates for complex physical systems by modeling the operator that maps from upstream conditions to wall-pressure data on a cone in hypersonic flow.

Bayesian inference↗

Ripening of Rh Nanoparticle Catalysts in Reverse Water–Gas Shift via a Data-Driven Model Combining Physics, Theory, and Experiment

Degradation via sintering is an ongoing challenge that impedes the broad commercial success of supported metallic nanoparticle catalysts. To mitigate degradation via informed catalyst design and process operations, here we aim to disambiguate the underlying mechanisms of sintering by combining theory and experiment in a quantitative framework. While mechanistic sintering models exist, they only model a single sintering pathway, even though multiple sintering mechanisms can occur simultaneously or dominate at different stages of the process. Data-driven machine learning models have emerged as a means to represent complex processes through data regression. However, machine learning models have very large data needs and lack mechanistic insights due to their black-box encoding. To develop an interpretive model of catalyst degradation via sintering, we constructed a hybrid model combining mechanistic “physics-based” models and data-driven methods to obtain both reliable predictions and mechanistic insights regarding experimentally observed sintering phenomena. Focusing on nanoparticle sintering in the Rh–TiO 2 catalyst for the reverse water–gas shift (RWGS) reaction, the hybrid model couples a mechanistic term for Ostwald ripening with energy values calculated via density functional theory (DFT) with a parametric, data-driven discrepancy function term for unmodeled mechanisms. The hybrid model is trained using Bayesian inference with data collected from small-angle X-ray scattering (SAXS) in situ experiments wherein average nanoparticle diameter versus time was measured at three relevant operating temperatures. The calibrated hybrid model results show that an Ostwald ripening-only model parameterized with fixed DFT energies does not fully capture the time and temperature dependence of the SAXS-observed sintering kinetics, and that an additional functional contribution, or DFT energy calibration, is required to reconcile simulation and experiment. Analysis of the hybrid-model error confirms that the hybrid model outperforms both the purely mechanistic and purely data-driven alternatives in terms of expected predictive accuracy for time-evolving average particle sizes. Furthermore, the results support the hypothesis that the Ostwald ripening mechanism is less important for explaining the sintering phenomena as operating temperature increases under an assumed fixed DFT parameterization. This could be explained in one of two ways: either latent, unmodeled sintering mechanisms dominate at higher temperatures, or the DFT uncertainty increases with temperature. The proposed modeling approach directly links theory to experiments and simulations via a statistical hybrid modeling framework and can be extended to other catalytic systems to improve predictive models and mechanistic understanding.

Bayesian hybrid modeling↗

Accelerating astronomical and cosmological inference with preconditioned Monte Carlo

ABSTRACT We introduce preconditioned Monte Carlo (PMC), a novel Monte Carlo method for Bayesian inference that facilitates efficient sampling of probability distributions with non-trivial geometry. PMC utilizes a Normalizing Flow (NF) in order to decorrelate the parameters of the distribution and then proceeds by sampling from the preconditioned target distribution using an adaptive Sequential Monte Carlo (SMC) scheme. The results produced by PMC include samples from the posterior distribution and an estimate of the model evidence that can be used for parameter inference and model comparison, respectively. The aforementioned framework has been thoroughly tested in a variety of challenging target distributions achieving state-of-the-art sampling performance. In the cases of primordial feature analysis and gravitational wave inference, PMC is approximately 50 and 25 times faster, respectively, than nested sampling (NS). We found that in higher dimensional applications, the acceleration is even greater. Finally, PMC is directly parallelisable, manifesting linear scaling up to thousands of CPUs.

79 ASTRONOMY AND ASTROPHYSICS↗

Sensitivity Study of Multiscale and Phenomenological Elasto-Viscoplastic Grade 91 Material Models for Component-Scale Response

Many advanced nuclear reactor concepts currently being developed are targeting higher operating temperatures relative to the current fleet of light water nuclear reactors, for efficiency gains and other operational considerations. The design of high temperature structural components with reliable long-term operational performance will depend on material models that accurately capture the inelastic deformation mechanisms active in these environments. In this work, we perform a detailed parameter sensitivity analysis of two unified elasto-viscoplastic Grade 91 material models capable of capturing long term high temperature creep deformation. The first model is a phenomelogical material model from the Nuclear Engineering Material Library (NEML) developed at Argonne National Lab. The NEML model parameters and their uncertainty were fit to a range of Grade 91 experimental data using Bayesian Markov Chain Monte Carlo analysis. The second model is a LAROMance data-driven surrogate material model developed at Los Alamos National Lab. The LAROMance model is fit to a large database of responses produced by a mechanistic crystal plasticity based polycrystal model. Parameters for the LAROMance surrogate material model reflect the pedigree of the Grade 91 microstructure. Both material models have been integrated into the Grizzly code, based on the open-source MOOSE multiphysics simulation framework, to simulate both the progression of aging mechanisms and the effects of that aging on nuclear power plant structures. Grizzly is used analyze a three-dimensional Grade 91 piping system to compare the long-term inelastic response predicted by these two fundamentally different models and assess the sensitivity of the material model input parameters on this quantity of interest.

42 ENGINEERING↗

Uncertainty quantification and reliability assessment for intermodal freight transportation

Intermodal freight optimization models support cost-effective, low-emission, and timely goods movement by coordinating trucks, rail, and barges. These models determine optimal flows, routing, and modal switches while respecting infrastructure and operational constraints. However, their real-world utility is often undermined by pervasive uncertainties-such as fluctuating transportation costs and emissions, variable terminal capacities, and uncertain freight demand-that distort key performance outcomes, including total system cost, carbon footprint, and transit time reliability. This study presents a structured framework for quantifying uncertainty in intermodal freight transportation (IFT) optimization. The framework evaluates how input uncertainty affects system performance and reliability, a critical need for ensuring that model-based decisions remain robust under real-world variability, especially amid volatile fuel prices, shifting demand, and growing disruptions. It integrates three complementary methods: (1) Sobol-based global sensitivity analysis to identify influential parameters affecting cost, emissions, and transit time, (2) Monte Carlo-based capacity perturbation analysis to assess robustness under probabilistic facility disruptions, and (3) Monte Carlo filtering with Bayesian inference to detect threshold-based performance vulnerabilities. The results highlight diesel truck unit cost as the dominant driver of variability. To improve system resilience, planners should prioritize uncertainty in fuel-related parameters when designing intermodal strategies.

Intermodal freight transportation↗

An initial framework for the rapid qualification of long-term creep rupture strength via microstructural modeling

This report describes the development and testing of a new method for extrapolating short-term creep rupture test data to predict long-term rupture strength. The goal of this work is to reduce the time required to qualify new materials for nuclear service by reducing the lead time required for dedicated, long-term material testing to establish key long-term material properties. The new approach described here uses a physics-based model to predict the long-term creep rupture strength of 316H stainless steel using only short-term test data. The key idea is to use Bayesian inference to find the statistical distribution of the model parameters that best explain the short-term rupture data. Because the model is physics-based these parameters are all microstructural quantities that can be measured through detailed material characterization experiments. The Bayesian prior distributions provide a means for incorporating this characterization data into the final model to improve the accuracy of the long-term model predictions. However, where such data is not available the process still produces an accurate model based on an uniformed prior. Our hypothesis is that this approach more accurately extrapolates the short-term test data when compared to current, empirical methods. The report proves this hypothesis using actual long-term rupture data available for 316H, including tests with rupture times greater than 200,000 hours. The general approach developed here could be applied to other materials and other time-dependent material properties. Applying this new technique to develop long-term qualified material properties, potentially in conjunction with other accelerated qualification approaches like staggered qualification test programs, could greatly reduce the time required to qualify new materials for nuclear service.

11 NUCLEAR FUEL CYCLE AND FUEL MATERIALS↗