Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Inference”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,747 records · Page 97

B-PINNs: Bayesian Physics-informal Neural Networks for Forward and Inverse PDE Problems with Noisy Data

We propose a Bayesian physics-informed neural network (B-PINN) to solve both forward and inverse nonlinear problems described by partial differential equations (PDEs) and noisy data. In this Bayesian framework, the Bayesian neural network (BNN) combined with a PINN for PDEs serves as the prior while the Hamiltonian Monte Carlo (HMC) or the variational inference (VI) could serve as an estimator of the posterior. B-PINNs make use of both physical laws and scattered noisy measurements to provide predictions and quantify the aleatoric uncertainty arising from the noisy data in the Bayesian framework. Compared with PINNs, in addition to uncertainty quantification, B-PINNs obtain more accurate predictions in scenarios with large noise due to their capability of avoiding overfitting. We conduct a systematic comparison between the two different approaches for the B-PINNs posterior estimation (i.e., HMC or VI), along with dropout used for quantifying uncertainty in deep neural networks. Our experiments show that HMC is more suitable than VI with mean field Gaussian approximation for the B-PINNs posterior estimation, while dropout employed in PINNs can hardly provide accurate predictions with reasonable uncertainty. Finally, we replace the BNN in the prior with a truncated Karhunen-Loève (KL) expansion combined with HMC or a deep normalizing flow (DNF) model as posterior estimators. The KL is as accurate as BNN and much faster but this framework cannot be easily extended to high-dimensional problems unlike the BNN based framework.

Non linear PDEs, Noisy data, Bysian physics inform↗

Analyzing count data with measurement error

In this article, we analyze observed count data such as the number of defects in a steel product where the observed counts are the true counts measured with errors. We account for the measurement error by using a measurement error model based on a latent lognormal (LLN) distribution. We consider making inference about a single population (e.g., from samples of a production lot) and a regression model (e.g., from runs of a designed experiment), where the measurement system properties are known, that is, the parameters of the LLN distribution are known. Then, we consider simultaneous inference for the single population and regression model as well as the measurement system. We demonstrate the proposed methodology with both simulated and real observed counts.

42 ENGINEERING↗

Conditional deep surrogate models for stochastic, high-dimensional, and multi-fidelity systems

Here, we present a probabilistic deep learning methodology that enables the construction of predictive data-driven surrogates for stochastic systems. Leveraging recent advances in variational inference with implicit distributions, we put forth a statistical inference framework that enables the end-to-end training of surrogate models on paired input–output observations that may be stochastic in nature, originate from different information sources of variable fidelity, or be corrupted by complex noise processes. The resulting surrogates can accommodate high-dimensional inputs and outputs and are able to return predictions with quantified uncertainty. Furthermore, the effectiveness our approach is demonstrated through a series of canonical studies, including the regression of noisy data, multi-fidelity modeling of stochastic processes, and uncertainty propagation in high-dimensional dynamical systems.

97 MATHEMATICS AND COMPUTING↗

Analytical methods for superresolution dislocation identification in dark-field X-ray microscopy

In this work, we develop several inference methods to estimate the position of dislocations from images generated using dark-field X-ray microscopy (DFXM)—achieving superresolution accuracy and principled uncertainty quantification. Using the framework of Bayesian inference, we incorporate models of the DFXM contrast mechanism and detector measurement noise, along with initial position estimates, into a statistical model coupling DFXM images with the dislocation position of interest. We motivate several position estimation and uncertainty quantification algorithms based on this model. We then demonstrate the accuracy of our primary estimation algorithm on synthetic realistic DFXM images of edge dislocations in single-crystal aluminum. We conclude with a discussion of our methods’ impact on future dislocation studies and possible future research avenues.

36 MATERIALS SCIENCE↗

Bayesian learning of orthogonal embeddings for multi-fidelity Gaussian Processes

Uncertainty propagation in complex engineering systems often poses significant computational challenges related to modeling and quantifying probability distributions of model outputs, as those emerge as the result of various sources of uncertainty that are inherent in the system under investigation. Gaussian Processes regression (GPs) is a robust meta-modeling technique that allows for fast model prediction and exploration of response surfaces. Multi-fidelity variations of GPs further leverage information from cheap and low fidelity model simulations in order to improve their predictive performance on the high fidelity model. In order to cope with the high volume of data required to train GPs in high dimensional design spaces, a common practice is to introduce latent design variables that are typically projections of the original input space to a lower dimensional subspace, and therefore substitute the problem of learning the initial high dimensional mapping, with that of training a GP on a low dimensional space. Here in this paper, we present a Bayesian approach to identify optimal transformations that map the input points to low dimensional latent variables. The \projection" mapping consists of an orthonormal matrix that is considered a priori unknown and needs to be inferred jointly with the GP parameters, conditioned on the available training data. The proposed Bayesian inference scheme relies on a two-step iterative algorithm that samples from the marginal posteriors of the GP parameters and the projection matrix respectively, both using Markov Chain Monte Carlo (MCMC) sampling. In order to take into account the orthogonality constraints imposed on the orthonormal projection matrix, a Geodesic Monte Carlo sampling algorithm is employed, that is suitable for exploiting probability measures on manifolds. We extend the proposed framework to multi-fidelity models using GPs including the scenarios of training multiple outputs together. We validate our framework on three synthetic problems with a known lower-dimensional subspace. The benefits of our proposed framework, are illustrated on the computationally challenging aerodynamic optimization of a last-stage blade for an industrial gas turbine, where we study the effect of an 85-dimensional shape parameterization of a three-dimensional airfoil on two output quantities of interest, specifically on the aerodynamic efficiency and the degree of reaction

42 ENGINEERING↗

Reliable extrapolation of deep neural operators informed by physics or sparse observations

Deep neural operators can learn nonlinear mappings between infinite-dimensional function spaces via deep neural networks. As promising surrogate solvers of partial differential equations (PDEs) for real-time prediction, deep neural operators such as deep operator networks (DeepONets) provide a new simulation paradigm in science and engineering. Pure data-driven neural operators and deep learning models, in general, are usually limited to interpolation scenarios, where new predictions utilize inputs within the support of the training set. However, in the inference stage of real-world applications, the input may lie outside the support, i.e., extrapolation is required, which may result to large errors and unavoidable failure of deep learning models. Here, we address this challenge of extrapolation for deep neural operators. First, we systematically investigate the extrapolation behavior of DeepONets by quantifying the extrapolation complexity, via the 2-Wasserstein distance between two function spaces and propose a new strategy of bias–variance trade-off for extrapolation with respect to model capacity. Subsequently, we develop a complete workflow, including extrapolation determination, and we propose five reliable learning methods that guarantee a safe prediction under extrapolation by requiring additional information—the governing PDEs of the system or sparse new observations. The proposed methods are based on either fine-tuning a pre-trained DeepONet or multifidelity learning. We demonstrate the effectiveness of the proposed framework for various types of parametric PDEs. Furthermore, our systematic comparisons provide practical guidelines for selecting a proper extrapolation method depending on the available information, desired accuracy, and required inference speed.

42 ENGINEERING↗

Protorheology in practice: Avoiding misinterpretation

Protorheology is the paradigm that any observed flow or deformation is a chance to infer quantitative rheological properties. While this creates many opportunities for insight, there is significant risk of misunderstanding the physics involved, e.g. misinterpreting a liquid as a solid or mistaking viscous flow time as viscoelastic relaxation time. We describe these and other potential mistakes, use case studies to show how serious the problems can be, and contrast misinterpretations with correct approaches and interpretations. Some issues are especially important with materials involving colloidal particles and flows involving surface tension. Whether the reader is making inference from a tilted vial, time-lapse gravity-driven flow, a bounce test, die swell, or any other protorheology observation, the examples here serve as a guide for avoiding bad data in protorheology.

42 ENGINEERING↗

Unlocking hidden information in sparse small-angle neutron scattering measurements

Hypothesis Small-Angle Neutron Scattering (SANS) is a powerful technique for studying soft matter systems such as colloids, polymers, and lyotropic phases, providing nanoscale structural insights. However, its effectiveness is limited by low neutron flux, leading to long acquisition times and noisy data. Here, we hypothesize that Bayesian statistical inference using Gaussian Process Regression (GPR) can reconstruct high-fidelity scattering data from sparse measurements by leveraging intensity smoothness and continuity. Experiments and Simulations The method was benchmarked computationally and validated through SANS experiments on various soft matter systems, including wormlike micelles, colloidal suspensions, polymeric structures, and lyotropic phases. GPR-based inference was applied to both experimental and synthetic data to evaluate its effectiveness in noise reduction and intensity reconstruction. Findings GPR significantly enhances SANS data quality and therefore reducing measurement times by up to two orders of magnitude. This cost-effective approach maximizes experimental efficiency, enabling high-throughput studies and real-time monitoring of dynamic systems. It is particularly beneficial for weakly scattering and time-sensitive studies. Beyond SANS, this framework applies to other low-SNR techniques, including laboratory-based small-angle X-ray scattering and various dynamical scattering methods. Furthermore, it offers transformative potential for compact neutron sources, enhancing their viability for structural analysis in resource-limited settings.

Small angle neutron scattering↗

A score-based diffusion model approach for adaptive learning of stochastic partial differential equation solutions

In this paper, we propose a novel framework for adaptively learning the time-evolving solutions of stochastic partial differential equations (SPDEs) using score-based diffusion models within a recursive Bayesian inference setting. SPDEs play a central role in modeling complex physical systems under uncertainty, but their numerical solutions often suffer from model errors and reduced accuracy due to incomplete physical knowledge and environmental variability. To address these challenges, we encode the governing physics into the score function of a diffusion model using simulation data and incorporate observational information via a likelihood-based correction in a reverse-time stochastic differential equation. This enables adaptive learning through iterative refinement of the solution as new data becomes available. To improve computational efficiency in high-dimensional settings, we introduce the ensemble score filter, a training-free approximation of the score function designed for real-time inference. Numerical experiments on benchmark SPDEs demonstrate the accuracy and robustness of the proposed method under sparse and noisy observations.

97 MATHEMATICS AND COMPUTING↗

Bayesian uncertainty quantification of tristructural isotropic particle fuel silver release: Decomposing model inadequacy plus experimental noise and parametric uncertainties

Tristructural isotropic (TRISO) particle fuel is one of the most promising fuel concepts enabling high temperature and high burnup reactor operation. One dominant source of radioactivity released from the TRISO particles is silver (Ag), which is subject to a high release fraction and long decay life compared to other fission products. Previous modeling efforts using the fuel performance code BISON indicated nonnegligible uncertainties in modeling the diffusion process of fission products in TRISO compared to the Advanced Gas Reactor experiments. The overall uncertainties observed when modeling the fission product diffusion can result from uncertainties in model parameters, noisy experimental measurements, and deficiencies in the developed models. The three types of underlying uncertainties have not yet been properly quantified in open literature. Here, this paper presents the Bayesian uncertainty quantification (UQ) using massively parallelizable Markov chain Monte Carlo samplers. The uncertainties due to model parameters, model inadequacy, and experimental measurement noise are quantified, with the σ term used to represent the sum of the model inadequacy and measurement noise uncertainties. It is worth noting that this is the first time the σ term is inferred for nuclear fuel experiments, as compared to using prescribed values for uncertainty quantification in previous work. The parallelizable Markov chain Monte Carlo samplers efficiently infer the model parameters and the σ term, giving insight into physical parameters like diffusion coefficients and the combined model discrepancy and measurement noise. A subsequent forward uncertainty quantification (UQ) is also performed based on the calibration results to generate more accurate predictions of the Ag release. The model inadequacy plus experimental noise is the most dominant source of uncertainty compared to the parametric uncertainty. All the UQ analyses presented in this work are based on the second series of the irradiation experiments in the Advanced Gas Reactor program.

11 NUCLEAR FUEL CYCLE AND FUEL MATERIALS↗

MOOSE ProbML: Parallelized probabilistic machine learning and uncertainty quantification for computational energy applications

Here, this paper presents the development and demonstration of massively parallel probabilistic machine learning (ML) and uncertainty quantification (UQ) capabilities within the Multiphysics Object-Oriented Simulation Environment (MOOSE), an open-source computational platform for parallel finite element and finite volume analyses. In addressing the computational expense and uncertainties inherent in complex multiphysics simulations, this paper integrates Gaussian process (GP) variants, active learning, Bayesian inverse UQ, adaptive forward UQ, Bayesian optimization, evolutionary optimization, and Markov chain Monte Carlo (MCMC) within MOOSE. It also elaborates on the interaction among key MOOSE systems — Sampler, MultiApp, Reporter, and Surrogate — in enabling these capabilities. The modularity offered by these systems enables development of a multitude of probabilistic ML and UQ algorithms in MOOSE. Example code demonstrations include parallel active learning and parallel Bayesian inference via active learning. The impact of these developments is illustrated through five applications relevant to computational energy applications: UQ of nuclear fuel fission product release, using parallel active learning Bayesian inference; very rare events analysis in nuclear microreactors using active learning; advanced manufacturing process modeling using multi-output GPs (MOGPs) and dimensionality reduction; fluid flow using deep GPs (DGPs); and tritium transport model parameter optimization for fusion energy, using batch Bayesian optimization. These capabilities are part of the MOOSE framework.

97 - MATHEMATICS AND COMPUTING↗

Statistical characterization of twin transmission across grain boundaries in magnesium

Here, a detailed statistical analysis of twin transmission (TT) across grain boundaries is performed in rolled, commercial purity magnesium compressed along the rolling direction. EBSD images are acquired from two different cuts: a section containing the rolling (RD) and normal (ND) directions; and a section at 35° to the ND which contains transverse direction (TD). An automated twinning analysis software, METIS, is used to obtain the statistical correlations between deformation twins and other microstructural features by analyzing EBSD microstructures comprising thousands of grains and twins. This detailed statistical analysis reveals that the TT propensity is sensitive to the grain boundary (GB) misorientation angle but not to the GB misorientation axis. Specifically, TT propensity decreases with increasing GB misorientation angle; however, the decreasing trend is not monotonic. Further, the detailed analysis of TT events combining macroscopic Schmid factor and the geometric measures (m’: accounts for the alignment of both twin plane normal and shear directions; m”: accounts for the alignment of only twin shear directions; and m”’: accounts for the alignment of only twin plane normal) helps in understanding and identifying the process of twin-pair formation, i.e., co-nucleation versus transmission, also to identify the role of local stresses at GBs induced by inter-grain interaction versus twinning shear transformation processes. In addition, Bayesian inference is used to draw statistically meaningful conclusions as to the likelihood of transmission given misorientation angles. Lastly, twin chain frequency as a function of chain length is predicted using the probability distribution obtained from the Bayesian inference and compared with the actual data from the EBSD microstructures.

36 MATERIALS SCIENCE↗

Jensen–Shannon divergence based novel loss functions for Bayesian neural networks

Bayesian neural networks (BNNs) are state-of-the-art machine learning methods that can naturally regularize and systematically quantify uncertainties using their stochastic parameters. Kullback–Leibler (KL) divergence-based variational inference used in BNNs suffer from unstable optimization and challenges in approximating light-tailed posteriors due to the unbounded nature of the KL divergence. To resolve these issues, we formulate a novel loss function for BNNs based on a new modification to the generalized Jensen–Shannon (JS) divergence, which is bounded. In addition, we propose a Geometric JS divergence-based loss, which is computationally efficient since it can be evaluated analytically. We found that the JS divergence-based variational inference is intractable, and hence employed a constrained optimization framework to formulate these losses. Our theoretical analysis and empirical experiments on multiple regression and classification data sets suggest that the proposed losses perform better than the KL divergence-based loss, especially when the data sets are noisy or biased. Specifically, there are approximately 5% and 8% improvements in accuracy for a noise-added CIFAR-10 dataset and a regression dataset, respectively. There is about 13% reduction in false negative predictions of a biased histopathology dataset. Additionally, we quantify and compare the uncertainty metrics for the regression and classification tasks.

97 MATHEMATICS AND COMPUTING↗

A flexible event reconstruction based on machine learning and likelihood principles

Event reconstruction is a central step in many particle physics experiments, turning detector observables into parameter estimates; for example estimating the energy of an interaction given the sensor readout of a detector. A corresponding likelihood function is often intractable, and approximations need to be constructed. Here, in our work, we first show how the full likelihood for a many-sensor detector can be broken apart into smaller terms, and secondly how we can train neural networks to approximate all terms solely based on forward simulation. Our technique results in a fast, flexible, and close-to-optimal surrogate model proportional to the likelihood and can be used in conjunction with standard inference techniques allowing for a consistent treatment of uncertainties. We illustrate our technique for parameter inference in neutrino telescopes based on maximum likelihood and Bayesian posterior sampling. Given its great flexibility, we also showcase our method for geometry optimization enabling to learn optimal detector designs. Lastly, we apply our method to realistic simulation of a ton-scale water-based liquid scintillator detector.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗

Learning from crowds with variational Gaussian processes

Solving a supervised learning problem requires to label a training set. This task is traditionally performed by an expert, who provides a label for each sample. The proliferation of social web services (e.g., Amazon Mechanical Turk) has introduced an alternative crowdsourcing approach. Anybody with a computer can register in one of these services and label, either partially or completely, a dataset. The effort of labeling is then shared between a great number of annotators. However, this approach introduces scientifically challenging problems such as combining the unknown expertise of the annotators, handling disagreements on the annotated samples, or detecting the existence of spammer and adversarial annotators. All these problems require probabilistic sound solutions which go beyond the naive use of majority voting plus classical classification methods. In this work we introduce a new crowdsourcing model and inference procedure which trains a Gaussian Process classifier using the noisy labels provided by the annotators. Variational Bayes inference is used to estimate all unknowns. The proposed model can predict the class of new samples and assess the expertise of the involved annotators. Moreover, the Bayesian treatment allows for a solid uncertainty quantification. Since when predicting the class of a new sample we might have access to some annotations for it, we also show how our method can naturally incorporate this additional information. Furthermore, a comprehensive experimental section evaluates the proposed method with synthetic and real experiments, showing that it consistently outperforms other state-of-the-art crowdsourcing approaches.

97 MATHEMATICS AND COMPUTING↗

A Bayesian framework to investigate radiation reaction in strong fields

Recent experiments aiming to measure phenomena predicted by strong-field quantum electrodynamics (SFQED) have done so by colliding relativistic electron beams and high-power lasers. In such experiments, measurements of collision parameters are not always feasible. However, precise knowledge of these parameters is required to accurately test SFQED. Here, we present a novel Bayesian inference procedure that infers collision parameters that could not be measured on-shot. This procedure is applicable to all-optical non-linear Compton scattering experiments investigating radiation reaction. The framework allows multiple diagnostics to be combined self-consistently and facilitates the inclusion of known information pertaining to the collision parameters. Using this Bayesian analysis, the relative validity of the classical, quantum-continuous and quantum-stochastic models of radiation reaction was compared for several test cases, which demonstrates the accuracy and model selection capability of the framework and highlight its robustness if the experimental values of fixed parameters differ from their values in the models.

47 OTHER INSTRUMENTATION↗

Particle transport constraints via Bayesian spectral fitting of multiple atomic lines

Optimized operation of fusion devices demands detailed understanding of plasma transport, a problem that must be addressed with advances in both measurement and data analysis techniques. In this work, we adopt Bayesian inference methods to determine experimental particle transport, leveraging opportunities from high-resolution He-like ion spectra in a tokamak plasma. The Bayesian spectral fitting code is used to analyze resonance (w), forbidden (z), intercombination (x, y), and satellite (k, j) lines of He-like Ca following laser blow-off injections on Alcator C-Mod. This offers powerful transport constraints since these lines depend differently on electron temperature and density, but also differ in their relation to Li-like, He-like, and H-like ion densities, often the dominant Ca charge states over most of the C-Mod plasma radius. Using synthetic diagnostics based on the AURORA package, we demonstrate improved effectiveness of impurity transport inferences when spectroscopic data from a progressively larger number of lines are included.

47 OTHER INSTRUMENTATION↗

Radiation Source Localization Using Surrogate Models Constructed from 3-D Monte Carlo Transport Physics Simulations

Recent research has focused on the development of surrogate models for radiation source localization in a simulated urban domain. We employ the Monte Carlo N-Particle (MCNP) code to provide high- delity simulations of radiation transport within an urban domain. The model is constructed to employ a source location (x, y, z) as input and return the estimated count rate for a set of speci ed detector locations. Because MCNP simulations are computationally expensive, we develop e cient and accurate surrogate models of the detector responses. We construct surrogate models using Gaussian processes (GP) and neural networks (NN) that we train and verify using the MCNP simulations. The trained surrogate models provide an e cient framework for Bayesian inference and experimental design. We employ Delayed Rejection Adaptive Metropolis (DRAM), a Markov Chain Monte Carlo (MCMC) algorithm, to infer the location and intensity of an unknown source. The DRAM results yield a posterior probability distribution for the source's location conditioned on the observed detector count rates. The posterior distribution exhibits regions of high and low probability within the simulated environment identifying potential source locations. In this manner, we can quantify the source location to within at least one of these regions of high probability in the considered cases. Employing these methods, we are able to reduce the space of potential source locations by at least 60%.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗