Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “UQ”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 217 records · Page 12

A flexible uncertainty quantification method for linearly coupled multi-physics systems

Highlights: •We propose a “modularly hybrid” UQ methodology suitable for independent development of module-based multi-physics simulation. •Our algorithmic framework allows for each module to have its own UQ method (either intrusive or non-intrusive). •Information from each module is combined systematically to propagate “global uncertainty”. •Our proposed approach can allow for easy swapping of new methods for any modules without the need to address incompatibilities. •We demonstrate the proposed framework on a practical application involving a multi-species reactive transport model. -- Abstract: This paper presents a novel approach to building an integrated uncertainty quantification (UQ) methodology suitable for modern-day component-based approach for multi-physics simulation development. Our “hybrid” UQ methodology supports independent development of the most suitable UQ method, intrusive or non-intrusive, for each physics module by providing an algorithmic framework to couple these “stochastic” modules for propagating “global” uncertainties. We address algorithmic and computational issues associated with the construction of this hybrid framework. We demonstrate the utility of such a framework on a practical application involving a linearly coupled multi-species reactive transport model.

97 MATHEMATICS AND COMPUTING↗

Unpacking model inadequacy: The quantification of silver release from TRISO fuel by considering empirical and mechanistic approaches

Increasing adoption of the proposed tristructural isotropic (TRISO) particle fuel for both advanced and existing reactors makes it critical to assess and address any uncertainties and inadequacies of TRISO fission product release models. Model inadequacy stems from simplifications made to the computational model when compared to the experiments. The modeling and simulation efforts conducted using the BISON fuel performance code, along with the experimental campaigns carried out under the Advanced Gas Reactor Fuel Development and Qualification Program, afford a unique opportunity to conduct a rigorous modeling inadequacy assessment within the Bayesian uncertainty quantification (UQ) framework. Here, this study compares the standard Bayesian framework against the Kennedy-O'Hagan (KOH) framework, which explicitly represents modeling inadequacy, in regard to UQ for TRISO silver release models. For this purpose, both the traditional Arrhenius equation fitted to experimental data and the more advanced lower-length-scale (LLS)-informed model, which considers microstructure information, are independently considered. Applying the inverse UQ process on the AGR-2 and -3/4 datasets revealed modeling inadequacy to be the most dominant source of uncertainty. Experimental noise uncertainty is also significant; however, model parameter uncertainty can be considered negligible. Interestingly, both the Arrhenius equation and the LLS-informed model demonstrated similar levels of modeling inadequacy. For the forward predictive UQ, the KOH framework improved both the accuracy and quality of quantified uncertainties in comparison to the standard Bayesian framework. This is true for both the Arrhenius equation and the LLS-informed model. In comparing these modeling approaches, both demonstrated similar performance at the engineering scale, while the LLS-informed model expectedly outperformed the Arrhenius equation at the mesoscale. These conclusions highlight the importance of explicitly accounting for modeling inadequacy in the UQ process, and reinforce the need for continuous refinement of physics-based models in order to address the modeling inadequacy.

Advanced reactors↗

UQpy: A general purpose Python package and development environment for uncertainty quantification

In this paper, we present the UQpy software toolbox, an open-source Python package for general uncertainty quantification (UQ) in mathematical and physical systems. The software serves as both a user-ready toolbox that includes many of the latest methods for UQ in computational modeling and a convenient development environment for Python programmers advancing the field of UQ. The paper presents an introduction to the software's architecture and existing capabilities, divided in the code in a set of modules centered around different UQ tasks such as sampling methods, generation of random processes and random fields, probabilistic inverse modeling, reliability analysis, surrogate modeling, and active learning. The paper also highlights the importance of the RunModel module, which is used to drive simulations in the uncertainty analyses performed in UQpy. This module conveniently allows the user to define computational models directly in Python, or to run simulations from a third-party software in serial or in parallel. To illustrate the various capabilities, two examples are tracked throughout the paper and analyzed repeatedly for various UQ tasks. The first is a Python model solving a nonlinear structural dynamics problem, used to illustrate UQpy's capabilities in sampling and forward propagation of high dimensional random vectors (stochastic processes), and probabilistic inference. The second model is a third-party Abaqus finite element model solving the thermomechanical response of a beam structure. This example is used to illustrate UQpy's capabilities in variance reduction sampling techniques, reliability analysis, surrogate modeling and active learning techniques.

97 MATHEMATICS AND COMPUTING↗

MOOSE ProbML: Parallelized probabilistic machine learning and uncertainty quantification for computational energy applications

Here, this paper presents the development and demonstration of massively parallel probabilistic machine learning (ML) and uncertainty quantification (UQ) capabilities within the Multiphysics Object-Oriented Simulation Environment (MOOSE), an open-source computational platform for parallel finite element and finite volume analyses. In addressing the computational expense and uncertainties inherent in complex multiphysics simulations, this paper integrates Gaussian process (GP) variants, active learning, Bayesian inverse UQ, adaptive forward UQ, Bayesian optimization, evolutionary optimization, and Markov chain Monte Carlo (MCMC) within MOOSE. It also elaborates on the interaction among key MOOSE systems — Sampler, MultiApp, Reporter, and Surrogate — in enabling these capabilities. The modularity offered by these systems enables development of a multitude of probabilistic ML and UQ algorithms in MOOSE. Example code demonstrations include parallel active learning and parallel Bayesian inference via active learning. The impact of these developments is illustrated through five applications relevant to computational energy applications: UQ of nuclear fuel fission product release, using parallel active learning Bayesian inference; very rare events analysis in nuclear microreactors using active learning; advanced manufacturing process modeling using multi-output GPs (MOGPs) and dimensionality reduction; fluid flow using deep GPs (DGPs); and tritium transport model parameter optimization for fusion energy, using batch Bayesian optimization. These capabilities are part of the MOOSE framework.

97 - MATHEMATICS AND COMPUTING↗

Simulation of a TRISO MiniFuel irradiation experiment with data-informed uncertainty quantification

An irradiation experiment using tristructural isotropic (TRISO) fuel particles and the miniature fuel (MiniFuel) irradiation vehicle was performed in Oak Ridge National Laboratory’s High Flux Isotope Reactor (HFIR) to support development of the Kairos Power fluoride salt–cooled, high-temperature reactor (KP-FHR). Here, this paper describes modeling predictions of temperatures and fuel burnup for the as-built experiment. An uncertainty quantification (UQ) analysis was performed to determine the effect of TRISO particle volume and position on the temperature predictions at various fuel heat generation rates (HGRs). This UQ study utilized fuel kernel position and volume measurements previously collected using X-ray computed tomography (XCT) techniques and Monte Carlo sampling methods to generate fuel compact cases that were then analyzed using a finite element thermal model. The UQ analysis indicated that uncertainty in calculated temperatures caused by varying TRISO particle arrangement is relatively small, even at high fuel HGR. Final predictions of particle temperatures throughout the irradiation are shown to be relevant to KP-FHR normal and off-normal operating conditions and to previous TRISO irradiation experiments. The combination of XCT with UQ analyses will inform post-irradiation examination (PIE) of the irradiated fuel compacts, and these analyses can be used to develop fuel performance models for coated particle fuel forms. Both PIE of separate-effects irradiation data and enhanced fuel performance modeling support accelerated qualification of TRISO fuels for a broad range of advanced reactor applications. The novel approach demonstrated here of measuring TRISO particle configurations with XCT methods and generating representative fuel compacts for finite element modeling and UQ analysis could be leveraged by the broader particle fuel community in the development of other TRISO fuel experiments in which these variables may have a significant impact on key outcomes.

11 NUCLEAR FUEL CYCLE AND FUEL MATERIALS↗

Nonintrusive Uncertainty Quantification of Computational Fluid Dynamics Simulations of a Bench-Scale Fluidized-Bed Gasifier

Uncertainty quantification (UQ) analysis is increasingly becoming one of the major requirements of simulation-based engineering to assess the confidence in the results and make better-informed decisions based on the insight derived from the simulations. In an earlier study, Bayesian UQ analysis was applied to existing bench-scale fluidized-bed gasifier experiment results. In the current study, a series of simulations were carried over with the open-source computational fluid dynamics software MFiX to reproduce the experimental conditions, where three operating factors, i.e., coal flow rate, coal particle diameter, and steam-to-oxygen ratio, were systematically varied to understand their effect on the syngas composition. Bayesian UQ analysis was this time performed on the numerical results for comparison purposes. This is part of ongoing research efforts to explore the applicability of advanced UQ methods and processes such as Bayesian methods for large-scale complex multiphase flow simulations. As part of Bayesian UQ analysis, a global sensitivity analysis was performed based on the simulation results, which shows that the predicted syngas composition is strongly affected not only by the steam-to-oxygen ratio (which was observed in experiments as well) but also by variation in the coal flow rate and particle diameter (which was not observed in experiments). The carbon monoxide mole fraction is underpredicted at lower steam-to-oxygen ratios and overpredicted at higher steam-to-oxygen ratios. The opposite trend is observed for the carbon dioxide mole fraction. These discrepancies are attributed to either excessive segregation of the phases that leads to the fuel-rich or -lean regions or alternatively the selection of reaction models, where different reaction models and kinetics can lead to different syngas compositions throughout the gasifier.

01 COAL, LIGNITE, AND PEAT↗

Uncertainty quantification in multivariable regression for material property prediction with Bayesian neural networks

With the increased use of data-driven approaches and machine learning-based methods in material science, the importance of reliable uncertainty quantification (UQ) of the predicted variables for informed decision-making cannot be overstated. UQ in material property prediction poses unique challenges, including multi-scale and multi-physics nature of materials, intricate interactions between numerous factors, limited availability of large curated datasets, etc. In this work, we introduce a physics-informed Bayesian Neural Networks (BNNs) approach for UQ, which integrates knowledge from governing laws in materials to guide the models toward physically consistent predictions. To evaluate the approach, we present case studies for predicting the creep rupture life of steel alloys. Experimental validation with three datasets of creep tests demonstrates that this method produces point predictions and uncertainty estimations that are competitive or exceed the performance of conventional UQ methods such as Gaussian Process Regression. Additionally, we evaluate the suitability of employing UQ in an active learning scenario and report competitive performance. The most promising framework for creep life prediction is BNNs based on Markov Chain Monte Carlo approximation of the posterior distribution of network parameters, as it provided more reliable results in comparison to BNNs based on variational inference approximation or related NNs with probabilistic outputs.

36 MATERIALS SCIENCE↗

The Sensitivity of Variational Bayesian Neural Network Performance to Hyperparameters

In scientific applications, predictive modeling is often of limited use without accurate uncertainty quantification (UQ) to indicate when a model may be extrapolating or when more data needs to be collected. Bayesian Neural Networks (BNNs) produce predictive uncertainty by propagating uncertainty in neural network (NN) weights and offer the promise of obtaining not only an accurate predictive model but also accurate UQ. However, in practice, obtaining accurate UQ with BNNs is difficult due in part to the approximations used for model training (such as those made in variational inference) and in part to the need to choose a suitable set of hyperparameters; these hyperparameters outnumber those needed for traditional NNs and often have opaque effects on the results. We aim to shed light on the effects of hyperparameter choices for variational BNNs by performing a global sensitivity analysis of variational BNN performance under varying hyperparameter settings. Our results indicate that many of the hyperparameters interact with each other to affect both predictive accuracy and UQ. For improved usage of variational BNNs in real-world applications, we suggest that thorough hyperparameter tuning, including tuning of prior hyperparameters and loss function parameters, is essential for accurate UQ in variational BNNs.

97 MATHEMATICS AND COMPUTING↗

Deep neural network uncertainty quantification for LArTPC reconstruction

We evaluate uncertainty quantification (UQ) methods for deep learning applied to liquid argon time projection chamber (LArTPC) physics analysis tasks. As deep learning applications enter widespread usage among physics data analysis, neural networks with reliable estimates of prediction uncertainty and robust performance against overconfidence and out-of-distribution (OOD) samples are critical for their full deployment in analyzing experimental data. While numerous UQ methods have been tested on simple datasets, performance evaluations for more complex tasks and datasets are scarce. Here we assess the application of selected deep learning UQ methods on the task of particle classification using the PiLArNet monte carlo 3D LArTPC point cloud dataset. We observe that UQ methods not only allow for better rejection of prediction mistakes and OOD detection, but also generally achieve higher overall accuracy across different task settings. We assess the precision of uncertainty quantification using different evaluation metrics, such as distributional separation of prediction entropy across correctly and incorrectly identified samples, receiver operating characteristic curves (ROCs), and expected calibration error from observed empirical accuracy. We conclude that ensembling methods can obtain well calibrated classification probabilities and generally perform better than other existing methods in deep learning UQ literature.

47 OTHER INSTRUMENTATION↗

Clarifying trust of materials property predictions using neural networks with distribution-specific uncertainty quantification

It is critical that machine learning (ML) model predictions be trustworthy for high-throughput catalyst discovery approaches. Uncertainty quantification (UQ) methods allow estimation of the trustworthiness of an ML model, but these methods have not been well explored in the field of heterogeneous catalysis. Herein, we investigate different UQ methods applied to a crystal graph convolutional neural network to predict adsorption energies of molecules on alloys from the Open Catalyst 2020 dataset, the largest existing heterogeneous catalyst dataset. We apply three UQ methods to the adsorption energy predictions, namely k-fold ensembling, Monte Carlo dropout, and evidential regression. The effectiveness of each UQ method is assessed based on accuracy, sharpness, dispersion, calibration, and tightness. Evidential regression is demonstrated to be a powerful approach for rapidly obtaining tunable, competitively trustworthy UQ estimates for heterogeneous catalysis applications when using neural networks. Recalibration of model uncertainties is shown to be essential in practical screening applications of catalysts using uncertainties.

36 MATERIALS SCIENCE↗

Evaluating probabilistic deep learning methods for uncertainty quantification of temperature downscaling

Deep learning (DL) has emerged as a promising tool for downscaling coarse-resolution climate data to high-resolution outputs, enabling improved regional climate predictions. A critical aspect of DL-based downscaling is the incorporation of uncertainty quantification (UQ), which enhances the interpretability and reliability of predictions—key factors for climate risk assessment and decision-making. This study develops a DL model to downscale 2 m temperature across the contiguous United States using reanalysis datasets. We systematically evaluate three epistemic UQ methods—deep ensembles (DEns), Monte Carlo dropout (MCD), and Flipout—based on their probabilistic accuracy, downscaling performance, sensitivity to geographical features, and computational efficiency. Results indicate that MCD generally outperforms Flipout and DEns in terms of calibration and downscaling accuracy. However, DEns demonstrate lower calibration errors in coastal regions, indicating its higher confidence within these areas. Flipout, in contrast, is more sensitive to elevation gradients and exhibits higher calibration errors in mountainous regions. Hence, the choice of UQ method for this task depends on the specific requirements of the application. For applications that prioritize overall calibration, downscaling accuracy, and computational efficiency, MCD is a strong candidate. These findings highlight the importance of selecting UQ methods based on application-specific requirements, such as geographical context and computational constraints. By addressing the trade-offs between UQ methods, this study provides actionable insights for improving the reliability, scalability, and utility of DL-based downscaling in climate science.

Environmental sciences↗

Evaluating Probabilistic Deep Learning Methods for Uncertainty Quantification of Precipitation Bias Correction

Climate models often exhibit biases in their precipitation predictions, particularly underestimating high-intensity events and overestimating low precipitation. Deep learning approaches offer promising solutions, but their epistemic uncertainty associated with a deep learning–based bias correction method has not previously been quantified for reliable downstream climate impact studies. While methods for capturing the epistemic uncertainty in deep learning frameworks exist, there is currently no consensus on the best method. In this work, we compare three uncertainty quantification (UQ) methods—Deep Ensembles (DEns), Monte Carlo Dropout (MCD), and Flipout—by assessing the reliability of their uncertainty estimates using standard measures such as sharpness and calibration. These UQ methods are applied to an existing deep learning precipitation bias correction model known as UFNet: a coupled U-Net and fully connected neural network. The methods utilized to assess the models’ uncertainties are 1) calibration, which ensures that the expected probabilities of the model align with reality and 2) sharpness, which is a measure of the precision of the model’s probabilistic predictions. Of the three UQ methods evaluated, the DEns and MCD methods demonstrated the best-calibrated performance (expected calibration error of 0.36 and 0.35, respectively), compared to Flipout (0.58). In contrast, Flipout had the sharpest predictions and the highest metric performance in bias correcting precipitation—especially for higher-order moments such as kurtosis with a spatial correlation of 72% compared to 32% and 55% spatial correlation for DEns and MCD, respectively. Of the three UQ methods, MCD was found to be the most suitable method for UQ purposes based on its calibration, sharpness, and computational requirements.

Bayesian methods↗

Analysis of the Challenges in Developing Sample-Based Multi-fidelity Estimators for Non-deterministic Models

Multifidelity (MF) uncertainty quantification (UQ) seeks to leverage and fuse information from a collection of models to achieve greater statistical accuracy with respect to a single-fidelity counterpart, while maintaining an efficient use of computational resources. Despite many recent advancements in MF UQ, several challenges remain and these often limit its practical impact in certain application areas. In this manuscript, we focus on the challenges introduced by nondeterministic models to sampling MF UQ estimators. Nondeterministic models produce different responses for the same inputs, which means their outputs are effectively noisy. MF UQ is complicated by this noise since many state-of-the-art approaches rely on statistics, e.g., the correlation among models, to optimally fuse information and allocate computational resources. Here, we demonstrate how the statistics of the quantities of interest, which impact the design, effectiveness, and use of existing MF UQ techniques, change as functions of the noise. With this in hand, we extend the unifying approximate control variate framework to account for nondeterminism, providing for the first time a rigorous means of comparing the effect of nondeterminism on different multifidelity estimators and analyzing their performance with respect to one another. Numerical examples are presented throughout the manuscript to illustrate and discuss the consequences of the presented theoretical results.

97 MATHEMATICS AND COMPUTING↗

Final Technical Report: Quantification of Uncertainty in Extreme Scale Computations (QUEST)

QUEST is a SciDAC Institute comprising Sandia National Laboratories, Los Alamos National Laboratory, University of Southern California, Massachusetts Institute of Technology, University of Texas at Austin, and Duke University. The mission of QUEST is to: (1) develop a broad class of uncertainty quantification (UQ) methods/tools, and (2) provide UQ expertise and software to other SciDAC projects, thereby enabling/guiding their UQ activities. The Duke effort focused on the development of algorithms and utility software for non-intrusive sparse UQ representations, and on participation in the organization of annual workshops and tutorials to disseminate UQ tools to the community, and to gather input in order to adapt approaches to the needs of SciDAC customers. In particular, fundamental developments were made in (a) multiscale stochastic preconditioners, (b) gradient-based approaches to inverse problems, (c) adaptive pseudo-spectral approximations, (d) stochastic limit cycles, and (e) sensitivity analysis tools for noisy systems. In addition, large-scale demonstrations were performed, namely in the context of ocean general circulation models.

97 MATHEMATICS AND COMPUTING↗

Leveraging Intrinsic Principal Directions for Multifidelity Uncertainty Quantification

In this work we propose an approach for accelerating Uncertainty Quantification (UQ) analysis in the context of Multifidelity applications. In the presence of complex multiphysics applications, which often require a prohibitive computational cost for each evaluation, multifidelity UQ techniques try to accelerate the convergence of statistics by leveraging the in- formation collected from a larger number of a lower fidelity model realizations. However, at the-state-of-the-art, the performance of virtually all the multifidelity UQ techniques is related to the correlation between the high and low-fidelity models. In this work we proposed to design a multifidelity UQ framework based on the identification of independent important directions for each model. The main idea is that if the responses of each model can be represented in a common space, this latter can be shared to enhance the correlation when the samples are drawn with respect to it instead of the original variables. There are also two main additional advantages that follow from this approach. First, the models might be correlated even if their original parametrizations are chosen independently. Second, if the shared space between models has a lower dimensionality than the original spaces, the UQ analysis might benefit from a dimension reduction standpoint. In this work we designed this general framework and we also tested it on several test problems ranging from analytical functions for verification purpose, up to more challenging application problems as an aero-thermo-structural analysis and a scramjet flow analysis.

42 ENGINEERING↗

Uncertainty Quantification for JPL Retrievals

We define the notions of verification, validation, and uncertainty quantification (VVUQ), and place them in the context of retrievals of Earth science data. Uncertainty quantification (UQ) is the process of characterizing the uncertainty of a measurement system relative to the true, physical quantity it measures. UQ is needed when measurements are used for scientific inference, assimilated into models with other data, and used for decision-making and risk analysis. Many measurements come with measures of standard error and bias, but these uncertainty measures are often inaccurate, due to the operational need to assume crisp values for parameters which are in fact uncertain, as well as the failure of technical assumptions needed by operational error-estimation approaches. UQ provides a way to study which assumptions fail to hold, and a tool to improve the measurement. We describe a particular, broadly-applicable UQ setup that has been successful for JPL retrievals including OCO-2, and provide recommendations for furthering the UQ state of the practice at JPL.

Turmon, Michael↗

Integration of Uncertainty Quantification in a Model-Based Systems Analysis and Engineering Framework

This paper presents a technical approach to improve the confidence in the systems analysis process by integrating Uncertainty Quantification (UQ) techniques within a Model-Based Systems Analysis and Engineering (MBSA&E) framework. The MBSA&E architecture uses system models and multidisciplinary analytical solutions as central artifacts for system design and analysis. The integration of UQ enables engineers to assess and mitigate uncertainties associated with a system model, design parameters, and constraint inputs, leading to more complete design studies and further informed decision-making processes. The proposed approach leverages the strengths of MBSA&E and extends it with a UQ methodology to quantify uncertainties in the input parameters and to trace the uncertainties as they propagate throughout the system model. To demonstrate the effectiveness of an integrated MBSA&E-UQ approach, a case study involving a simplified analysis of a Transonic Truss-Braced Wing (TTBW) concept vehicle is performed. This integration enables a more comprehensive evaluation of system performance and behavior under uncertainty and a more robust approach for system design and analysis. Lastly, the paper addresses the challenges and considerations associated with integrating UQ into an MBSA&E framework.

systems analysis↗

Integration of Uncertainty Quantification in a Model-Based Systems Analysis and Engineering Framework

This paper presents a technical approach to improve the confidence in the systems analysis process by integrating Uncertainty Quantification (UQ) techniques within a Model-Based Systems Analysis and Engineering (MBSA&E) framework. The MBSA&E architecture uses system models and multidisciplinary analytical solutions as central artifacts for system design and analysis. The integration of UQ enables engineers to assess and mitigate uncertainties associated with a system model, design parameters, and constraint inputs, leading to more complete design studies and further informed decision-making processes. The proposed approach leverages the strengths of MBSA&E and extends it with a UQ methodology to quantify uncertainties in the input parameters and to trace the uncertainties as they propagate throughout the system model. To demonstrate the effectiveness of an integrated MBSA&E-UQ approach, a case study involving a simplified analysis of a Transonic Truss-Braced Wing (TTBW) concept vehicle is performed. This integration enables a more comprehensive evaluation of system performance and behavior under uncertainty and a more robust approach for system design and analysis. Lastly, the paper addresses the challenges and considerations associated with integrating UQ into an MBSA&E framework.

systems analysis↗