Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “uncertainty quantification (UQ)”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 55 records · Page 3
All sourcesNASA NTRSDOE OSTI
DOE OSTI2024

Uncertainty quantification for molecular property predictions with graph neural architecture search

Graph Neural Networks (GNNs) have emerged as a prominent class of data-driven methods for molecular property prediction. However, a key limitation of typical GNN models is their inability to quantify uncertainties in the predictions. This capability is crucial for ensuring the trustworthy use and deployment of models in downstream tasks. To that end, we introduce AutoGNNUQ, an automated uncertainty quantification (UQ) approach for molecular property prediction. AutoGNNUQ leverages architecture search to generate an ensemble of high-performing GNNs, enabling the estimation of predictive uncertainties. Our approach employs variance decomposition to separate data (aleatoric) and model (epistemic) uncertainties, providing valuable insights for reducing them. In our computational experiments, we demonstrate that AutoGNNUQ outperforms existing UQ methods in terms of both prediction accuracy and UQ performance on multiple benchmark datasets, and generalizes well to out-of-distribution datasets. Additionally, we utilize t-SNE visualization to explore correlations between molecular features and uncertainty, offering insight for dataset improvement. AutoGNNUQ has broad applicability in domains such as drug discovery and materials science, where accurate uncertainty quantification is crucial for decision-making.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2025

Uncertainty Quantification for Smooth Functional Data with Application to Material Properties

This document outlines a method for processing functional output (i.e., curves) for the ultimate purpose of sampling curves under specified input conditions for use in modeling and simulation uncertainty quantification (UQ) studies. A set of benchmark curves sufficiently representative of the relevant scenario(s) being simulated are provided to the process and formatted as described in Section 1. Principal Component Analysis (PCA) is utilized to discover the components of uncertainty in the benchmark curves and is outlined in Section 2. Section 3 describes the application of uncertainty quantification to the PCA results for the purpose of sampling curves to be used in UQ analysis. Section 4 applies these techniques to an example benchmark dataset. Concluding remarks are provided in the final section.

36 MATERIALS SCIENCE↗
DOE OSTI2025

Evaluating Probabilistic Deep Learning Methods for Uncertainty Quantification of Precipitation Bias Correction

Climate models often exhibit biases in their precipitation predictions, particularly underestimating high-intensity events and overestimating low precipitation. Deep learning approaches offer promising solutions, but their epistemic uncertainty associated with a deep learning–based bias correction method has not previously been quantified for reliable downstream climate impact studies. While methods for capturing the epistemic uncertainty in deep learning frameworks exist, there is currently no consensus on the best method. In this work, we compare three uncertainty quantification (UQ) methods—Deep Ensembles (DEns), Monte Carlo Dropout (MCD), and Flipout—by assessing the reliability of their uncertainty estimates using standard measures such as sharpness and calibration. These UQ methods are applied to an existing deep learning precipitation bias correction model known as UFNet: a coupled U-Net and fully connected neural network. The methods utilized to assess the models’ uncertainties are 1) calibration, which ensures that the expected probabilities of the model align with reality and 2) sharpness, which is a measure of the precision of the model’s probabilistic predictions. Of the three UQ methods evaluated, the DEns and MCD methods demonstrated the best-calibrated performance (expected calibration error of 0.36 and 0.35, respectively), compared to Flipout (0.58). In contrast, Flipout had the sharpest predictions and the highest metric performance in bias correcting precipitation—especially for higher-order moments such as kurtosis with a spatial correlation of 72% compared to 32% and 55% spatial correlation for DEns and MCD, respectively. Of the three UQ methods, MCD was found to be the most suitable method for UQ purposes based on its calibration, sharpness, and computational requirements.

Bayesian methods↗
DOE OSTI2026

The Sensitivity of Variational Bayesian Neural Network Performance to Hyperparameters

In scientific applications, predictive modeling is often of limited use without accurate uncertainty quantification (UQ) to indicate when a model may be extrapolating or when more data needs to be collected. Bayesian Neural Networks (BNNs) produce predictive uncertainty by propagating uncertainty in neural network (NN) weights and offer the promise of obtaining not only an accurate predictive model but also accurate UQ. However, in practice, obtaining accurate UQ with BNNs is difficult due in part to the approximations used for model training (such as those made in variational inference) and in part to the need to choose a suitable set of hyperparameters; these hyperparameters outnumber those needed for traditional NNs and often have opaque effects on the results. We aim to shed light on the effects of hyperparameter choices for variational BNNs by performing a global sensitivity analysis of variational BNN performance under varying hyperparameter settings. Our results indicate that many of the hyperparameters interact with each other to affect both predictive accuracy and UQ. For improved usage of variational BNNs in real-world applications, we suggest that thorough hyperparameter tuning, including tuning of prior hyperparameters and loss function parameters, is essential for accurate UQ in variational BNNs.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2020

Uncertainty Quantification in Atomistic Modeling of Metals and Its Effect on Mesoscale and Continuum Modeling: A Review

The design of next-generation alloys through the integrated computational materials engineering (ICME) approach relies on multiscale computer simulations to provide thermodynamic properties when experiments are difficult to conduct. Atomistic methods such as density functional theory (DFT) and molecular dynamics (MD) have been successful in predicting properties of never before studied compounds or phases. However, uncertainty quantification (UQ) of DFT and MD results is rarely reported due to computational and UQ methodology challenges. Over the past decade, studies that mitigate this gap have emerged. These advances are reviewed in the context of thermodynamic modeling and information exchange with mesoscale methods such as the phase-field method (PFM) and calculation of phase diagrams (CALPHAD). The importance of UQ is illustrated using properties of metals, with aluminum as an example, and highlighting deterministic, frequentist, and Bayesian methodologies. Finally, challenges facing routine uncertainty quantification and an outlook on addressing them are also presented.

36 MATERIALS SCIENCE↗
DOE OSTI2021

Processing Aleatory and Epistemic Uncertainties in Experimental Data From Sparse Replicate Tests of Stochastic Systems for Real-Space Model Validation

This paper presents a practical methodology for propagating and processing uncertainties associated with random measurement and estimation errors (that vary from test-to-test) and systematic measurement and estimation errors (uncertain but similar from test-to-test) in inputs and outputs of replicate tests to characterize response variability of stochastically varying test units. Also treated are test condition control variability from test-to-test and sampling uncertainty due to limited numbers of replicate tests. These aleatory variabilities and epistemic uncertainties result in uncertainty on computed statistics of output response quantities. The methodology was developed in the context of processing experimental data for “real-space” (RS) model validation comparisons against model-predicted statistics and uncertainty thereof. The methodology is flexible and sufficient for many types of experimental and data uncertainty, offering the most extensive data uncertainty quantification (UQ) treatment of any model validation method the authors are aware of. It handles both interval and probabilistic uncertainty descriptions and can be performed with relatively little computational cost through use of simple and effective dimension- and order-adaptive polynomial response surfaces in a Monte Carlo (MC) uncertainty propagation approach. A key feature of the progressively upgraded response surfaces is that they enable estimation of propagation error contributed by the surrogate model. Sensitivity analysis of the relative contributions of the various uncertainty sources to the total uncertainty of statistical estimates is also presented. Finally, the methodologies are demonstrated on real experimental validation data involving all the mentioned sources and types of error and uncertainty in five replicate tests of pressure vessels heated and pressurized to failure. Simple spreadsheet procedures are used for all processing operations.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2023

Combined Data and Deep Learning Model Uncertainties: An Application to the Measurement of Solid Fuel Regression Rate

In complex physical process characterization, such as the measurement of the regression rate for solid hybrid rocket fuels, where both the observation data and the model used have uncertainties originating from multiple sources, combining these in a systematic way for quantities of interest (QoI) remains a challenge. In this paper, we present a forward propagation uncertainty quantification (UQ) process to produce a probabilistic distribution for the observed regression rate r. We characterized two input data uncertainty sources from the experiment (the distortion from the camera U c and the non-zero-angle fuel placement U Y ), the prediction and model form uncertainty from the deep neural network (U m ), as well as the variability from the manually segmented images used for training it (U s ). Here, we conducted seven case studies on combinations of these uncertainty sources with the model form uncertainty. The main contribution of this paper is the investigation and inclusion of the experimental image data uncertainties involved, and how to include them in a workflow when the QoI is the result of multiple sequential processes.

42 ENGINEERING↗
DOE OSTI2024

Stochastic finite volume method for uncertainty quantification of transient flow in gas pipeline networks

We develop a weakly intrusive framework to simulate the propagation of uncertainty in solutions of generic hyperbolic partial differential equation systems on graph-connected domains with nodal coupling and boundary conditions. The method is based on the Stochastic Finite Volume (SFV) approach and can be applied for uncertainty quantification (UQ) of the dynamical state of fluid flow over actuated transport networks. The numerical scheme has specific advantages for modeling intertemporal uncertainty in time-varying boundary parameters, which cannot be characterized by strict upper and lower (interval) bounds. We describe the scheme for a single pipe, and then formulate the controlled junction Riemann problem (JRP) that enables the extension to general network structures. In conclusion, we demonstrate the method's capabilities and performance characteristics using a standard benchmark test network.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2025

Uncertainty quantification for neural network potential foundation models

Abstract For neural network potentials (NNPs) to gain widespread use, researchers must be able to trust model outputs. However, the blackbox nature of neural networks and their inherent stochasticity are often deterrents, especially for foundation models trained over broad swaths of chemical space. Uncertainty information provided at the time of prediction can help reduce aversion to NNPs. In this work, we detail two uncertainty quantification (UQ) methods. Readout ensembling, by finetuning the readout layers of an ensemble of foundation models, provides information about model uncertainty, while quantile regression, by replacing point predictions with distributional predictions, provides information about uncertainty within the underlying training data. We demonstrate our approach with the MACE-MP-0 model, applying UQ to the foundation model and a series of finetuned models. The uncertainties produced by the readout ensemble and quantile methods are demonstrated to be distinct measures by which the quality of the NNP output can be judged.

36 MATERIALS SCIENCE↗
DOE OSTI2026

Experimental uncertainty quantification using templates of expected measurement uncertainties for fast neutron-induced total, capture, and scattering cross sections

Careful experimental uncertainty quantification (UQ) is key for developing trustworthy evaluated nuclear data. Templates to account for missing or under-reported experimental uncertainties were recently developed by the covariance committee of Cross Section Evaluation Working Group (CSEWG). In this work, we illustrate the practical application and limitations of these templates for selected neutron-induced reactions, including (n, tot), (n, γ), and (n, xn) in the fast energy range, to illustrate their use in data analyses for nuclear data evaluations. We show that while the templates provide consistent framework, proper implementation still requires detailed knowledge of experimental conditions and careful treatment of nonlinear effects in cross section derivation. Case studies highlight how template-assisted UQ improves consistency with previous evaluations such as ENDF/B and reveals open challenges in propagating uncertainties across different energy regimes. The main contribution of this paper is to connect formal template recommendations with their use in practical evaluation workflows, clarifying both their benefits and current limitations.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗
DOE OSTI2022

Advanced Graphite Creep Uncertainty Analysis

Radiation damage estimation is an important component of the post irradiation analysis of the Advanced Graphite Creep (AGC) experiment. It depends primarily on the fast fluence, which is determined using well established methods of spectral adjustment. These are based on best estimates from models such as Monte Carlo N-Particle (MCNP), input cross-sections, and measured activities from flux wires in the experiment. Each of these parameters can propagate uncertainties which will affect the uncertainty in the calculated dose levels for AGC, or any experiment irradiated within a reactor. While the methods of propagating uncertainty are well-established, the final uncertainty estimates they provide are only as good as the estimates of uncertainty in the inputs on which they are based. The purpose of this work is to outline some deficiencies in the ways these input uncertainties are presently estimated, and to outline a methodology by which they can be improved. The fast fluence and radiation damage received by graphite specimens irradiated in the Advanced Graphite Creep (AGC) experiments is presently estimated using spectral adjustment methods that are based on both flux wire activity measurements, and MCNP model predictions. This work describes an ongoing effort to quantify and propagate uncertainties in inputs to the spectral adjustment process, and thereby quantify the resultant error in radiation damage (dpa) estimates. The effort is multi-faceted, and we consider the impacts of both the set of flux wires selected, and the counting process. An expanded set of flux wires is identified that provides a more comprehensive data set on the fast spectrum. To address the counting process itself, a series of round-robin measurements in several reactor metrology laboratories across the Department of Energy (DOE) complex and nuclear industry are being undertaken to refine the American Society for Testing and Materials (ASTM) standards for flux wire measurements. To address the contribution of uncertainty in the MCNP model predictions, an uncertainty quantification (UQ) tool has been developed that statistically samples the model input parameters, runs a series of cases, and assimilates the results to provide an overall uncertainty. The impact of the MCNP UQ tool results is demonstrated by re-analyzing previous AGC flux wire and irradiation data. While the expanded flux wire set obviously cannot be added to these experiments retroactively, plans for future graphite irradiations are outlined.

11 NUCLEAR FUEL CYCLE AND FUEL MATERIALS↗
DOE OSTI2025

Examples of Mission-driven Data Science from Jefferson Lab and ACES

This presentation details mission-driven data science initiatives at Jefferson Lab and the Joint Institute for Advanced Computing on Environmental Studies (ACES). JLab, a U.S. Department of Energy Office of Science national laboratory, operates the Continuous Electron Beam Accelerator Facility (CEBAF), and is the lead institute for the new High Performance Data Facility (HPDF) Hub. The Joint Institute for ACES brings together interdisciplinary teams in health informatics, climate modeling, computer science, and physics to address environmental challenges, including flood modeling. The Hampton Roads region, particularly Norfolk and Virginia Beach, faces increasing flood risks, motivating the need for rapid, reliable, and risk-aware decision support. ACES’s flooding work has a focus on uncertainty quantification (UQ) and machine learning (ML) for coastal flood management. The work is motivated by the increasing vulnerability of communities such as Norfolk and Virginia Beach, Virginia, to frequent coastal flooding events, and the need for rapid, reliable decision support. The research develops computationally efficient ML surrogate models to forecast water levels and flooding risk. A central theme is the quantification and calibration of predictive uncertainty, especially for out-of-distribution (OOD) scenarios, using techniques such as Monte Carlo Dropout, Deep Ensembles, Gaussian Processes, and Deep Quantile Regression (DQR). The study demonstrates that distance-aware UQ is critical for reliable scientific AI, particularly in high-dimensional, safety-critical, and real-time applications.

McSpadden, Diana [Thomas Jefferson National Accele↗
DOE OSTI2025

Bayesian Optimized Deep Ensemble for Uncertainty Quantification of Deep Neural Networks: a System Safety Case Study on Sodium Fast Reactor Thermal Stratification Modeling

Deep neural networks (DNNs) are increasingly important to scientific computing and engineering system simulations. Accurate uncertainty quantification (UQ) for DNNs is critical in safety-sensitive engineering domains. Traditional Deep Ensemble (DE) methods, while easy to implement, frequently suffer from poorly calibrated uncertainty estimates and limited predictive accuracy due to reliance on fixed architectures with varied weight initializations. To address these issues, we introduce a workflow that combines Bayesian Optimization (BO) and DE. The workflow is modular, scalable, and integrates parallel BO initialized with Sobol sequences to individually optimize the hyperparameters of each ensemble member. This method enhances ensemble diversity, improves predictive accuracy, and provides reliable uncertainty estimates. We evaluate the proposed BODE approach in a sodium fast reactor thermal stratification modeling case study, where we used a densely connected convolutional neural network to predict turbulent viscosity during the reactor transient with consideration of data noise. We benchmark its performance against several optimization approaches, including baseline deep ensemble, evolutionary algorithm-optimized ensemble, ensemble formed via random search combined with greedy selection, and a BO ensemble using random initialization. Here, our results demonstrate superior performance of the developed BODE approach. In noise-free scenarios, BODE notably reduces incorrect aleatoric uncertainty and significantly enhances predictive accuracy. Under conditions of 5% and 10% Gaussian noise, BODE adaptively quantifies uncertainty proportional to data noise, achieving up to an 80% reduction in root mean square error compared to baseline methods and producing well-calibrated prediction intervals.

Bayesian optimization↗
DOE OSTI2023

Uncertainty Quantification of Capacitor Switching Transient Location using Machine Learning

Identification of capacitor switching transient location provides valuable insight into the state of the associated equipment. Machine learning (ML) models, and convolutional neural networks (CNNs) in particular, have demonstrated remarkable performance in signal location. However, ML models are data driven whose predictions are affected by noise in data and may also suffer from large extrapolation errors when applied to new conditions. Uncertainty quantification (UQ) is necessary to ensure model trustworthiness and avoid overconfident predictions in extrapolation. Here, in this work, we propose a novel UQ method, called PI3NN, to quantify prediction uncertainty of ML models and integrate the method with CNNs for transient source location. PI3NN calculates Prediction Intervals by training 3 Neural Networks and uses root-finding methods to determine the interval precisely. Additionally, PI3NN can identify out-of-distribution (OOD) data in a nonstationary condition to avoid overconfident prediction. Results indicate that with PI3NN, transient signals are not only correctly identified, but when said signals are subject to corruptions characteristic of an actual power monitoring system (e.g. non-ideal sensors), the model recognizes when it is uncertain about its predictions, effectively letting the user know when to accept or discard the results.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2025

pnnl/SNAP

In this work, we detail two uncertainty quantification (UQ) methods that provide complementary information. Readout ensembling, by finetuning only the readout layers of an ensemble of foundation models, provides information about model uncertainty. Amending the final readout layer to predict upper and lower quantiles replaces point predictions with distributional predictions, which provide information about uncertainty within the underlying training data. We demonstrate our approach with the MACE-MP-0 model, applying UQ to both the foundation model and a series of finetuned models. The uncertainties produced by the ensemble and quantile methods are demonstrated to be distinct measures by which the quality of the NNP output can be judged.

Pope, Jenna (Bilbrey) [Pacific Northwest National ↗
DOE OSTI2020

Conservative Estimation of Tail Probabilities from Limited Sample Data

Several sparse-sample uncertainty quantification (UQ) methods are compared for conservative but not overly conservative estimation of small tail probabilities involving responses that lay beyond specified thresholds in the tails of probability distributions. Sixteen very differently shaped distributions (or probability density functions, PDFs) and tail probability magnitudes ranging from 10 -5 to 10 -1 are considered in order for the study to be relevant to a wide range of risk analysis and quantification of margins and uncertainty (QMU) problems. The emphasis of the study is on limited data regimes ranging from N = 2 to 20 samples, reflective of most experimental and some expensive computational situations. Relatively simple sparse-sample UQ methods tested for this regime involve statistical tolerance interval "Equivalent Normal and related "Ensemble of Normals" and "Superdistribution (SD) approaches. (The independently derived SD is effectively equivalent to the Bayesian posterior predictive distribution given the assumptions of the derivation.) The performance of the methods was generally improved for N ≥ 5 samples with a generalized Jackknife resampling technique, which determines a tail probability estimate by averaging estimates from smaller sub-samples. Several quantitative metrics for method conservatism and accuracy of tail probability estimation are used to assess and rank the methods' performance over many random trials for each test PDF and probability magnitude. A variant of Bootstrap resampling was also tried, but did not significantly improve tail probability estimates in most cases. Detailed results are presented from over 100-million tests over the above factors that provide useful granular information on which methods or combination of methods perform best in various areas of the factor space.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2023

Multi-fidelity Uncertainty Quantification for Homogenization Problems in Structure-Property Relationships from Crystal Plasticity Finite Elements

Crystal plasticity finite element method (CPFEM) has been an integrated computational materials engineering (ICME) workhorse to study materials behaviors and structure-property relationships for the last few decades. These relations are mappings from the microstructure space to the materials properties space. Due to the stochastic and random nature of microstructures, there is always some uncertainty associated with materials properties, for example, in homogenized stress-strain curves. For critical applications with strong reliability needs, it is often desirable to quantify the microstructure-induced uncertainty in the context of structure-property relationships. However, this uncertainty quantification (UQ) problem often incurs a large computational cost because many statistically equivalent representative volume elements (SERVEs) are needed. In this article, we apply a multi-level Monte Carlo (MLMC) method to CPFEM to study the uncertainty in stress-strain curves, given an ensemble of SERVEs at multiple mesh resolutions. By using the information at coarse meshes, we show that it is possible to approximate the response at fine meshes with a much reduced computational cost. We focus on problems where the model output is multi-dimensional, which requires us to track multiple quantities of interest (QoIs) at the same time. In conclusion, our numerical results show that MLMC can accelerate UQ tasks around 2.23x, compared to the classical Monte Carlo (MC) method, which is widely known as ensemble average in the CPFEM literature.

36 MATERIALS SCIENCE↗
DOE OSTI2025

Data Assimilation for Robust UQ Within Agent-Based Simulation on HPC Systems

Agent-based simulation provides a powerful tool for in silico system modeling. However, these simulations do not provide built-in methods for uncertainty quantification (UQ). Within these types of models a typical approach to UQ is to run multiple realizations of the model then compute aggregate statistics. This approach is limited due to the compute time required for a solution. When faced with an emerging biothreat, public health decisions need to be made quickly and solutions for integrating near real-time data with analytic tools are needed. We propose an integrated Bayesian UQ framework for agent-based models based on sequential Monte Carlo sampling. Given streaming or static data about the evolution of an emerging pathogen this Bayesian framework provides a distribution over the parameters governing the spread of a disease through a population. These estimates of the spread of a disease may be provided to public health agencies seeking to abate the spread. By coupling agent-based simulations with Bayesian modeling in a data assimilation, our proposed framework provides a powerful tool for modeling dynamical systems in silico. We propose a method which reduces model error and provides a range of realistic possible outcomes. Moreover, our method addresses two primary limitations of ABMs: the lack of UQ and an inability to assimilate data. Our proposed framework combines the flexibility of an agent-based model with UQ provided by the Bayesian paradigm in a workflow which scales well to HPC systems. We provide algorithmic details and results on a simulated outbreak with both static and streaming data.

Spannaus, Adam [ORNL] (ORCID:0000000225213657)↗