Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “uncertainty”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 685 records · Page 38

Hierarchical Bayesian modeling for Inverse Uncertainty Quantification of system thermal-hydraulics code using critical flow experimental data

The best estimate plus uncertainty methodology in nuclear system thermal-hydraulic studies necessitates a comprehensive understanding of uncertainties in system code predictions. The forward uncertainty quantification (UQ) process involves the propagation of input uncertainties through the computational models to obtain uncertainties in the outputs. To this end, achieving an accurate estimation of input uncertainties is important, which is the focus of inverse UQ (IUQ). Traditionally, research in Bayesian IUQ within the nuclear engineering domain has largely relied on single-level Bayesian inference. While being effective for relatively small datasets, this approach encounters limitations for cases with large datasets. The use of a single-level model may prove inefficient, as the resultant posterior distributions can significantly differ when distinct subsets of data are employed. To address this issue, we employ an hierarchical Bayesian model for IUQ. Furthermore, this approach involves organizing observations into different groups based on the test conditions, thereby accommodating varying calibration parameters across these distinct groups. In this study, we developed and implemented a hierarchical Bayesian IUQ method to consider the grouping effect of critical flow measurement data from various geometries. Comparing the outcomes of IUQ under different selections of test data using hierarchical Bayesian IUQ against those obtained from single-level Bayesian IUQ, the forward propagation of hierarchical Bayesian IUQ results demonstrates a notably improved agreement with the experimental data.

42 ENGINEERING↗

Uncertainty quantification in scientific machine learning: Methods, metrics, and comparisons

Neural networks (NNs) are currently changing the computational paradigm on how to combine data with mathematical laws in physics and engineering in a profound way, tackling challenging inverse and ill-posed problems not solvable with traditional methods. However, quantifying errors and uncertainties in NN-based inference is more complicated than in traditional methods. This is because in addition to aleatoric uncertainty associated with noisy data, there is also uncertainty due to limited data, but also due to NN hyperparameters, overparametrization, optimization and sampling errors as well as model misspecification. Although there are some recent works on uncertainty quantification (UQ) in NNs, there is no systematic investigation of suitable methods towards quantifying the total uncertainty effectively and efficiently even for function approximation, and there is even less work on solving partial differential equations and learning operator mappings between infinite-dimensional function spaces using NNs. In this work, we present a comprehensive framework that includes uncertainty modeling, new and existing solution methods, as well as evaluation metrics and post-hoc improvement approaches. Further, to demonstrate the applicability and reliability of our framework, we present an extensive comparative study in which various methods are tested on prototype problems, including problems with mixed input-output data, and stochastic problems in high dimensions. In the Appendix, we include a comprehensive description of all the UQ methods employed. Further, to help facilitate the deployment of UQ in Scientific Machine Learning research and practice, we present and develop in [1] an open-source Python library (github.com/Crunch-UQ4MI/neuraluq), termed NeuralUQ, that is accompanied by an educational tutorial and additional computational experiments.

11 physics-informed neural networks↗

Active operator learning with predictive uncertainty quantification for partial differential equations

With the increased prevalence of neural operators being used to provide rapid solutions to partial differential equations (PDEs), understanding the accuracy of model predictions and the associated error levels is necessary for deploying reliable surrogate models in scientific applications. Existing uncertainty quantification (UQ) frameworks employ ensembles or Bayesian methods, which can incur substantial computational costs during both training and inference. Here, we propose a lightweight predictive UQ method tailored for Deep operator networks (DeepONets) that also generalizes to other operator networks. Numerical experiments on linear and nonlinear PDEs demonstrate that the framework’s uncertainty estimates are unbiased and provide accurate out-of-distribution uncertainty predictions with a sufficiently large training dataset. Our framework provides fast inference and uncertainty estimates that can efficiently drive outer-loop analyses that would be prohibitively expensive with conventional solvers. We demonstrate how predictive uncertainties can be used in the context of Bayesian optimization and active learning problems to yield improvements in accuracy and data-efficiency for outer-loop optimization procedures. In the active learning setup, we extend the framework to Fourier Neural Operators (FNO) and describe a generalized method for other operator networks. To enable real-time deployment, we introduce an inference strategy based on precomputed trunk outputs and a sparse placement matrix, reducing evaluation time by more than a factor of five. Our method provides a practical route to uncertainty-aware operator learning in time-sensitive settings.

97 MATHEMATICS AND COMPUTING↗

A robust Bayesian approach to modeling epistemic uncertainty in common-cause failure models

In a standard Bayesian approach to the alpha-factor model for common-cause failure, a precise Dirichlet prior distribution models epistemic uncertainty in the alpha-factors. This Dirichlet prior is then updated with observed data to obtain a posterior distribution, which forms the basis for further inferences. In this paper, we adapt the imprecise Dirichlet model of Walley to represent epistemic uncertainty in the alpha-factors. In this approach, epistemic uncertainty is expressed more cautiously via lower and upper expectations for each alpha-factor, along with a learning parameter which determines how quickly the model learns from observed data. For this application, we focus on elicitation of the learning parameter, and find that values in the range of 1 to 10 seem reasonable. The approach is compared with Kelly and Atwood's minimally informative Dirichlet prior for the alpha-factor model, which incorporated precise mean values for the alpha-factors, but which was otherwise quite diffuse. Next, we explore the use of a set of Gamma priors to model epistemic uncertainty in the marginal failure rate, expressed via a lower and upper expectation for this rate, again along with a learning parameter. As zero counts are generally less of an issue here, we find that the choice of this learning parameter is less crucial. Finally, we demonstrate how both epistemic uncertainty models can be combined to arrive at lower and upper expectations for all common-cause failure rates. Thereby, we effectively provide a full sensitivity analysis of common-cause failure rates, properly reflecting epistemic uncertainty of the analyst on all levels of the common-cause failure model.

99 GENERAL AND MISCELLANEOUS↗

Investigation of Ethane Dehydrogenation and Hydrogenolysis on Pt(111), Pt(211), and Pt(100): Bayesian Quantification and Correction of DFT-Based Enthalpic and Entropic Uncertainties

Computational investigations of heterogeneously catalyzed reactions using density functional theory (DFT) are often inaccurate, largely due to uncertainties in the choice of DFT functional (enthalpic uncertainty) and approximations for modeling adsorbate movement along the catalyst surface (entropic uncertainty). This work illustrates that both uncertainties are significant in the investigation of ethane dehydrogenation (EDH) and hydrogenolysis on Pt catalysts by considering the complete deconstruction of ethane on Pt(111), Pt(211), and Pt(100) using microkinetic modeling (MKM). Hence, this work uses both noncalibrated and Bayesian-calibrated MKMs to quantify and correct inaccuracies in macroscopic properties due to both uncertainties. A Bayesian approach to the correction of entropic errors was introduced using a “Modified Fermi Function (MFF)” to calibrate between the two bounds of entropy represented by the harmonic oscillator (HO) and free translator (FT) approximations. Regardless of enthalpic and entropic uncertainties, all three surfaces are capable of ethane activation; however, Pt(211) was found to be the most active and is largely responsible for methane production. Next, Pt(111) is largely responsible for acetylene production, and Pt(100) has the highest ethylene selectivity but is most susceptible to coking. By comparison of different calibrated models, the FT entropy approximation was found to better describe EDH under typical experimental conditions. Statistical evidence was found to support Pt(111) as the active site for EDH, assuming that one single site is responsible for the chemistry. On the three surfaces, competing second dehydrogenations to CH 2 CH 2 and CH 3 CH were observed as well as isomerization of CH 3 CH back to CH 2 CH 2 and deeper dehydrogenation of CH 3 CH. In conclusion, C–C cleavage was found to largely proceed via the CH 3 C intermediate on Pt(100) and Pt(111), while on Pt(211), it was via both CHC and CH 3 C.

Bayesian model selection↗

Coupled Lake‐Atmosphere‐Land Physics Uncertainties in a Great Lakes Regional Climate Model

Abstract This study develops a surrogate‐based method to assess the uncertainty within a convective permitting integrated modeling system of the Great Lakes region, arising from interacting physics parameterizations across the lake, atmosphere, and land surface. Perturbed physics ensembles of the model during the 2018 summer are used to train a neural network surrogate model to predict lake surface temperature (LST) and near‐surface air temperature (T2m). Average physics uncertainties are determined to be 1.5C for LST and T2m over land, and 1.9C for T2m over lake, but these have significant spatiotemporal variations. We find that atmospheric physics parameterizations alone are the dominant sources of uncertainty (45%–53%), while lake and land parameterizations account for 33% and 38% of the uncertainty of LST and T2m over land respectively. Interactions of atmosphere physics parameterizations with those of the land and lake contribute to an additional 13%–17% of the total variance. LST and T2m over the lake are more uncertain in the deeper northern lakes, particularly during the rapid warming phase that occurs in late spring/early summer. The LST uncertainty increases with sensitivity to the lake model's surface wind stress scheme. T2m over land is more uncertain over forested areas in the north, where it is most sensitive to the land surface model, than the more agricultural land in the south, where it is most sensitive to the atmospheric planetary boundary and surface layer scheme. Uncertainty also increases in the southwest during multiday temperature declines with higher sensitivity to the land surface model.

54 ENVIRONMENTAL SCIENCES↗

Uncertainties in Simulating Flooding During Hurricane Harvey Using 2D Shallow Water Equations

Abstract Flooding is one of the most impactful weather‐related natural hazards. Numerical models that solve the two dimensional (2D) shallow water equations (SWE) represent the first‐principles approach to simulate all types of spatial flooding, such as pluvial, fluvial, and coastal flooding, and their compound dynamics. High spatial resolution (e.g., () m) is needed in 2D SWE simulations to capture flood dynamics accurately, resulting in formidable computational challenges. Thus, relatively coarser spatial resolutions are used for large‐scale simulations of flooding, which introduce uncertainties in the results. It is unclear how the uncertainty associated with the model resolution compares to the uncertainties in precipitation data sets and assumptions regarding boundary conditions when channelized flows interact with other water bodies. In this study, we compare these three sources of uncertainties in 2D SWE simulations for the 2017 Houston flooding event. Our results show that precipitation uncertainty and mesh resolution have more significant impacts on the simulated streamflow and inundation dynamics than the choice of the downstream boundary condition at the watershed outlet. We point out the viability to confine the uncertainty of coarsening mesh resolution by using the variable resolution mesh (VRM) which refines critical topographic features with far fewer grid cells. Specifically, in simulations with VRM, the simulated inundation depths over the refined region are comparable to that use the finest uniform mesh. This study contributes to understanding the challenges and pathways for applying 2D SWE models to improve the realism of flood simulations over large scales.

54 ENVIRONMENTAL SCIENCES↗

Position-momentum uncertainty relations in the presence of quantum memory

A prominent formulation of the uncertainty principle identifies the fundamental quantum feature that no particle may be prepared with certain outcomes for both position and momentum measurements. Often the statistical uncertainties are thereby measured in terms of entropies providing a clear operational interpretation in information theory and cryptography. Recently, entropic uncertainty relations have been used to show that the uncertainty can be reduced in the presence of entanglement and to prove security of quantum cryptographic tasks. However, much of this recent progress has been focused on observables with only a finite number of outcomes not including Heisenberg’s original setting of position and momentum observables. Here, we show entropic uncertainty relations for general observables with discrete but infinite or continuous spectrum that take into account the power of an entangled observer. As an illustration, we evaluate the uncertainty relations for position and momentum measurements, which is operationally significant in that it implies security of a quantum key distribution scheme based on homodyne detection of squeezed Gaussian states.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

Particle Dark Matter constraints: the effect of Galactic uncertainties

Collider, space, and Earth based experiments are now able to probe several extensions of the Standard Model of particle physics which provide viable dark matter candidates. Direct and indirect dark matter searches rely on inputs of astrophysical nature, such as the local dark matter density or the shape of the dark matter density profile in the target in object. The determination of these quantities is highly affected by astrophysical uncertainties. The latter, especially those for our own Galaxy, are ill-known, and often not fully accounted for when analyzing the phenomenology of particle physics models. In this paper we present a systematic, quantitative estimate of how astrophysical uncertainties on Galactic quantities (such as the local galactocentric distance, circular velocity, or the morphology of the stellar disk and bulge) propagate to the determination of the phenomenology of particle physics models, thus eventually affecting the determination of new physics parameters. We present results in the context of two specific extensions of the Standard Model (the Singlet Scalar and the Inert Doublet) that we adopt as case studies for their simplicity in illustrating the magnitude and impact of such uncertainties on the parameter space of the particle physics model itself. Our findings point toward very relevant effects of current Galactic uncertainties on the determination of particle physics parameters, and urge a systematic estimate of such uncertainties in more complex scenarios, in order to achieve constraints on the determination of new physics that realistically include all known uncertainties.

79 ASTRONOMY AND ASTROPHYSICS↗

Uncertainty evaluation in reference material characterization by double isotope dilution inductively coupled plasma mass spectrometry (ID-ICP-MS)

Double isotope dilution inductively coupled plasma mass spectrometry (ID-ICP-MS) is one of the most important measurement methods in the establishment of measurement standards for inorganic analysis. However, there is still no good consensus on approaches for uncertainty evaluation in double ID-ICP-MS. Particularly, approaches to treating the instrumental isotopic fractionation (IIF) phenomenon and correlation between uncertainty sources are diverse and often unclear in the literature. Here, an approach for uncertainty evaluation in double ID-ICP-MS measurements is described. Using three certified reference materials (CRMs), it is shown that the IIF effects cancel out and should not be included in the uncertainty budget. Further, a consistent way of considering correlated sources of uncertainty to prevent double counting of uncertainties is described in detail along with an example of measuring the mass fraction of magnesium in an oyster powder CRM.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗

Robust and scalable uncertainty estimation with conformal prediction for machine-learned interatomic potentials

Abstract Uncertainty quantification (UQ) is important to machine learning (ML) force fields to assess the level of confidence during prediction, as ML models are not inherently physical and can therefore yield catastrophically incorrect predictions. Established a-posteriori UQ methods, including ensemble methods, the dropout method, the delta method, and various heuristic distance metrics, have limitations such as being computationally challenging for large models due to model re-training. In addition, the uncertainty estimates are often not rigorously calibrated. In this work, we propose combining the distribution-free UQ method, known as conformal prediction (CP), with the distances in the neural network’s latent space to estimate the uncertainty of energies predicted by neural network force fields. We evaluate this method (CP+latent) along with other UQ methods on two essential aspects, calibration, and sharpness, and find this method to be both calibrated and sharp under the assumption of independent and identically-distributed (i.i.d.) data. We show that the method is relatively insensitive to hyperparameters selected, and test the limitations of the method when the i.i.d. assumption is violated. Finally, we demonstrate that this method can be readily applied to trained neural network force fields with traditional and graph neural network architectures to obtain estimates of uncertainty with low computational costs on a training dataset of 1 million images to showcase its scalability and portability. Incorporating the CP method with latent distances offers a calibrated, sharp and efficient strategy to estimate the uncertainty of neural network force fields. In addition, the CP approach can also function as a promising strategy for calibrating uncertainty estimated by other approaches.

74 ATOMIC AND MOLECULAR PHYSICS↗

Assessing uncertainties in land cover projections

Understanding uncertainties in land cover projections is critical to investigating land-based climate mitigation policies, assessing the potential of climate adaptation strategies and quantifying the impacts of land cover change on the climate system. In this work, we identify and quantify uncertainties in global and European land cover projections over a diverse range of model types and scenarios, extending the analysis beyond the agro-economic models included in previous comparisons. The results from 75 simulations over 18 models are analysed and show a large range in land cover area projections, with the highest variability occurring in future cropland areas. We demonstrate systematic differences in land cover areas associated with the characteristics of the modelling approach, which is at least as great as the differences attributed to the scenario variations. The results lead us to conclude that a higher degree of uncertainty exists in land use projections than currently included in climate or earth system projections. To account for land use uncertainty, it is recommended to use a diverse set of models and approaches when assessing the potential impacts of land cover change on future climate. Additionally, further work is needed to better understand the assumptions driving land use model results and reveal the causes of uncertainty in more depth, to help reduce model uncertainty and improve the projections of land cover.

54 ENVIRONMENTAL SCIENCES↗

A Bayes network approach to uncertainty quantification in hierarchically developed computational models

Here, performance assessment of complex systems is ideally accomplished through system-level testing, but because they are expensive, such tests are seldom performed. On the other hand, for economic reasons, data from tests on individual components that are parts of complex systems are more readily available. The lack of system-level data leads to a need to build computational models of systems and use them for performance prediction in lieu of experiments. Because their complexity, models are sometimes built in a hierarchical manner, starting with simple components, progressing to collections of components, and finally, to the full system. Quantification of uncertainty in the predicted response of a system model is required in order to establish confidence in the representation of actual system behavior. This paper proposes a framework for the complex, but very practical problem of quantification of uncertainty in system-level model predictions. It is based on Bayes networks and uses the available data at multiple levels of complexity (i.e., components, subsystem, etc.). Because epistemic sources of uncertainty were shown to be secondary, in this application, aleatoric only uncertainty is included in the present uncertainty quantification. An example showing application of the techniques to uncertainty quantification of measures of response of a real, complex aerospace system is included.

97 MATHEMATICS AND COMPUTING↗

Towards Improved Understanding of the Applicability of Uncertainty Forecasts in the Electric Power Industry

Around the world wind energy is starting to become a major energy provider in electricity markets, as well as participating in ancillary services markets to help maintain grid stability. The reliability of system operations and smooth integration of wind energy into electricity markets has been strongly supported by years of improvement in weather and wind power forecasting systems. Deterministic forecasts are still predominant in utility practice although truly optimal decisions and risk hedging are only possible with the adoption of uncertainty forecasts. One of the main barriers for the industrial adoption of uncertainty forecasts is the lack of understanding of its information content (e.g., its physical and statistical modeling) and standardization of uncertainty forecast products, which frequently leads to mistrust towards uncertainty forecasts and their applicability in practice. Our paper aims at improving this understanding by establishing a common terminology and reviewing the methods to determine, estimate, and communicate the uncertainty in weather and wind power forecasts. This conceptual analysis of the state of the art highlights that: (i) end-users should start to look at the forecast's properties in order to map different uncertainty representations to specific wind energy-related user requirements; (ii) a multidisciplinary team is required to foster the integration of stochastic methods in the industry sector. Furthermore, a set of recommendations for standardization and improved training of operators are provided along with examples of best practices.

24 POWER TRANSMISSION AND DISTRIBUTION↗

Post-Calibration Uncertainty Analysis for Travel Times at a Naval Weapons Industrial Reserve Plant

The Naval Weapons Industrial Reserve Plant (NWIRP) in McGregor, Texas began manufacturing explosives in 1980 and several hazardous chemicals were discovered in lakes and streams surrounding the site in 1998. Contaminants traveled to local lakes and streams much faster than initially predicted. This research estimated contaminant travel times and identified locations where monitoring wells should be installed to yield the greatest reductions in uncertainties in travel-time predictions. To this end, groundwater and particle-tracking models for NWIRP site were built to predict hydraulic heads and contaminant travel times. Next, parameter (hydraulic conductivities) uncertainties, parameter identifiabilities, observation (hydraulic heads) worth, and predictive (contaminant travel times) uncertainties were quantified. Parameter uncertainties were reduced by up to 92%; a total of 19 of 158 parameters were at least moderately identifiable; travel-time uncertainties were reduced up to 92%. Additionally, travel-time predictions and post-calibration parameter distributions were generated using the null-space Monte Carlo (NSMC) technique. NSMC predicted that conservative tracers exited the flow system within a year, which matches with field data. Finally, an observations-worth analysis found that additional 11 more measurements would reduce travel-time uncertainties by factors from 1.04 to 4.3 over existing data if monitoring wells were installed at the suggested locations.

45 MILITARY TECHNOLOGY, WEAPONRY, AND NATIONAL DEF↗

Visualization and Decision Making Design Under Uncertainty

This report documents the program and the outcomes of Dagstuhl Seminar 22331 "Visualization and Decision Making Design Under Uncertainty". The seminar brought together 33 researchers and practitioners from different domains concerned with visualization and decision making under uncertainty including visualization, visual analytics, human-computer interaction, artificial intelligence, climate research, geography and geology. The programme was organized in two parts: In the first part which lasted two days, participants gave short talks where they discussed current practices and the uncertainty visualization challenges they encountered in their own research. At the end of day two, participants brainstormed collectively around the main uncertainty visualization research challenges across domains and applications. In the second part, participants voted for the following three main challenges they wished to discuss for the remainder of the seminar (one and a half days): applications, human-centered uncertainty visualization, a design process for uncertainty visualization. Thus three break-out groups were formed to discuss these challenges. Abstracts for the individual talks and the break-out group activities are included in this report.

decision making↗

Combined Error and Uncertainty Estimates for CFD Problems

Given input sources of uncertainty, non-intrusive uncertainty propagation methods quantify the uncertainty in output quantities of interest (QoI) by performing a nite number of CFD (Computational Fluid Dynamics) instance realizations needed in the calculation of output statistics. It is well known that this introduces multiple sources of error. CFD codes often utilize finite-dimensional approximation (grids, basis functions, etc.) thus incurring CFD numerical errors often approximately reinterpreted as a statistical bias. Uncertainty propagation methods calculate uncertainty statistics for output quantities of interest using a numerical method (e.g. deterministic quadrature, sampling, etc.) thus incurring UQ (Uncertainty Quantification) numerical errors. Importance of quantifying these errors in large scale scientific computing: How accurate is an output statistic?; How should additional computational resources be invested to further reduce the error in a statistic?

Posteriori↗

Determination of Uncertainties for Analytically Derived Material Properties to Be Used in Monte Carlo Based Orion Heatshield Sizing

Ablative materials are often used for spacecraft heatshields to protect underlying structures from the extreme environments associated with atmospheric reentry. NASA's Orion EM-1 capsule has been designed to use a molded Avcoat material system. In order to determine the required heatshield thickness, a Monte Carlo approach to the sizing process was proposed. To perform the Monte Carlo simulation, statistical uncertainties on all material property input parameters were required. Obtaining these values for measured properties is straightforward, however input parameters that are derived analytically have historically used uncertainties based on engineering judgment. A MATLAB program was created to use laboratory generated thermogravimetric analysis (TGA) data to calculate uncertainties on the Arrhenius parameters for molded Avcoat. Uncertainties associated with the normalized ablation rate and pyrolysis gas enthalpy were also generated using a wrapper script and the ACE code. These uncertainties could then be tied directly to measured values of individual elemental constituents. The resulting uncertainty values will allow for a probabilistic sizing approach on molded Avcoat with a higher level of confidence in the input parameters.

material property uncertainties↗