Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Uncertainty”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 37 records · Page 2
All sourcesNASA NTRSDOE OSTI
NASA NTRS2014

Numerical Uncertainty Analysis for Computational Fluid Dynamics using Student T Distribution -- Application of CFD Uncertainty Analysis Compared to Exact Analytical Solution

Computational Fluid Dynamics (CFD) is the standard numerical tool used by Fluid Dynamists to estimate solutions to many problems in academia, government, and industry. CFD is known to have errors and uncertainties and there is no universally adopted method to estimate such quantities. This paper describes an approach to estimate CFD uncertainties strictly numerically using inputs and the Student-T distribution. The approach is compared to an exact analytical solution of fully developed, laminar flow between infinite, stationary plates. It is shown that treating all CFD input parameters as oscillatory uncertainty terms coupled with the Student-T distribution can encompass the exact solution.

Groves, Curtis E.↗
NASA NTRS1976

The uncertainty threshold principle - Fundamental limitations of optimal decision making under dynamic uncertainty

The fundamental limitations of the optimal control of dynamic systems with random parameters are analyzed by studying a scalar linear-quadratic optimal control example. It is demonstrated that optimum long-range decision making is possible only if the dynamic uncertainty (quantified by the means and covariances of the random parameters) is below a certain threshold. If this threshold is exceeded, there do not exist optimum decision rules. This phenomenon is called the 'uncertainty threshold principle'. The implications of this phenomenon to the field of modelling, identification, and adaptive control are discussed.

Athans, M.↗
NASA NTRS1977

The uncertainty threshold principle - Some fundamental limitations of optimal decision making under dynamic uncertainty

This note shows that the optimal control of dynamic systems with uncertain parameters has certain limitations. In particular, by means of a simple scalar linear-quadratic optimal control example, it is shown that the infinite horizon solution does not exist if the parameter uncertainty exceeds a certain quantifiable threshold; we call this the uncertainty threshold principle. The philosophical and design implications of this result are discussed.

Athans, M.↗
NASA NTRS1991

Measuring uncertainty by extracting fuzzy rules using rough sets and extracting fuzzy rules under uncertainty and measuring definability using rough sets

Computers are not designed to handle terms where uncertainty is present. To deal with uncertainty, techniques other than classical logic must be developed. This paper examines the concepts of statistical analysis, the Dempster-Shafer theory, rough set theory, and fuzzy set theory to solve this problem. The fundamentals of these theories are combined to provide the possible optimal solution. By incorporating principles from these theories, a decision-making process may be simulated by extracting two sets of fuzzy rules: certain rules and possible rules. From these rules a corresponding measure of how much we believe these rules is constructed. From this, the idea of how much a fuzzy diagnosis is definable in terms of its fuzzy attributes is studied.

Worm, Jeffrey A.↗
DOE OSTI2026

Temperature profiling at the American WAKE ExperimeNt (AWAKEN): methodology and uncertainty quantification

We quantify the accuracy of the temperature profiling from ground-based spectral infrared radiance observations at the American WAKE ExperimeNt (AWAKEN). Results from pre-campaign tests and comparisons with in-situ ground-based and airborne sensors at AWAKEN indicate that temperature profiles agree satisfactorily with traditional instruments for wind energy applications. The bias is within a fraction of a degree and appears to be related to atmospheric stability. Root-mean-square differences from the reference instruments are always smaller than a degree and are often well described by the online uncertainty estimation product. Height-to-height and site-to-site temperature differences are in excellent agreement with in-situ observations, which justifies the use of temperature profilers to characterize static stability and spatial gradients of temperature.

17 WIND ENERGY↗
NASA NTRS2011

MODIS Cloud Optical Property Retrieval Uncertainties Derived from Pixel-Level VNIR/SWIR Radiometric Uncertainties

Moderate Resolution Imaging Spectroradiometer (MODIS) retrievals of optical thickness and effective particle radius for liquid water and ice phase clouds employ a well-known VNIR/ SWIR solar reflectance technique. For this type of algorithm, we evaluate the quantitative uncertainty in simultaneous retrievals of these two cloud parameters to pixel-level radiometric calibration estimates and other fundamental (and tractable) error sources.

Platnick, S.↗
NASA NTRS2011

Assessment of Radiative Heating Uncertainty for Hyperbolic Earth Entry

This paper investigates the shock-layer radiative heating uncertainty for hyperbolic Earth entry, with the main focus being a Mars return. In Part I of this work, a baseline simulation approach involving the LAURA Navier-Stokes code with coupled ablation and radiation is presented, with the HARA radiation code being used for the radiation predictions. Flight cases representative of peak-heating Mars or asteroid return are de ned and the strong influence of coupled ablation and radiation on their aerothermodynamic environments are shown. Structural uncertainties inherent in the baseline simulations are identified, with turbulence modeling, precursor absorption, grid convergence, and radiation transport uncertainties combining for a +34% and ..24% structural uncertainty on the radiative heating. A parametric uncertainty analysis, which assumes interval uncertainties, is presented. This analysis accounts for uncertainties in the radiation models as well as heat of formation uncertainties in the flow field model. Discussions and references are provided to support the uncertainty range chosen for each parameter. A parametric uncertainty of +47.3% and -28.3% is computed for the stagnation-point radiative heating for the 15 km/s Mars-return case. A breakdown of the largest individual uncertainty contributors is presented, which includes C3 Swings cross-section, photoionization edge shift, and Opacity Project atomic lines. Combining the structural and parametric uncertainty components results in a total uncertainty of +81.3% and ..52.3% for the Mars-return case. In Part II, the computational technique and uncertainty analysis presented in Part I are applied to 1960s era shock-tube and constricted-arc experimental cases. It is shown that experiments contain shock layer temperatures and radiative ux values relevant to the Mars-return cases of present interest. Comparisons between the predictions and measurements, accounting for the uncertainty in both, are made for a range of experiments. A measure of comparison quality is de ned, which consists of the percent overlap of the predicted uncertainty bar with the corresponding measurement uncertainty bar. For nearly all cases, this percent overlap is greater than zero, and for most of the higher temperature cases (T >13,000 K) it is greater than 50%. These favorable comparisons provide evidence that the baseline computational technique and uncertainty analysis presented in Part I are adequate for Mars-return simulations. In Part III, the computational technique and uncertainty analysis presented in Part I are applied to EAST shock-tube cases. These experimental cases contain wavelength dependent intensity measurements in a wavelength range that covers 60% of the radiative intensity for the 11 km/s, 5 m radius flight case studied in Part I. Comparisons between the predictions and EAST measurements are made for a range of experiments. The uncertainty analysis presented in Part I is applied to each prediction, and comparisons are made using the metrics defined in Part II. The agreement between predictions and measurements is excellent for velocities greater than 10.5 km/s. Both the wavelength dependent and wavelength integrated intensities agree within 30% for nearly all cases considered. This agreement provides confidence in the computational technique and uncertainty analysis presented in Part I, and provides further evidence that this approach is adequate for Mars-return simulations. Part IV of this paper reviews existing experimental data that include the influence of massive ablation on radiative heating. It is concluded that this existing data is not sufficient for the present uncertainty analysis. Experiments to capture the influence of massive ablation on radiation are suggested as future work, along with further studies of the radiative precursor and improvements in the radiation properties of ablation products.

Johnston, Christopher O.↗
DOE OSTI2024

Evidential Deep Learning: Enhancing Predictive Uncertainty Estimation for Earth System Science Applications

Abstract Robust quantification of predictive uncertainty is a critical addition needed for machine learning applied to weather and climate problems to improve the understanding of what is driving prediction sensitivity. Ensembles of machine learning models provide predictive uncertainty estimates in a conceptually simple way but require multiple models for training and prediction, increasing computational cost and latency. Parametric deep learning can estimate uncertainty with one model by predicting the parameters of a probability distribution but does not account for epistemic uncertainty. Evidential deep learning, a technique that extends parametric deep learning to higher-order distributions, can account for both aleatoric and epistemic uncertainties with one model. This study compares the uncertainty derived from evidential neural networks to that obtained from ensembles. Through applications of the classification of winter precipitation type and regression of surface-layer fluxes, we show evidential deep learning models attaining predictive accuracy rivaling standard methods while robustly quantifying both sources of uncertainty. We evaluate the uncertainty in terms of how well the predictions are calibrated and how well the uncertainty correlates with prediction error. Analyses of uncertainty in the context of the inputs reveal sensitivities to underlying meteorological processes, facilitating interpretation of the models. The conceptual simplicity, interpretability, and computational efficiency of evidential neural networks make them highly extensible, offering a promising approach for reliable and practical uncertainty quantification in Earth system science modeling. To encourage broader adoption of evidential deep learning, we have developed a new Python package, Machine Integration and Learning for Earth Systems (MILES) group Generalized Uncertainty for Earth System Science (GUESS) (MILES-GUESS) ( https://github.com/ai2es/miles-guess ), that enables users to train and evaluate both evidential and ensemble deep learning. Significance Statement This study demonstrates a new technique, evidential deep learning, for robust and computationally efficient uncertainty quantification in modeling the Earth system. The method integrates probabilistic principles into deep neural networks, enabling the estimation of both aleatoric uncertainty from noisy data and epistemic uncertainty from model limitations using a single model. Our analyses reveal how decomposing these uncertainties provides valuable insights into reliability, accuracy, and model shortcomings. We show that the approach can rival standard methods in classification and regression tasks within atmospheric science while offering practical advantages such as computational efficiency. With further advances, evidential networks have the potential to enhance risk assessment and decision-making across meteorology by improving uncertainty quantification, a longstanding challenge. This work establishes a strong foundation and motivation for the broader adoption of evidential learning, where properly quantifying uncertainties is critical yet lacking.

Schreck, John S.↗
NASA NTRSDate not supplied

SLS Integrated Modal Test Uncertainty Quantification using the Hybrid Parametric Variation Method

Uncertainty in structural loading during launch is a significant concern in the development of spacecraft and launch vehicles. Small variations in launch vehicle and payload mode shapes and their interaction can result in significant variation in system loads. In many cases involving large aerospace systems it is difficult, not economical, or impossible to perform a system modal test. However, it is still vital to obtain test results that can be compared with analytical predictions to validate models. Instead, the “Building Block Approach” is used in which system components are tested individually. Component models are correlated and updated to agree as best they can with test results. The Space Launch System consists of a number of components that are assembled into a launch vehicle. Finite element models of the components are developed, reduced to Hurty/Craig-Bampton models and assembled to represent different phases of flight. The only opportunity to obtain modal test data from an assembled Space Launch System will be during the Integrated Modal Test. There is always uncertainty in every model, which flows into uncertainty in predicted system results. Uncertainty Quantification is used to determine statistical bounds on prediction accuracy based on model uncertainty. For the Space Launch System, model uncertainty is at the Hurty/Craig-Bampton component level. Uncertainty in the Hurty/Craig-Bampton components is quantified using the hybrid parametric variation approach that combines parametric and nonparametric uncertainty. Uncertainty in model form is one of the biggest contributors to uncertainty in complex built-up structures. This type of uncertainty cannot be represented by variations infinite element model input parameters and thus cannot be included in a parametric approach. However, model-form uncertainty can be modeled using a nonparametric approach based on random matrix theory. The hybrid parametric variation method requires the selection of dispersion values for the Hurty/Craig-Bampton fixed-interface eigenvalues, and the Hurty/Craig-Bampton stiffness matrices. Component test/analysis frequency error is used to identify the fixed-interface eigenvalue dispersions, while test/analysis cross-orthogonality is used to identify stiffness dispersion values. The hybrid parametric variation uncertainty quantification approach is applied to the Space Launch System Integrated Modal Test configuration. Monte Carlo analysis is performed, and statistics are determined for modal correlation metrics, frequency response from Integrated Modal Test shakers to selected accelerometers, as well as other metrics for determining how well target modes are excited and identified. If the predicted uncertainty envelopes future Integrated Modal Test results, then there will be increased confidence in the utility of the component-based hybrid parametric variation uncertainty quantification approach.

Uncertainty Quantification↗
NASA NTRS1996

Uncertainty Modeling for Structural Control Analysis and Synthesis

The development of an accurate model of uncertainties for the control of structures that undergo a change in operational environment, based solely on modeling and experimentation in the original environment is studied. The application used throughout this work is the development of an on-orbit uncertainty model based on ground modeling and experimentation. A ground based uncertainty model consisting of mean errors and bounds on critical structural parameters is developed. The uncertainty model is created using multiple data sets to observe all relevant uncertainties in the system. The Discrete Extended Kalman Filter is used as an identification/parameter estimation method for each data set, in addition to providing a covariance matrix which aids in the development of the uncertainty model. Once ground based modal uncertainties have been developed, they are localized to specific degrees of freedom in the form of mass and stiffness uncertainties. Two techniques are presented: a matrix method which develops the mass and stiffness uncertainties in a mathematical manner; and a sensitivity method which assumes a form for the mass and stiffness uncertainties in macroelements and scaling factors. This form allows the derivation of mass and stiffness uncertainties in a more physical manner. The mass and stiffness uncertainties of the ground based system are then mapped onto the on-orbit system, and projected to create an analogous on-orbit uncertainty model in the form of mean errors and bounds on critical parameters. The Middeck Active Control Experiment is introduced as experimental verification for the localization and projection methods developed. In addition, closed loop results from on-orbit operations of the experiment verify the use of the uncertainty model for control analysis and synthesis in space.

Campbell, Mark E.↗
NASA NTRS2022

Estimating pixel-level uncertainty in ocean color retrievals from MODIS

The spectral distribution of marine remote sensing reflectance, R(rs), is the fundamental measurement of ocean color science, from which a host of bio-optical and biogeochemical properties of the water column can be derived. Estimation of uncertainty in these derived properties is thus dependent on knowledge of the uncertainty in satellite-retrieved R(rs) (u(c)(R(rs))) at each pixel. Uncertainty in R(rs), in turn, is dependent on the propagation of various uncertainty sources through the R(rs) retrieval process, namely the atmospheric correction (AC). A derivative-based method for uncertainty propagation is established here to calculate the pixel-level uncertainty in R(rs), as retrieved using NASA’s multiple-scattering epsilon (MSEPS) AC algorithm and verified using Monte Carlo (MC) analysis. The approach is then applied to measurements from the Moderate Resolution Imaging Spectroradiometer (MODIS) on the Aqua satellite, with uncertainty sources including instrument random noise, instrument systematic uncertainty, and forward model uncertainty. The uc(Rrs) is verified by comparison with statistical analysis of coincident retrievals from MODIS and in situ Rrs measurements, and our approach performs well in most cases. Based on analysis of an example 8-day global products, we also show that relative uncertainty in R(rs) at blue bands has a similar spatial pattern to the derived concentration of the phytoplankton pigment chlorophyll-a (chl-a), and around 7.3%, 17.0%, and 35.2% of all clear water pixels (chl-a ≤ 0.1 mg/cu.m) with valid u(c)(R(rs)) have a relative uncertainty ≤ 5% at bands 412 nm, 443 nm, and 488 nm respectively, which is a common goal of ocean color retrievals for clear waters. While the analysis shows that u(c)(R(rs)) calculated from our derivative-based method is reasonable, some issues need further investigation, including improved knowledge of forward model uncertainty and systematic uncertainty in instrument calibration.

Pixel-level uncertainty↗
NASA NTRS2023

Uncertainty Analysis of the NASA Glenn 8- by 6-Foot Supersonic Wind Tunnel 2019 Characterization Test 14-Foot, 5.8 Percent Test Section Porosity

This paper presents methods and results of a measurement uncertainty analysis that was performed for the 8- by 6-foot Supersonic Wind Tunnel located at the NASA Glenn Research Center. The uncertainty analysis is based on data collected during a characterization test following major structural modifications to the facility and upgrades to the facility’s data and control systems. The statistical methods and engineering judgments used to estimate elemental uncertainties are described in this report. The Monte Carlo method of propagating uncertainty was selected to determine the uncertainty of calculated variables of interest. A description of the Monte Carlo method as applied for this analysis is provided. The primary variable of interest (VOI) for this facility is free stream Mach number. In addition to determining the uncertainty in Mach number, the uncertainty in free stream values of static pressure, total pressure, dynamic pressure, total temperature, static temperature, Reynolds number, and air speed were also calculated. Uncertainty results are presented as random (unpredictable variation in repeated measurements), systematic (potential offset between observed and true value), and total (random and systematic combined) uncertainty for Mach number. Systematic uncertainty results are presented for the other VOIs. Individual uncertainty sources are presented both dimensionally and as percent contributions to uncertainty in all VOIs, to aid in the identification of primary uncertainty sources.

Uncertainty Analysis↗
NASA NTRS2016

Characterizing Epistemic Uncertainty for Launch Vehicle Designs

NASA Probabilistic Risk Assessment (PRA) has the task of estimating the aleatory (randomness) and epistemic (lack of knowledge) uncertainty of launch vehicle loss of mission and crew risk, and communicating the results. Launch vehicles are complex engineered systems designed with sophisticated subsystems that are built to work together to accomplish mission success. Some of these systems or subsystems are in the form of heritage equipment, while some have never been previously launched. For these cases, characterizing the epistemic uncertainty is of foremost importance, and it is anticipated that the epistemic uncertainty of a modified launch vehicle design versus a design of well understood heritage equipment would be greater. For reasons that will be discussed, standard uncertainty propagation methods using Monte Carlo simulation produce counter intuitive results, and significantly underestimate epistemic uncertainty for launch vehicle models. Furthermore, standard PRA methods, such as Uncertainty-Importance analyses used to identify components that are significant contributors to uncertainty, are rendered obsolete, since sensitivity to uncertainty changes are not reflected in propagation of uncertainty using Monte Carlo methods. This paper provides a basis of the uncertainty underestimation for complex systems and especially, due to nuances of launch vehicle logic, for launch vehicles. It then suggests several alternative methods for estimating uncertainty and provides examples of estimation results. Lastly, the paper describes how to implement an Uncertainty-Importance analysis using one alternative approach, describes the results, and suggests ways to reduce epistemic uncertainty by focusing on additional data or testing of selected components.

Novack, Steven D.↗
NASA NTRS2016

Characterizing Epistemic Uncertainty for Launch Vehicle Designs

NASA Probabilistic Risk Assessment (PRA) has the task of estimating the aleatory (randomness) and epistemic (lack of knowledge) uncertainty of launch vehicle loss of mission and crew risk and communicating the results. Launch vehicles are complex engineered systems designed with sophisticated subsystems that are built to work together to accomplish mission success. Some of these systems or subsystems are in the form of heritage equipment, while some have never been previously launched. For these cases, characterizing the epistemic uncertainty is of foremost importance, and it is anticipated that the epistemic uncertainty of a modified launch vehicle design versus a design of well understood heritage equipment would be greater. For reasons that will be discussed, standard uncertainty propagation methods using Monte Carlo simulation produce counter intuitive results and significantly underestimate epistemic uncertainty for launch vehicle models. Furthermore, standard PRA methods such as Uncertainty-Importance analyses used to identify components that are significant contributors to uncertainty are rendered obsolete since sensitivity to uncertainty changes are not reflected in propagation of uncertainty using Monte Carlo methods.This paper provides a basis of the uncertainty underestimation for complex systems and especially, due to nuances of launch vehicle logic, for launch vehicles. It then suggests several alternative methods for estimating uncertainty and provides examples of estimation results. Lastly, the paper shows how to implement an Uncertainty-Importance analysis using one alternative approach, describes the results, and suggests ways to reduce epistemic uncertainty by focusing on additional data or testing of selected components.

Novack, Steven D.↗
DOE OSTI2024

DeepUQ: Assessing the Aleatoric Uncertainties from two Deep Learning Methods

Assessing the quality of aleatoric uncertainty estimates from uncertainty quantification (UQ) deep learning methods is important in scientific contexts, where uncertainty is physically meaningful and important to characterize and interpret exactly. We systematically compare aleatoric uncertainty measured by two UQ techniques, Deep Ensembles (DE) and Deep Evidential Regression (DER). Our method focuses on both zero-dimensional (0D) and two-dimensional (2D) data, to explore how the UQ methods function for different data dimensionalities. We investigate uncertainty injected on the input and output variables and include a method to propagate uncertainty in the case of input uncertainty so that we can compare the predicted aleatoric uncertainty to the known values. We experiment with three levels of noise. The aleatoric uncertainty predicted across all models and experiments scales with the injected noise level. However, the predicted uncertainty is miscalibrated to $\rm{std}(\sigma_{\rm al})$ with the true uncertainty for half of the DE experiments and almost all of the DER experiments. The predicted uncertainty is the least accurate for both UQ methods for the 2D input uncertainty experiment and the high-noise level. While these results do not apply to more complex data, they highlight that further research on post-facto calibration for these methods would be beneficial, particularly for high-noise and high-dimensional settings.

Nevin, Rebecca↗