Results for “UQ” | Engineering Papers

Uncertainty quantification (UQ) is a rapidly growing and evolving discipline, especially within the aerospace community. Performing analysis with UQ can provide decision makers with a wealth of information about a candidate design. However, the value of UQ is fully realized when the information gained during UQ analysis is leveraged in a feedback loop of a design optimization process, often referred to as design under uncertainty. Although design under uncertainty can be a powerful risk mitigation technique, there are a number of roadblocks that prevent its implementation. Two primary factors are computational costs and added complexity of the analysis. High fidelity simulations on the order tens of uncertain variables quickly become computationally infeasible. Also, implementing UQ into an existing multidisciplinary design and optimization (MDO) process often requires extensive knowledge of the UQ methods and careful treatment of the problem formulation. The objective of this work is to address these two primary roadblocks and enable practitioners to efficiently perform design under uncertainty with limited knowledge of the UQ discipline. Methods outlined in this paper demonstrate MDO incorporating UQ into the design process, leveraging an analytic derivative tool chain through the entire optimization. The proposed approach leverages machine learning techniques to generate a differentiable confidence interval output from polynomial chaos models. This technique, coupled with the incorporation of analytical derivatives through the Polynomial Chaos Expansion (PCE) process, eliminates the need to estimate derivatives which are usually obtained from finite difference, complex step, or similar methods. Developing a differentiable confidence interval allows mixed uncertainty problems (both epistemic and aleatory) to be modeled. Without such modeling, these problems cannot accurately predict objective functions containing statistical quantities such as mean and variance. The addition of analytic derivatives to a polynomial chaos-based UQ method decreases the computational costs of performing design under uncertainty by orders of magnitude in comparison with methods such as complex step. The method and codes developed are modular in nature and are a drop-in solution for design under uncertainty within existing MDO problems. A low-fidelity analytical multidisciplinary optimization under uncertainty for a wing design in OpenMDAO is detailed in this paper. This demonstration case will include both objective functions and constraints which are influenced by uncertain parameters.

Ben D Phillips↗

NASA NTRSDate not supplied

End-To-End Uncertainty Quantification with Analytical Derivatives for Design Under Uncertainty

Uncertainty quantification (UQ) is a rapidly growing and evolving discipline, especially within the aerospace community. Performing analysis with UQ can provide decision makers with a wealth of information about a candidate design. However, the value of UQ is fully realized when the information gained during UQ analysis is leveraged in a feedback loop of a design optimization process, often referred to as design under uncertainty. Although design under uncertainty can be a powerful risk mitigation technique, there are a number of roadblocks that prevent its implementation. Two primary factors are computational costs and added complexity of the analysis. High fidelity simulations on the order tens of uncertain variables quickly become computationally infeasible. Also, implementing UQ into an existing multidisciplinary design and optimization (MDO) process often requires extensive knowledge of the UQ methods and careful treatment of the problem formulation. The objective of this work is to address these two primary roadblocks and enable practitioners to efficiently perform design under uncertainty with limited knowledge of the UQ discipline. Methods outlined in this paper demonstrate MDO incorporating UQ into the design process, leveraging an analytic derivative tool chain through the entire optimization. The proposed approach leverages machine learning techniques to generate a differentiable confidence interval output from polynomial chaos models. This technique, coupled with the incorporation of analytical derivatives through the Polynomial Chaos Expansion (PCE) process, eliminates the need to estimate derivatives which are usually obtained from finite difference, complex step, or similar methods. Developing a differentiable confidence interval allows mixed uncertainty problems (both epistemic and aleatory) to be modeled. Without such modeling, these problems cannot accurately predict objective functions containing statistical quantities such as mean and variance. The addition of analytic derivatives to a polynomial chaos-based UQ method decreases the computational costs of performing design under uncertainty by orders of magnitude in comparison with methods such as complex step. The method and codes developed are modular in nature and are a drop-in solution for design under uncertainty within existing MDO problems. A low-fidelity analytical multidisciplinary optimization under uncertainty for a wing design in OpenMDAO is detailed in this paper. This demonstration case will include both objective functions and constraints which are influenced by uncertain parameters.

Ben Phillips↗

NASA NTRSDate not supplied

Automation of the Uncertainty Quantification Process Based on Probability Boxes with DAKOTA

To date, while the use of CFD is prevalent, very few efforts have been undertaken that truly attempt to document all (or even most) of the sources of uncertainty in the simulations. Instead, the current state-of-the-art relies heavily on the experience of the CFD practitioner to estimate the uncertainty associated with their simulations through simple sensitivity studies or subject matter expertise. This practice will have to be replaced with a formal uncertainty quantification (UQ) process if CFD is to play an expanded role in the design research and engineering community, test and evaluation community, and ultimately certification for flight. This is especially true for hypersonic air-breathing propulsion systems due to the environment, scale, and duration limitations of ground test facilities. Accounting for uncertainties in a formal manner is a tedious process. Moreover, the typical CFD practitioner is not likely to be familiar with formal UQ methods. Hence, a major obstacle that has prevented the adoption of UQ methods for engineering design and development work is the lack of a tool set to automate most (if not all) of the UQ workflow. Towards this end, the SANDIA package DAKOTA (which has been developed to drive both UQ and optimization processes) will be tightly wrapped around the VULCAN-CFD code to automate the uncertainty quantification process. The automated process will be applied to an isolator turbulence model validation exercise that has previously been documented using a manual approach to the UQ process. Hence, the focus of this paper will be documenting the level to which automation can hide the UQ process details from the CFD practitioner rather than the UQ method itself.

CFD↗

NASA NTRS2022

Uncertainty Models for the Hybrid Parametric Variation Method of Uncertainty Quantification; Analysis

There is some level of uncertainty in every finite element model (FEM), which flows to a level of uncertainty in predicted results. The purpose of uncertainty quantification (UQ) is to provide statistical bounds on prediction accuracy based on model uncertainty. This is distinct from model updating, which attempts to modify models to improve their accuracy. UQ does not improve the accuracy of models, but accepts that the models are inherently inaccurate and attempts to quantify the impact of that inaccuracy on predicted results. Previously, an alternate method for UQ, called the Hybrid Parametric Variation (HPV) method, was applied to Space Launch System (SLS) Hurty/Craig-Bampton (HCB) components to predict system-level statistics for launch vehicle attitude control transfer functions and core stage section loads due to buffet. The HPV method combines a parametric variation of the HCB fixed-interface (FI) modal frequencies with a nonparametric variation (NPV) method that randomly varies the HCB mass and stiffness matrices as Wishart random matrix distributions using random matrix theory (RMT). Alternatively, the most common method for modeling uncertainty in the structural dynamics community is a parametric approach, which varies physical parameters in the model. However, there are several disadvantages associated with the parametric method. Determining a reduced set of parameters that have a significant impact on the system response can be time consuming, and the selected parameter probability distributions are rarely reliably known. Therefore, in practice, the parameters are surrogates for the actual errors, and the link to parameter uncertainty is unknown. Another major drawback is that the uncertainty that can be represented is limited to the form of the nominal FEM. It is the experience of the authors that based on numerous aerospace programs, almost all FEM errors are in form rather than parameter values. This hypothesis is supported by the observation of the authors that it is almost never possible to ‘tune’ a FEM to match modal test results by only modifying model parameters. Model-form uncertainty cannot be directly represented by FEM input parameters nor included in a parametric approach. However, model-form uncertainty can be modeled using RMT, where a probability distribution is developed for the matrix ensemble of interest. The major advantage of the NPV method is that it covers errors in model form. The HPV method anchors uncertainty at the HCB component level to component modal test results by matching the HCB and test modes based on mode descriptions or other methods, and then applying differing levels of frequency variation. The specific variations depend on the confidence to which a component FEM has been validated through modal testing. The NPV method is layered on the frequency variation to match modal test self-orthogonality and cross-orthogonality (XO) results. Once the component uncertainty models are identified, they are assembled, and the uncertainty is propagated to the system level using a Monte Carlo (MC) analysis approach that generates statistics for system-level predictions This provides a UQ method that can be traced to test data, which can be updated as additional data and improved correlated models become available. The purpose of this paper is to collect and present all of the theory for HPV that has been previously published in reports and papers and to present examples of its application. Specifically, component uncertainty models based on the dispersion of corresponding mass and stiffness matrices using proposed test/analysis correlation metrics are investigated. The first example is purely academic so that the true answers are known, and the validity of the HPV method and the corresponding uncertainty models can be determined. The purpose of this paper is to collect and present all of the theory for HPV that has been previously published in reports and papers and to present examples of its application. Specifically, component uncertainty models based on the dispersion of corresponding mass and stiffness matrices using proposed test/analysis correlation metrics are investigated. The first example is purely academic so that the true answers are known, and the validity of the HPV method and the corresponding uncertainty models can be determined. The second example is an application to a component that is design specific to the SLS. Based on this work and other assessments, the HPV method provides another tool to the toolset used for complex system UQ analysis. From experience gathered to date using the HPV method, additional design specific applications must be investigated to provide further confidence in the validity of the HPV method of UQ analysis.

Uncertainty quantification↗

NASA NTRSDate not supplied

A Practical Approach to Uncertainty Quantification Using Probability Boxes

To date, while the use of CFD for aerospace vehicle design and development is prevalent, the documentation of uncertainties associated with the simulations are rare. Instead, the current state-of-the-art relies heavily on the experience of the CFD practitioner to estimate the uncertainty associated with their simulations through simple sensitivity studies or subject matter expertise. This practice will have to be replaced with a formal uncertainty quantification (UQ) process if CFD is to play an expanded role in the research and engineering design community, test and evaluation community, and ultimately certification for flight. Accounting for uncertainties in a formal manner is a tedious process. Moreover, the typical CFD practitioner is not likely to be familiar with formal UQ methods. These factors have prevented the adoption of UQ methods in the engineering design and development cycle. This presentation will outline a credible approach to UQ using Probability Boxes that is straightforward to apply, and can readily be automated using existing UQ tool sets such as the DAKOTA packaged developed at Sandia. The added expense incurred when moving away from a deterministic CFD process to a stochastic one that captures uncertainties to enable risk-informed decision making will be discussed, as well as effective ways to reduce the computational costs.

Uncertainty Quantification↗

NASA NTRS2019

Uncertainty Quantification for JPL Retrievals

We define the notions of verification, validation, and uncertainty quantification (VVUQ), and place them in the context of retrievals of Earth science data. Uncertainty quantification (UQ) is the process of characterizing the uncertainty of a measurement system relative to the true, physical quantity it measures. UQ is needed when measurements are used for scientific inference, assimilated into models with other data, and used for decision-making and risk analysis. Many measurements come with measures of standard error and bias, but these uncertainty measures are often inaccurate, due to the operational need to assume crisp values for parameters which are in fact uncertain, as well as the failure of technical assumptions needed by operational error-estimation approaches. UQ provides a way to study which assumptions fail to hold, and a tool to improve the measurement. We describe a particular, broadly-applicable UQ setup that has been successful for JPL retrievals including OCO-2, and provide recommendations for furthering the UQ state of the practice at JPL.

Turmon, Michael↗

NASA NTRSDate not supplied

Integration of Uncertainty Quantification in a Model-Based Systems Analysis and Engineering Framework

This paper presents a technical approach to improve the confidence in the systems analysis process by integrating Uncertainty Quantification (UQ) techniques within a Model-Based Systems Analysis and Engineering (MBSA&E) framework. The MBSA&E architecture uses system models and multidisciplinary analytical solutions as central artifacts for system design and analysis. The integration of UQ enables engineers to assess and mitigate uncertainties associated with a system model, design parameters, and constraint inputs, leading to more complete design studies and further informed decision-making processes. The proposed approach leverages the strengths of MBSA&E and extends it with a UQ methodology to quantify uncertainties in the input parameters and to trace the uncertainties as they propagate throughout the system model. To demonstrate the effectiveness of an integrated MBSA&E-UQ approach, a case study involving a simplified analysis of a Transonic Truss-Braced Wing (TTBW) concept vehicle is performed. This integration enables a more comprehensive evaluation of system performance and behavior under uncertainty and a more robust approach for system design and analysis. Lastly, the paper addresses the challenges and considerations associated with integrating UQ into an MBSA&E framework.

systems analysis↗

NASA NTRSDate not supplied

Integration of Uncertainty Quantification in a Model-Based Systems Analysis and Engineering Framework

This paper presents a technical approach to improve the confidence in the systems analysis process by integrating Uncertainty Quantification (UQ) techniques within a Model-Based Systems Analysis and Engineering (MBSA&E) framework. The MBSA&E architecture uses system models and multidisciplinary analytical solutions as central artifacts for system design and analysis. The integration of UQ enables engineers to assess and mitigate uncertainties associated with a system model, design parameters, and constraint inputs, leading to more complete design studies and further informed decision-making processes. The proposed approach leverages the strengths of MBSA&E and extends it with a UQ methodology to quantify uncertainties in the input parameters and to trace the uncertainties as they propagate throughout the system model. To demonstrate the effectiveness of an integrated MBSA&E-UQ approach, a case study involving a simplified analysis of a Transonic Truss-Braced Wing (TTBW) concept vehicle is performed. This integration enables a more comprehensive evaluation of system performance and behavior under uncertainty and a more robust approach for system design and analysis. Lastly, the paper addresses the challenges and considerations associated with integrating UQ into an MBSA&E framework.

systems analysis↗