Models and data: a critical relationship for constraining uncertainty in ice sheet model projections
No abstract provided
Larour, Eric↗
SEARCH · Engineering Papers
Results for “Uncertainty modeling”
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
Random Predictor Models for Rigorous Uncertainty Quantification: Part 2
This and a companion paper propose techniques for constructing parametric mathematical models describing key features of the distribution of an output variable given input-output data. By contrast to standard models, which yield a single output value at each value of the input, Random Predictors Models (RPMs) yield a random variable at each value of the input. Optimization-based strategies for calculating RPMs having a polynomial dependency on the input and a linear dependency on the parameters are proposed. These formulations yield RPMs having various levels of fidelity in which the mean, the variance, and the range of the model's parameter, thus of the output, are prescribed. As such they encompass all RPMs conforming to these prescriptions. The RPMs are optimal in the sense that they yield the tightest predictions for which all (or, depending on the formulation, most) of the observations are less than a fixed number of standard deviations from the mean prediction. When the data satisfies mild stochastic assumptions, and the optimization problem(s) used to calculate the RPM is convex (or, when its solution coincides with the solution to an auxiliary convex problem), the model's reliability, which is the probability that a future observation would be within the predicted ranges, is bounded rigorously.
Crespo, Luis G.↗
Random Predictor Models for Rigorous Uncertainty Quantification: Part 1
This and a companion paper propose techniques for constructing parametric mathematical models describing key features of the distribution of an output variable given input-output data. By contrast to standard models, which yield a single output value at each value of the input, Random Predictors Models (RPMs) yield a random variable at each value of the input. Optimization-based strategies for calculating RPMs having a polynomial dependency on the input and a linear dependency on the parameters are proposed. These formulations yield RPMs having various levels of fidelity in which the mean and the variance of the model's parameters, thus of the predicted output, are prescribed. As such they encompass all RPMs conforming to these prescriptions. The RPMs are optimal in the sense that they yield the tightest predictions for which all (or, depending on the formulation, most) of the observations are less than a fixed number of standard deviations from the mean prediction. When the data satisfies mild stochastic assumptions, and the optimization problem(s) used to calculate the RPM is convex (or, when its solution coincides with the solution to an auxiliary convex problem), the model's reliability, which is the probability that a future observation would be within the predicted ranges, can be bounded tightly and rigorously.
Crespo, Luis G.↗
CFD Validation Study of a Hypersonic Cone-Slice-Flap Variable Geometry Configuration
Model validation is the process of determining the degree of accuracy between physical reality and the model. The result of model validation can either be used to improve the model through calibration or quantify the model-form uncertainty. This work focuses on providing the model-form uncertainty through an area metric for a hypersonic cone-slice-flap variable geometry configuration given uncertainty in both the simulation and experimental data. For a variable geometry, performing uncertainty quantification to capture the model-form uncertainty on every configuration is computationally challenging. This work lays out a procedure that can give an accurate representation of the model-form uncertainty using a small number of high-fidelity runs and many low-fidelity runs on multiple configurations. Running this comparison provides a quantifiable measurement for the accuracy of each turbulence model for this type of design. The high-fidelity CFD solver used was VULCAN-CFD and the low-fidelity results came from Cart3D. The experimental data came from the 20-Inch Mach 6 Tunnel located at NASA Langley Research Center. The present work showed that the turbulence simulation overpredicted the drag and lift coefficient, while underpredicting the pitching moment coefficient. The model-form uncertainty estimate resulted up to a 13.6% change in the total uncertainty for the drag coefficient, up to a 57.4% change in total uncertainty for the lift coefficient, and up to a 100% change in total uncertainty for the pitching moment coefficient.
Laura White↗
Strategies for Automation of Model Tuning in Multifidelity Trajectory Uncertainty Propagation
Multi-model Monte Carlo methods are efficient strategies to perform forward uncertainty quantification studies in entry, descent, and landing (EDL) applications. These multi-model methods are based on the classical Monte Carlo estimator, but fuse predictions from several low-fidelity models to obtain estimators with greater precision given a prescribed computational budget. The effectiveness of these approaches relies on the magnitudes of correlations between the low-fidelity models and the high-fidelity model, as well as the relative computational costs of all models. Identifying and exploiting the best trade-off between correlation and cost, which ultimately depends on the selection of hyperparameters in the low-fidelity models, is a task often performed by hand or simply inspired by the deterministic understanding available for a specific application. This work extends a preliminary effort,
Marten Thompson↗
Uncertainty Quantification of Modeled Electron Backstreaming Failure for the NEXT Ion Thruster
Excessive electron back streaming is one of the primary life-limiting failure modes for gridded ion thrusters. Physics-based modeling of the optics grid erosion and electron back streaming margins is augmented with statistical uncertainty quantification techniques to generate a life expectancy distribution of this particular failure mechanism for the NEXT ion thruster. Generation of distributions instead of a single point estimate provide a more comprehensive picture of thruster failure probabilities and life expectation for various mission applications.
Yim, John T.↗
Role of Forcing Uncertainty and Background Model Error Characterization in Snow Data Assimilation
Accurate specification of the model error covariances in data assimilation systems is a challenging issue. Ensemble land data assimilation methods rely on stochastic perturbations of input forcing and model prognostic fields for developing representations of input model error covariances. This article examines the limitations of using a single forcing dataset for specifying forcing uncertainty inputs for assimilating snow depth retrievals. Using an idealized data assimilation experiment, the article demonstrates that the use of hybrid forcing input strategies (either through the use of an ensemble of forcing products or through the added use of the forcing climatology) provide a better characterization of the background model error, which leads to improved data assimilation results, especially during the snow accumulation and melt-time periods. The use of hybrid forcing ensembles is then employed for assimilating snow depth retrievals from the AMSR2 (Advanced Microwave Scanning Radiometer 2) instrument over two domains in the continental USA with different snow evolution characteristics. Over a region near the Great Lakes, where the snow evolution tends to be ephemeral, the use of hybrid forcing ensembles provides significant improvements relative to the use of a single forcing dataset. Over the Colorado headwaters characterized by large snow accumulation, the impact of using the forcing ensemble is less prominent and is largely limited to the snow transition time periods. The results of the article demonstrate that improving the background model error through the use of a forcing ensemble enables the assimilation system to better incorporate the observational information.
assimilation↗
CFD Validation Study of a Hypersonic Cone-Slice-Flap Configuration
Model validation is the process of determining the degree of accuracy between the real world and the model. The result of model validation can be used to either improve the model through calibration or quantify the model-form uncertainty. This work focuses on estimating the model-form uncertainty through an area metric for a hypersonic cone-slice-flap geometry configuration given the uncertainty in both the simulation and experimental data. A procedure that can give an accurate representation of the model-form uncertainty using a small number of high-fidelity runs is outlined. The work also assesses the impact of using different high-fidelity solvers and different turbulence models. The goal of performing this comparison is to provide a quantifiable measurement of the accuracy of each solver and turbulence model for this type of design. A low-fidelity analysis is also performed to get a model-form uncertainty for this type of analysis. The two high-fidelity CFD solvers used here are VULCAN-CFD and FUN3D, and the low-fidelity results comes from Cart3D.The experimental data comes from the 20-inch Mach 6 Tunnel located at NASA Langley Research Center. The work here shows that the models used tend to under-predict the drag and moment aerodynamic coefficients for this type of design leading to a model-form uncertainty estimate which can account for up to an 86% of the total uncertainty in the model predictions.
Laura M White↗
Toward Scientific Numerical Modeling
Ultimately, scientific numerical models need quantified output uncertainties so that modeling can evolve to better match reality. Documenting model input uncertainties and verifying that numerical models are translated into code correctly, however, are necessary first steps toward that goal. Without known input parameter uncertainties, model sensitivities are all one can determine, and without code verification, output uncertainties are simply not reliable. To address these two shortcomings, two proposals are offered: (1) an unobtrusive mechanism to document input parameter uncertainties in situ and (2) an adaptation of the Scientific Method to numerical model development and deployment. Because these two steps require changes in the computational simulation community to bear fruit, they are presented in terms of the Beckhard-Harris-Gleicher change model.
Kleb, Bil↗
Ensemble Methodologies for Astronaut Cancer Risk Assessment in the face of Large Uncertainties
A new approach to NASA space radiation risk modeling has successfully extended the current NASA probabilistic cancer risk model to an ensemble framework able to consider sub-model parameter uncertainty (e.g. uncertainty in a radiation quality parameter) as well as model-form uncertainty associated with differing theoretical or empirical formalisms (e.g. combined dose-rate and radiation quality effects). Ensemble methodologies are already widely used in weather prediction, modeling of infectious disease outbreaks, and certain terrestrial radiation protection applications to better understand how uncertainty may influence risk decision-making. Applying ensemble methodologies to space radiation risk projections offers the potential to efficiently incorporate emerging research results, allow for the incorporation of future (including international) models, improve uncertainty quantification for underlying sub-models developed against sparse experimental data, and reduce the impact of subjective bias on risk projections. Moreover, risk forecasting across an ensemble of multiple predictive models can provide stakeholders additional information on risk acceptance if current health/medical standards cannot be met or the level of knowledge doesn’t permit a specific risk or exposure limit to be developed for future space exploration missions. In this work, ensemble risk projections implementing multiple sub-models of radiation quality, dose and dose-rate effectiveness factors, excess risk, and latency as ensemble members are presented. Initial consensus methods for ensemble model weights and correlations to account for individual model bias are discussed. In these analyses, the ensemble forecast compares well to results from NASA's current operational cancer risk projection model used to assess permissible exposure limits and permissible mission durations for astronauts. However, a large range of projected risk values are obtained at the upper 95th confidence level where models must extrapolate beyond available biological data sets; closer agreement is seen at the median + one sigma due to the inherent similarities in available models. Future work, including the addition of new models and methods for statistical correlation between predictive members are discussed to define alternate ways of thinking about risk and ‘acceptable’ uncertainty with respect to NASA’s current permissible exposure limits.
space radiation↗
Probing Model Sensitivity in Discharge Characteristics for a Li-O2 Battery using Uncertainty Quantification
For this presentation, we looked into using uncertainty quantification to understand the uncertainty in model parameters in predicting voltage and time response of a lithium-air battery.
Lithium-air↗
Translation of Land Surface Model Accuracy and Uncertainty into Coupled Land-Atmosphere Prediction
Land-atmosphere (L-A) Interactions playa critical role in determining the diurnal evolution of both planetary boundary layer (PBL) and land surface heat and moisture budgets, as well as controlling feedbacks with clouds and precipitation that lead to the persistence of dry and wet regimes. Recent efforts to quantify the strength of L-A coupling in prediction models have produced diagnostics that integrate across both the land and PBL components of the system. In this study, we examine the impact of improved specification of land surface states, anomalies, and fluxes on coupled WRF forecasts during the summers of extreme dry (2006) and wet (2007) land surface conditions in the U.S. Southern Great Plains. The improved land initialization and surface flux parameterizations are obtained through the use of a new optimization and uncertainty estimation module in NASA's Land Information System (US-OPT/UE), whereby parameter sets are calibrated in the Noah land surface model and classified according to a land cover and soil type mapping of the observation sites to the full model domain. The impact of calibrated parameters on the a) spinup of the land surface used as initial conditions, and b) heat and moisture states and fluxes of the coupled WRF Simulations are then assessed in terms of ambient weather and land-atmosphere coupling along with measures of uncertainty propagation into the forecasts. In addition, the sensitivity of this approach to the period of calibration (dry, wet, average) is investigated. Finally, tradeoffs of computational tractability and scientific validity, and the potential for combining this approach with satellite remote sensing data are also discussed.
Santanello, Joseph A.↗
CFD Validation Study of a Hypersonic Cone-Slice-Flap Variable Geometry Configuration
Model validation is the process of determining the degree of accuracy between physical reality and the model. The result of model validation can either be used to improve the model through calibration or quantify the model-form uncertainty. This work focuses on providing the model-form uncertainty through an area metric for a hypersonic cone-slice-flap variable geometry configuration given uncertainty in both the simulation and experimental data. The research here compares two different turbulence models for the simulations. For a variable geometry, performing uncertainty quantification to capture the model-form uncertainty on every configuration is computationally challenging. This work lays out a procedure that can give an accurate representation of the model-form uncertainty using a small number of high-fidelity runs and many low-fidelity runs on multiple configurations. Running this comparison provides a quantifiable measurement for the accuracy of each turbulence model for this type of design. The high-fidelity CFD solver used was VULCAN-CFD and the low-fidelity results came from Cart3D. The experimental data came from the 20-Inch Mach 6 Tunnel located at NASA Langley Research Center. The present work showed that the using both the Spalart and Allamaras and Menter Shear-Stress Transport turbulence models overpredicted the drag and lift coefficient, while underpredicting the pitching moment coefficient. The model-form uncertainty estimate resulted in up to a 13.6% change in the total uncertainty for the drag coefficient, up to a 57.4% change in total uncertainty for the lift coefficient, and up to a 100% change in total uncertainty for the pitching moment coefficient.
Laura M. White↗
Interval Predictor Models for Data with Measurement Uncertainty
An interval predictor model (IPM) is a computational model that predicts the range of an output variable given input-output data. This paper proposes strategies for constructing IPMs based on semidefinite programming and sum of squares (SOS). The models are optimal in the sense that they yield an interval valued function of minimal spread containing all the observations. Two different scenarios are considered. The first one is applicable to situations where the data is measured precisely whereas the second one is applicable to data subject to known biases and measurement error. In the latter case, the IPMs are designed to fully contain regions in the input-output space where the data is expected to fall. Moreover, we propose a strategy for reducing the computational cost associated with generating IPMs as well as means to simulate them. Numerical examples illustrate the usage and performance of the proposed formulations.
Lacerda, Marcio J.↗
Impact of Damping Uncertainty on SEA Model Response Variance
Statistical Energy Analysis (SEA) is commonly used to predict high-frequency vibroacoustic levels. This statistical approach provides the mean response over an ensemble of random subsystems that share the same gross system properties such as density, size, and damping. Recently, techniques have been developed to predict the ensemble variance as well as the mean response. However these techniques do not account for uncertainties in the system properties. In the present paper uncertainty in the damping loss factor is propagated through SEA to obtain more realistic prediction bounds that account for both ensemble and damping variance. The analysis is performed on a floor-equipped cylindrical test article that resembles an aircraft fuselage. Realistic bounds on the damping loss factor are determined from measurements acquired on the sidewall of the test article. The analysis demonstrates that uncertainties in damping have the potential to significantly impact the mean and variance of the predicted response.
Schiller, Noah↗
Advanced Methods for Determining Prediction Uncertainty in Model-Based Prognostics with Application to Planetary Rovers
Prognostics is centered on predicting the time of and time until adverse events in components, subsystems, and systems. It typically involves both a state estimation phase, in which the current health state of a system is identified, and a prediction phase, in which the state is projected forward in time. Since prognostics is mainly a prediction problem, prognostic approaches cannot avoid uncertainty, which arises due to several sources. Prognostics algorithms must both characterize this uncertainty and incorporate it into the predictions so that informed decisions can be made about the system. In this paper, we describe three methods to solve these problems, including Monte Carlo-, unscented transform-, and first-order reliability-based methods. Using a planetary rover as a case study, we demonstrate and compare the different methods in simulation for battery end-of-discharge prediction.
prognosis↗
CFD Validation Study of a Hypersonic Cone-Slice-Flap Configuration
Model validation is the process of determining the degree of accuracy between the real world and the model. The result of model validation can either be used to improve the model through calibration or quantify the model-form uncertainty. This work focuses on providing the model-form uncertainty through an area metric for a hypersonic cone-slice-flap geometry configuration given uncertainty in both the simulation and experimental data. A procedure will be presented that can give an accurate representation of the model-form uncertainty using a small number of high-fidelity runs. The work will also assess the impact between using different high-fidelity solvers and different turbulence models.The goal of performing this comparison is to provide a quantifiable measurement for the accuracy of each solver and turbulence model for this type of design. A low-fidelity analysis will also be performed to get a model-form uncertainty for the lower fidelity analysis. The two high-fidelty CFD solvers used are VULCAN-CFD and FUN3D and the low-fidelity results will come from Cart3D. The experimental data came from the 20-inch Mach 6Tunnel located at NASA Langley Research Center.
Laura M. White↗
On-Line Mu Method for Robust Flutter Prediction in Expanding a Safe Flight Envelope for an Aircraft Model Under Flight Test
A structured singular value (mu) analysis method of computing flutter margins has robust stability of a linear aeroelastic model with uncertainty operators (Delta). Flight data is used to update the uncertainty operators to accurately account for errors in the computed model and the observed range of aircraft dynamics of the aircraft under test caused by time-varying aircraft parameters, nonlinearities, and flight anomalies, such as test nonrepeatability. This mu-based approach computes predict flutter margins that are worst case with respect to the modeling uncertainty for use in determining when the aircraft is approaching a flutter condition and defining an expanded safe flight envelope for the aircraft that is accepted with more confidence than traditional methods that do not update the analysis algorithm with flight data by introducing mu as a flutter margin parameter that presents several advantages over tracking damping trends as a measure of a tendency to instability from available flight data.
Lind, Richard C.↗