Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Uncertainties”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 109 records · Page 6

The effect of modeling dose uncertainty on low-boom community noise dose-response curves

In logistic dose-response modeling, failing to account for uncertainty in estimated doses can cause an artificial flattening or attenuation of the slope of the summary curve. In Lee et al. [J. Acoust. Soc. Am. 147(4), pp. 2222-2234 (2020)], data from two NASA low-amplitude sonic boom community noise survey tests were modeled using a Bayesian multilevel logistic regression (MLR) statistical model that assumed there was no uncertainty in the noise dose estimates. However, in these community tests, the noise dose uncertainty was estimated by Page et al. [NASA/CR-2014-218180 and NASA/CR-2020-220589/Volume I] using a leave-one-out method. In the current work, a term was added to extend the Bayesian MLR model to account for the estimated noise dose uncertainty quantified in the Page et al. analyses. This uncertainty term was included in two ways, either as classical or as Berkson uncertainty, and yield similar results. When the uncertainty is accounted for in the Bayesian MLR model, the dose-response curves become 5-10% steeper, but the difference in the noise dose that elicits a 5% highly annoyed response is small (less than 1 dB). This result is encouraging for future X-59 community tests whose survey area will be sparsely populated with noise monitors.

X-59↗

Effective Uncertainty Quantification for Multi-Angle Polarimetric Aerosol Remote Sensing Over Ocean

Multi-angle polarimetric (MAP) measurements can enable detailed characterization of aerosol microphysical and optical properties and improve atmospheric correction in ocean color remote sensing. Advanced retrieval algorithms have been developed to obtain multiple geophysical parameters in the atmosphere–ocean system. Theoretical pixel-wise retrieval uncertainties based on error propagation have been used to quantify retrieval performance and determine the quality of data products. However, standard error propagation techniques in high-dimensional retrievals may not always represent true retrieval errors well due to issues such as local minima and the nonlinear dependence of the forward model on the retrieved parameters near the solution. In this work, we analyze these theoretical uncertainty estimates and validate them using a flexible Monte Carlo approach. The Fast Multi-Angular Polarimetric Ocean coLor (FastMAPOL) retrieval algorithm, based on efficient neural network forward models, is used to conduct the retrievals and uncertainty quantification on both synthetic HARP2 (Hyper-Angular Rainbow Polarimeter 2) and AirHARP (airborne version of HARP2) datasets. In addition, for practical application of the uncertainty evaluation technique in operational data processing, we use the automatic differentiation method to calculate derivatives analytically based on the neural network models. Both the speed and accuracy associated with uncertainty quantification for MAP retrievals are addressed in this study. Pixel-wise retrieval uncertainties are further evaluated for the real AirHARP field campaign data. The uncertainty quantification methods and results can be used to evaluate the quality of data products, as well as guide MAP algorithm development for current and future satellite systems such as NASA’s Plankton, Aerosol, Cloud, ocean Ecosystem (PACE) mission.

PACE↗

Uncertainty Improvement in the NASA Glenn Research Center 8- by 6-Foot Supersonic Wind Tunnel

Within the past decade, a measurement uncertainty analysis (MUA) team was assembled at the NASA Glenn Research Center to assist the wind tunnel characterization team with quantification of uncertainty estimates for variables of interest in wind tunnels and test cells across the center. The initial analysis performed by the team was conducted on the 8- by 6- Foot Supersonic Wind Tunnel using results from the 1996/97 full calibration test entry. These MUA results were published in 2016 which included recommendations for methods to improve the uncertainty estimates for various variables of interest, such as changes to regression models, tunnel operation philosophy, and instrumentation choices. The wind tunnel characterization team utilized the proposed methods in 2019 during a full characterization test entry. Following publication of the 2019 test section characterization results, the MUA team pursued an update to the uncertainty estimates for the facility, which validated previous recommendations and revealed significantly reduced systematic uncertainties across most variables of interest. Inclusion of within-test repeat points in the 2019 test entry allowed for higher fidelity estimates of random uncertainty to be generated, as well. This collaboration between the facility, wind tunnel characterization, and MUA teams serves as an example of the data quality benefits that can be achieved through rigorous analysis of the sources of uncertainty in a ground-test facility.

Uncertainty↗

Uncertainty Improvement in the NASA Glenn Research Center 8- by 6-Foot Supersonic Wind Tunnel

Within the past decade, a measurement uncertainty analysis (MUA) team was assembled at NASA Glenn Research Center to assist the wind tunnel characterization team with quantification of uncertainty estimates for variables of interest in wind tunnels and test cells across the center. The initial analysis performed by the team was conducted on the 8- by 6-Foot Supersonic Wind Tunnel using results from the 1996/97 full calibration test entry. These MUA results were published in 2016 which included recommendations for methods to improve the uncertainty estimates for various variables of interest, such as changes to regression models, tunnel operation philosophy, and instrumentation choices. The wind tunnel characterization team utilized the proposed methods in 2019 during a full characterization test entry. Following publication of the 2019 test section characterization results, the MUA team pursued an update to the uncertainty estimates for the facility, which validated previous recommendations and revealed significantly reduced systematic uncertainties across most variables of interest. Inclusion of within-test repeat points in the 2019 test entry allowed for higher fidelity estimates of random uncertainty to be generated, as well. This collaboration between the facility, wind tunnel characterization, and MUA teams serves as an example of the data quality benefits that can be achieved through rigorous analysis of the sources of uncertainty in a ground-test facility.

Uncertainty↗

Uncertainty Quantification for Empirical X-59 Sonic Boom Loudness Levels

Estimates of the total uncertainty for empirically determined loudness levels are documented when GRS (Ground Recording System) noise monitors are used to record X 59 sonic boom waveforms. The total uncertainty is characterized by combining nine different sources of uncertainty that may affect the apparent gain of the measurement chain. These uncertainty estimates are presented as expected measurement error relative to the true loudness level, and separate error estimates are provided for eight different noise metrics in which NASA has interest. The behavior of the Perceived Level (PL) metric is studied within the report body, while the total uncertainties for the seven other noise metrics are summarized in appendices for brevity. The effects of four sources of uncertainty are estimated simply from information found on hardware specification sheets provided by the manufacturer. However, mock acoustic recordings are created to estimate the effects of other sources because those effects are expected to induce spectral coloration, so they may vary with noise metric type and sound level. These sources are not well modeled by simple gain adjustments. Importantly, measurement error is computable when processing mock recordings since the true levels are knowable, which is not the case when processing data recorded in the field. Specifically, the true levels are knowable because the components of the mock recordings are separable – e.g., loudness levels of booms can be computed with or without superimposed background noise. Mock acoustic recordings also have the benefit of allowing analysis of sonic booms from vehicles that are not yet flying, like the X-59, since the mock recordings are created by combining vehicle-specific predicted waveforms with other audio sources. The estimates of total measurement error are documented as a function of the signal-to-noise ratio (SNR) of the loudness level, where the corrected SNR is computed while accounting for the effects of the method that is used to correct for background noise contamination when computing the noise metric values. The corrected SNR calculations used here can be applied to both mock recordings and in-field measurements, so the uncertainty of in-field recordings can be found using pre-computed lookup tables that identify the relationship between metric type, corrected SNR, and the expected measurement error.

Sonic Boom↗

Study of Uncertainties of Predicting Space Shuttle Thermal Environment

Quantitative estimates of the uncertainty in predicting aerodynamic heating rates for a fully reusable space shuttle system are developed and the impact of these uncertainties on Thermal Protection System (TPS) weight are discussed. The study approach consisted of statistical evaluations of the scatter of heating data on shuttle configurations about state-of-the-art heating prediction methods to define the uncertainty in these heating predictions. The uncertainties were then applied as heating rate increments to the nominal predicted heating rate to define the uncertainty in TPS weight. Separate evaluations were made for the booster and orbiter, for trajectories which included boost through reentry and touchdown. For purposes of analysis, the vehicle configuration is divided into areas in which a given prediction method is expected to apply, and separate uncertainty factors and corresponding uncertainty in TPS weight derived for each area.

Fehrman, A. L.↗

Aeroservoelastic Uncertainty Model Identification from Flight Data

Uncertainty modeling is a critical element in the estimation of robust stability margins for stability boundary prediction and robust flight control system development. There has been a serious deficiency to date in aeroservoelastic data analysis with attention to uncertainty modeling. Uncertainty can be estimated from flight data using both parametric and nonparametric identification techniques. The model validation problem addressed in this paper is to identify aeroservoelastic models with associated uncertainty structures from a limited amount of controlled excitation inputs over an extensive flight envelope. The challenge to this problem is to update analytical models from flight data estimates while also deriving non-conservative uncertainty descriptions consistent with the flight data. Multisine control surface command inputs and control system feedbacks are used as signals in a wavelet-based modal parameter estimation procedure for model updates. Transfer function estimates are incorporated in a robust minimax estimation scheme to get input-output parameters and error bounds consistent with the data and model structure. Uncertainty estimates derived from the data in this manner provide an appropriate and relevant representation for model development and robust stability analysis. This model-plus-uncertainty identification procedure is applied to aeroservoelastic flight data from the NASA Dryden Flight Research Center F-18 Systems Research Aircraft.

Brenner, Martin J.↗

Sources of Uncertainty in the Prediction of LAI / fPAR from MODIS

To explicate the sources of uncertainty in the prediction of biophysical variables over space, consider the general equation: where z is a variable with values on some nominal, ordinal, interval or ratio scale; y is a vector of input variables; u is the spatial support of y and z ; x and u are the spatial locations of y and z , respectively; f is a model and B is the vector of the parameters of this model. Any y or z has a value and a spatial extent which is called its support. Viewed in this way, categories of uncertainty are from variable (e.g. measurement), parameter, positional. support and model (e.g. structural) sources. The prediction of Leaf Area Index (LAI) and the fraction of absorbed photosynthetically active radiation (fPAR) are examples of z variables predicted using model(s) as a function of y variables and spatially constant parameters. The MOD15 algorithm is an example of f, called f(sub 1), with parameters including those defined by one of six biome types and solar and view angles. The Leaf Canopy Model (LCM)2, a nested model that combines leaf radiative transfer with a full canopy reflectance model through the phase function, is a simpler though similar radiative transfer approach to f(sub 1). In a previous study, MOD15 and LCM2 gave similar results for the broadleaf forest biome. Differences between these two models can be used to consider the structural uncertainty in prediction results. In an effort to quantify each of the five sources of uncertainty and rank their relative importance for the LAI/fPAR prediction problem, we used recent data for an EOS Core Validation Site in the broadleaf biome with coincident surface reflectance, vegetation index, fPAR and LAI products from the Moderate Resolution Imaging Spectrometer (MODIS). Uncertainty due to support on the input reflectance variable was characterized using Landsat ETM+ data. Input uncertainties were propagated through the LCM2 model and compared with published uncertainties from the MOD15 algorithm.

Dungan, Jennifer L.↗

Structured Uncertainty Bound Determination From Data for Control and Performance Validation

This report attempts to document the broad scope of issues that must be satisfactorily resolved before one can expect to methodically obtain, with a reasonable confidence, a near-optimal robust closed loop performance in physical applications. These include elements of signal processing, noise identification, system identification, model validation, and uncertainty modeling. Based on a recently developed methodology involving a parameterization of all model validating uncertainty sets for a given linear fractional transformation (LFT) structure and noise allowance, a new software, Uncertainty Bound Identification (UBID) toolbox, which conveniently executes model validation tests and determine uncertainty bounds from data, has been designed and is currently available. This toolbox also serves to benchmark the current state-of-the-art in uncertainty bound determination and in turn facilitate benchmarking of robust control technology. To help clarify the methodology and use of the new software, two tutorial examples are provided. The first involves the uncertainty characterization of a flexible structure dynamics, and the second example involves a closed loop performance validation of a ducted fan based on an uncertainty bound from data. These examples, along with other simulation and experimental results, also help describe the many factors and assumptions that determine the degree of success in applying robust control theory to practical problems.

Lim, Kyong B.↗

Uncertainties in estimates of the risks of late effects from space radiation

Methods used to project risks in low-Earth orbit are of questionable merit for exploration missions because of the limited radiobiology data and knowledge of galactic cosmic ray (GCR) heavy ions, which causes estimates of the risk of late effects to be highly uncertain. Risk projections involve a product of many biological and physical factors, each of which has a differential range of uncertainty due to lack of data and knowledge. Using the linear-additivity model for radiation risks, we use Monte-Carlo sampling from subjective uncertainty distributions in each factor to obtain an estimate of the overall uncertainty in risk projections. The resulting methodology is applied to several human space exploration mission scenarios including a deep space outpost and Mars missions of duration of 360, 660, and 1000 days. The major results are the quantification of the uncertainties in current risk estimates, the identification of factors that dominate risk projection uncertainties, and the development of a method to quantify candidate approaches to reduce uncertainties or mitigate risks. The large uncertainties in GCR risk projections lead to probability distributions of risk that mask any potential risk reduction using the "optimization" of shielding materials or configurations. In contrast, the design of shielding optimization approaches for solar particle events and trapped protons can be made at this time and promising technologies can be shown to have merit using our approach. The methods used also make it possible to express risk management objectives in terms of quantitative metrics, e.g., the number of days in space without exceeding a given risk level within well-defined confidence limits. Published by Elsevier Ltd on behalf of COSPAR.

Non-NASA Center↗

Uncertainties in Estimates of the Risks of Late Effects from Space Radiation

The health risks faced by astronauts from space radiation include cancer, cataracts, hereditary effects, and non-cancer morbidity and mortality risks related to the diseases of the old age. Methods used to project risks in low-Earth orbit are of questionable merit for exploration missions because of the limited radiobiology data and knowledge of galactic cosmic ray (GCR) heavy ions, which causes estimates of the risk of late effects to be highly uncertain. Risk projections involve a product of many biological and physical factors, each of which has a differential range of uncertainty due to lack of data and knowledge. Within the linear-additivity model, we use Monte-Carlo sampling from subjective uncertainty distributions in each factor to obtain a Maximum Likelihood estimate of the overall uncertainty in risk projections. The resulting methodology is applied to several human space exploration mission scenarios including ISS, lunar station, deep space outpost, and Mar's missions of duration of 360, 660, and 1000 days. The major results are the quantification of the uncertainties in current risk estimates, the identification of factors that dominate risk projection uncertainties, and the development of a method to quantify candidate approaches to reduce uncertainties or mitigate risks. The large uncertainties in GCR risk projections lead to probability distributions of risk that mask any potential risk reduction using the "optimization" of shielding materials or configurations. In contrast, the design of shielding optimization approaches for solar particle events and trapped protons can be made at this time, and promising technologies can be shown to have merit using our approach. The methods used also make it possible to express risk management objectives in terms of quantitative objective's, i.e., the number of days in space without exceeding a given risk level within well defined confidence limits.

Cucinotta, F. A.↗

Sources of Uncertainty in Predicting Land Surface Fluxes Using Diverse Data and Models

In the domain of predicting land surface fluxes, models are used to bring data from large observation networks and satellite remote sensing together to make predictions about present and future states of the Earth. Characterizing the uncertainty about such predictions is a complex process and one that is not yet fully understood. Uncertainty exists about initialization, measurement and interpolation of input variables; model parameters; model structure; and mixed spatial and temporal supports. Multiple models or structures often exist to describe the same processes. Uncertainty about structure is currently addressed by running an ensemble of different models and examining the distribution of model outputs. To illustrate structural uncertainty, a multi-model ensemble experiment we have been conducting using the Terrestrial Observation and Prediction System (TOPS) will be discussed. TOPS uses public versions of process-based ecosystem models that use satellite-derived inputs along with surface climate data and land surface characterization to produce predictions of ecosystem fluxes including gross and net primary production and net ecosystem exchange. Using the TOPS framework, we have explored the uncertainty arising from the application of models with different assumptions, structures, parameters, and variable definitions. With a small number of models, this only begins to capture the range of possible spatial fields of ecosystem fluxes. Few attempts have been made to systematically address the components of uncertainty in such a framework. We discuss the characterization of uncertainty for this approach including both quantifiable and poorly known aspects.

Dungan, Jennifer L.↗

The Impact of Model and Rainfall Forcing Errors on Characterizing Soil Moisture Uncertainty in Land Surface Modeling

The contribution of rainfall forcing errors relative to model (structural and parameter) uncertainty in the prediction of soil moisture is investigated by integrating the NASA Catchment Land Surface Model (CLSM), forced with hydro-meteorological data, in the Oklahoma region. Rainfall-forcing uncertainty is introduced using a stochastic error model that generates ensemble rainfall fields from satellite rainfall products. The ensemble satellite rain fields are propagated through CLSM to produce soil moisture ensembles. Errors in CLSM are modeled with two different approaches: either by perturbing model parameters (representing model parameter uncertainty) or by adding randomly generated noise (representing model structure and parameter uncertainty) to the model prognostic variables. Our findings highlight that the method currently used in the NASA GEOS-5 Land Data Assimilation System to perturb CLSM variables poorly describes the uncertainty in the predicted soil moisture, even when combined with rainfall model perturbations. On the other hand, by adding model parameter perturbations to rainfall forcing perturbations, a better characterization of uncertainty in soil moisture simulations is observed. Specifically, an analysis of the rank histograms shows that the most consistent ensemble of soil moisture is obtained by combining rainfall and model parameter perturbations. When rainfall forcing and model prognostic perturbations are added, the rank histogram shows a U-shape at the domain average scale, which corresponds to a lack of variability in the forecast ensemble. The more accurate estimation of the soil moisture prediction uncertainty obtained by combining rainfall and parameter perturbations is encouraging for the application of this approach in ensemble data assimilation systems.

Maggioni, V.↗

Advanced Modeling and Uncertainty Quantification for Flight Dynamics; Interim Results and Challenges

As part of the NASA Vehicle Systems Safety Technologies (VSST), Assuring Safe and Effective Aircraft Control Under Hazardous Conditions (Technical Challenge #3), an effort is underway within Boeing Research and Technology (BR&T) to address Advanced Modeling and Uncertainty Quantification for Flight Dynamics (VSST1-7). The scope of the effort is to develop and evaluate advanced multidisciplinary flight dynamics modeling techniques, including integrated uncertainties, to facilitate higher fidelity response characterization of current and future aircraft configurations approaching and during loss-of-control conditions. This approach is to incorporate multiple flight dynamics modeling methods for aerodynamics, structures, and propulsion, including experimental, computational, and analytical. Also to be included are techniques for data integration and uncertainty characterization and quantification. This research shall introduce new and updated multidisciplinary modeling and simulation technologies designed to improve the ability to characterize airplane response in off-nominal flight conditions. The research shall also introduce new techniques for uncertainty modeling that will provide a unified database model comprised of multiple sources, as well as an uncertainty bounds database for each data source such that a full vehicle uncertainty analysis is possible even when approaching or beyond Loss of Control boundaries. Methodologies developed as part of this research shall be instrumental in predicting and mitigating loss of control precursors and events directly linked to causal and contributing factors, such as stall, failures, damage, or icing. The tasks will include utilizing the BR&T Water Tunnel to collect static and dynamic data to be compared to the GTM extended WT database, characterizing flight dynamics in off-nominal conditions, developing tools for structural load estimation under dynamic conditions, devising methods for integrating various modeling elements into a real-time simulation capability, generating techniques for uncertainty modeling that draw data from multiple modeling sources, and providing a unified database model that includes nominal plus increments for each flight condition. This paper presents status of testing in the BR&T water tunnel and analysis of the resulting data and efforts to characterize these data using alternative modeling methods. Program challenges and issues are also presented.

Hyde, David C.↗

Helium Mass Spectrometer Leak Detection: A Method to Quantify Total Measurement Uncertainty

In applications where leak rates of components or systems are evaluated against a leak rate requirement, the uncertainty of the measured leak rate must be included in the reported result. However, in the helium mass spectrometer leak detection method, the sensitivity, or resolution, of the instrument is often the only component of the total measurement uncertainty noted when reporting results. To address this shortfall, a measurement uncertainty analysis method was developed that includes the leak detector unit's resolution, repeatability, hysteresis, and drift, along with the uncertainty associated with the calibration standard. In a step-wise process, the method identifies the bias and precision components of the calibration standard, the measurement correction factor (K-factor), and the leak detector unit. Together these individual contributions to error are combined and the total measurement uncertainty is determined using the root-sum-square method. It was found that the precision component contributes more to the total uncertainty than the bias component, but the bias component is not insignificant. For helium mass spectrometer leak rate tests where unit sensitivity alone is not enough, a thorough evaluation of the measurement uncertainty such as the one presented herein should be performed and reported along with the leak rate value.

error analysis↗

Uncertainties in Predicting Rice Yield by Current Crop Models Under a Wide Range of Climatic Conditions

Predicting rice (Oryza sativa) productivity under future climates is important for global food security. Ecophysiological crop models in combination with climate model outputs are commonly used in yield prediction, but uncertainties associated with crop models remain largely unquantified. We evaluated 13 rice models against multi-year experimental yield data at four sites with diverse climatic conditions in Asia and examined whether different modeling approaches on major physiological processes attribute to the uncertainties of prediction to field measured yields and to the uncertainties of sensitivity to changes in temperature and CO2 concentration [CO2]. We also examined whether a use of an ensemble of crop models can reduce the uncertainties. Individual models did not consistently reproduce both experimental and regional yields well, and uncertainty was larger at the warmest and coolest sites. The variation in yield projections was larger among crop models than variation resulting from 16 global climate model-based scenarios. However, the mean of predictions of all crop models reproduced experimental data, with an uncertainty of less than 10 percent of measured yields. Using an ensemble of eight models calibrated only for phenology or five models calibrated in detail resulted in the uncertainty equivalent to that of the measured yield in well-controlled agronomic field experiments. Sensitivity analysis indicates the necessity to improve the accuracy in predicting both biomass and harvest index in response to increasing [CO2] and temperature.

crop-model ensembles↗

Estimating Uncertainties in the SBUV Version 8.6 Merged Profile Ozone Data Set

The combined record of total and profile ozone measurements from the solar backscatter ultraviolet (SBUV) and SBUV/2 series of instruments, known as the SBUV Merged Ozone Data (MOD) product, constitutes the longest satellite-based ozone time series from a single instrument type and as such plays a key role in ozone trend analyses. Following the approach documented in Frith et al. (2014) to analyze the merging uncertainties in the MOD total ozone record, we use Monte Carlo simulations to estimate the potential for uncertainties in the calibration and drift of individual instruments in the profile ozone merged data set. We focus our discussion on the trends and associated merging uncertainty since 2001 in an effort to verify the start of ozone recovery as predicted by chemistry climate models. We find that merging uncertainty dominates the overall estimated uncertainty when considering only the 15 years of data since 2001. We derive trends versus pressure level for the MOD data set that are positive in the upper stratosphere as expected for ozone recovery. These trends appear to be significant when only statistical uncertainties are included but become not significant at the 2o level when instrument uncertainties are accounted for. However, when we use the entire data set from 1979 through 2015 and fit to the EESC (equivalent effective stratospheric chlorine) we find statistically significant fits throughout the upper stratosphere at all latitudes. This implies that the ozone profile data remain consistent with our expectation that chlorine is the dominant ozone forcing term.

Ozone profile↗

Improvements in the Uncertainty Model in the Goddard Institute for Space Studies Surface Temperature (GISTEMP) Analysis

We outline a new and improved uncertainty analysis for the Goddard Institute of Space Studies (GISS) Surface Temperature product version 4 (GISTEMP v4). Historical spatial variations in surface temperature anomalies are derived from historical weather station data and ocean data from ships, buoys and other sensors. Uncertainties arise from measurement uncertainty, changes in spatial coverage of the station record, and systematic biases due to technology shifts and land cover changes. Previously published uncertainty estimates for GISTEMP included only the effect of incomplete station coverage. Here, we update this term using currently available spatial distributions of source data, state‐of‐the‐art reanalyses and incorporate independently derived estimates for ocean data processing, station homogenization and other structural biases. The resulting 95% uncertainties are near 0.05° C in the global annual mean for the last 50 years, and increase going back further in time reaching 0.15° C in 1880. In addition, we quantify the benefits and inherent uncertainty due to the GISTEMP interpolation and averaging method. We use the total uncertainties to estimate the probability for each record year in the GISTEMP to actually be the true record year (to that date), and conclude with 86% likelihood that 2016 was indeed the hottest year of the instrumental period (so far).

Nathan J L Lenssen↗