Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “uncertainty Quantification”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 91 records · Page 5
All sourcesNASA NTRSDOE OSTI
NASA NTRSDate not supplied

Evaluation of CFD as a Surrogate for Wind-Tunnel Testing: Experimental Uncertainty Quantification for the UPWT Flow Survey Test

A series of wind tunnel tests are being performed at the Unitary Plan Wind Tunnel (UPWT)at NASA Langley Research Center to assess the validity of using computational fluid dynamics(CFD) as a surrogate for wind tunnel testing. In order to make proper comparisons, uncertainties in CFD results and experimental data must be well understood. The material presented highlights the methods, assumptions, and inputs used to achieve experimental uncertainty estimates. Results for a small subset of tunnel conditions and variables of interest from the first test in the series, the Flow Survey Test, are highlighted and sample CFD comparisons are shown. The bulk of the results from this work are used for comparisons in other AIAA conference papers related to this test series.

Erin P Hubbard↗
NASA NTRS2012

System Identification and Uncertainty Quantification Using Orthogonal Excitations and the Semi-span Super Sonic Transport (S4T) Model

Orthogonal harmonic multisine excitations were utilized in a wind tunnel test and in simulation of the SemiSpan Supersonic Transport model to assess aeroservoelastic characteristics. Fundamental issues associated with analyzing sinusoidal signals were examined, including spectral leakage, excitation truncation, and uncertainties on frequency response functions and mean-square coherence. Simulation allowed for evaluation of these issues relative to a truth model, while wind tunnel data introduced real-world implementation issues.

Heeg, Jennifer↗
NASA NTRS2022

Optimal Estimation Framework for Ocean Color Atmospheric Correction and Pixel-level Uncertainty Quantification

Ocean color remote sensing requires compensation for atmospheric scattering and absorption (aerosol, Rayleigh, and trace gases), referred to as atmospheric correction (AC). AC allows inference of parameters such as spectrally resolved remote sensing reflectance ( R rs )(λ) ; sr 1 ) at the ocean surface from the top-of-atmosphere reflectance. Often, the uncertainty of this process is not fully explored. Bayesian inference techniques provide a simultaneous AC and uncertainty assessment via a full posterior distribution of the relevant variables, given the prior distribution of those variables and the radiative transfer (RT) likelihood function. Given uncertainties in the algorithm inputs, the Bayesian framework enables better constraints on the AC process by using the complete spectral information compared to traditional approaches that use only a subset of bands for AC. This paper investigates a Bayesian inference research method (Optimal Estimation, OE) for ocean color AC by simultaneously retrieving atmospheric and ocean properties using all visible and near-infrared spectral bands. The OE algorithm analytically approximates the posterior distribution of parameters based on normality assumptions and provides a potentially viable operational algorithm with a reduced computational expense. We developed a Neural Network (NN) RT forward model look-up-table-based emulator to increase algorithm efficiency further and thus speed up the likelihood computations. We then applied the OE algorithm to synthetic data and observations from the MODerate resolution Imaging Spectroradiometer (MODIS) on NASA’s Aqua spacecraft. We compared the R rs )(λ) retrieval and its uncertainty estimates from the OE method with in-situ validation data from the SeaWiFS Bio-optical Archive and Storage System (SeaBASS) and Aerosol Robotic Network Ocean Color (AERONET-OC) datasets. The OE algorithm improved R rs )(λ) estimates relative to the NASA standard operational algorithm by improving all statistical metrics at 443, 555, and 667 nm. Unphysical negative R rs )(λ) , which often appear in complex water conditions, was reduced by a factor of 3. The OE-derived pixel-level R rs )(λ) uncertainty estimates were also assessed relative to in-situ data and were shown to have skill.

Atmospheric correction↗
NASA NTRS2019

Deep Learning and Uncertainty Quantification for Climate Resilience

Modeling and monitoring of earth’s processes through physical models and satellite observations at high resolutions is crucial for ensuring society’s ability to adapt to climate change. Deep learning (DL) has been shown to be a valuable tool for generating high resolution data, emulating physical models, and detecting weather patterns which can then be used to inform stakeholders and decision makers. However, both the data and model parameters contain substantial uncertainties that may alter users’ decisions. In this work we present two DL applications on high-resolution climate and satellite datasets using Bayesian neural networks to generate well calibrated uncertainty estimates.

Vandal, Thomas↗
NASA NTRSDate not supplied

Uncertainty Quantification of Classical Theories of Dendritic Growth Kinetics Applied to Nickel-Based Alloys

The solidification velocity in a model nickel-alloy single crystal during laser spot melting was recently characterized using synchrotron X-ray imaging. The measured solidification velocity was found to exceed the absolute stability threshold predicted by the Kurz-Giovanola-Trivedi (KGT) model. The discrepancies between the model and experiments motivate the further assessment of accurate material properties. This work quantifies the impact material property uncertainty has on model predictions of the absolute stability threshold velocities. Properties from the literature are reviewed and compared to those calculated using computational thermodynamics to provide uncertainty estimates on input properties to the KGT model. Global sensitivity analysis is used to quantify the influence of each uncertain input property on the predicted threshold velocity. This work supports the understanding of the nickel-alloy solidification during powder bed fusion additive manufacturing and identifies the solidification material properties that are the most important to assess from first-principles computations and experiments.

Computational thermodynamics↗
NASA NTRS2016

Progress of Aircraft System Noise Assessment with Uncertainty Quantification for the Environmentally Responsible Aviation Project

Aircraft system noise predictions have been performed for NASA modeled hybrid wing body aircraft advanced concepts with 2025 entry-into-service technology assumptions. The system noise predictions developed over a period from 2009 to 2016 as a result of improved modeling of the aircraft concepts, design changes, technology development, flight path modeling, and the use of extensive integrated system level experimental data. In addition, the system noise prediction models and process have been improved in many ways. An additional process is developed here for quantifying the uncertainty with a 95% confidence level. This uncertainty applies only to the aircraft system noise prediction process. For three points in time during this period, the vehicle designs, technologies, and noise prediction process are documented. For each of the three predictions, and with the information available at each of those points in time, the uncertainty is quantified using the direct Monte Carlo method with 10,000 simulations. For the prediction of cumulative noise of an advanced aircraft at the conceptual level of design, the total uncertainty band has been reduced from 12.2 to 9.6 EPNL dB. A value of 3.6 EPNL dB is proposed as the lower limit of uncertainty possible for the cumulative system noise prediction of an advanced aircraft concept.

Thomas, Russell H.↗
NASA NTRSDate not supplied

Comparison of Entry Descent and Landing Aerodynamic Databases with Uncertainty Quantification Developed Using Machine Learning Techniques

When developing the aerodynamic databases for use in trajectory simulations, it is important to develop a system of metrics to qualify which aerodynamic models are best to use. Since aerodynamics are just one input into trajectory simulations, the results of these simulations do not reflect on the quality of the aerodynamic database used. This means that aerodynamic database comparisons must be done offline. While traditional metrics that focus on mean/nominal predictions are a good first step, more robust estimates of the prediction interval become important as more focused uncertainty models are developed. We explore the limitations of evaluating aerodynamic models based purely on nominal-centered response surfaces. Before elaborating and evaluating metrics based on distributed models, the value of evaluating prediction interval and confidence interval are discussed to conclude that prediction intervals are more relevant to the use of trajectory analysis. Several metrics to evaluate the prediction interval are introduced with a focus on the standard calibration metric. Finally, we compare candidate models using both mean and distributed metrics. A finalized candidate model developed using state of the art machine learning methods is compared to a baseline model developed using traditional aerodynamic database modeling techniques.

Aerodynamic Database↗
NASA NTRS2016

Performance Metrics, Error Modeling, and Uncertainty Quantification

A common set of statistical metrics has been used to summarize the performance of models or measurements-­ the most widely used ones being bias, mean square error, and linear correlation coefficient. They assume linear, additive, Gaussian errors, and they are interdependent, incomplete, and incapable of directly quantifying un­certainty. The authors demonstrate that these metrics can be directly derived from the parameters of the simple linear error model. Since a correct error model captures the full error information, it is argued that the specification of a parametric error model should be an alternative to the metrics-based approach. The error-modeling meth­odology is applicable to both linear and nonlinear errors, while the metrics are only meaningful for linear errors. In addition, the error model expresses the error structure more naturally, and directly quantifies uncertainty. This argument is further explained by highlighting the intrinsic connections between the performance metrics, the error model, and the joint distribution between the data and the reference.

Quantification↗
NASA NTRS2016

Space Launch System Booster Separation Aerodynamic Database Development and Uncertainty Quantification

The development of the aerodynamic database for the Space Launch System (SLS) booster separation environment has presented many challenges because of the complex physics of the ow around three independent bodies due to proximity e ects and jet inter- actions from the booster separation motors and the core stage engines. This aerodynamic environment is dicult to simulate in a wind tunnel experiment and also dicult to simu- late with computational uid dynamics. The database is further complicated by the high dimensionality of the independent variable space, which includes the orientation of the core stage, the relative positions and orientations of the solid rocket boosters, and the thrust lev- els of the various engines. Moreover, the clearance between the core stage and the boosters during the separation event is sensitive to the aerodynamic uncertainties of the database. This paper will present the development process for Version 3 of the SLS booster separa- tion aerodynamic database and the statistics-based uncertainty quanti cation process for the database.

Chan, David T.↗
NASA NTRSDate not supplied

Probing Sensitivity of Discharge Characteristics to Model Selection using Uncertainty Quantification in an aprotic Li-Oxygen Battery

Currently, there are several models in the literature, such as kinetic models, microstructural models, and mass transport models that describe a Li-air battery's discharge behavior. Many of these models are calibrated and tested at low current densities and cannot be easily transferred to high current densities. Even at low current densities, there is no quantitative method for a researcher to choose a reaction kinetic model such as classical Butler-Volmer and its derivatives, and modified Marcus-Hush-Chidsey, a resistance model for lithium peroxide such as electron transport via tunneling or linear resistivity, a surface coverage model (lithium peroxide growth) such as partial coverage or full coverage, and mass transport model (discussed in Ref. [1]). Also, it is time-consuming to test different models at high current density (1C) due to a lack of well-tested models and well-calibrated model parameters. For this presentation, we will develop an analytical model, which acts as a surrogate model for a sophisticated finite element model to predict discharge time and discharge voltage. Next, we use an uncertainty quantifying technique called reduced-order stochastic optimization [2, 3] to determine the uncertainty in model parameters for rate kinetics, lithium peroxide resistivity, and parasitic resistance. Finally, a finite element simulation is performed to determine the error introduced by the surrogate model and its influence on the uncertainty in the model parameters.

M Mehta↗
NASA NTRS2022

Bayesian Rules of Thumb: Robust Uncertainty Quantification in Early Project Cost Estimation

Systems engineers often make use of cost Rules ofThumb in order to estimate cost during early phases of projectformulation. These Rules of Thumb typically take the form ofa sequence of percentages over which a total cost is allocatedacross NASA WBS elements. Rules of Thumb can then be usedto extrapolate cost from one or more known WBS elements tothe remaining unknown WBS elements, assisting early projectformulation architecture studies (such as those in JPL’s Team Xand A Team).A number of issues can arise when generating and using costRules of Thumb. For example, many records of project costsconsist of incomplete data. Typical methods of dealing withincomplete cost allocation data include (a) ignoring missionswith incomplete data, or (b) taking averages of the non-zero percentagesacross missions, but both of these methods can result inbiased estimates if the existence of incomplete data correlateswith total mission cost or any particular WBS element. Anothercommon example is cost reported in one or more incorrect WBSelements. This is especially prevalent in smaller missions whereit is more common for engineers to perform tasks that fall underthe purview of multiple WBS elements.Furthermore, a Rule of Thumb estimate is typically reported asa point estimate; there is no reported uncertainty around thepercentages used to generate an allocation. Even in the rarecase in which confidence intervals around mean percentages areprovided, there may be positive or negative correlations betweenWBS elements which can skew estimates.Here we attempt to address these problems by formulatingprobabilistic Rules of Thumb in which a distribution of allocationschemes, rather than a single allocation scheme, is generated.We use a bootstrap imputation method to simultaneouslyaccount for uncertainty in the missing data while using allavailable information contained in the dataset. The imputeddatasets are then input into a multivariate Bayesian modelwhich accounts for correlations between WBS elements andproperly accounts for uncertainty in the final Rule of Thumbpercentages and predictions. We describe the mathematicalmodel and provides snippets of R code utilizing the brms(Bayesian Regression Models using Stan) package. To illustratethis model, we generate a Bayesian Level 2 WBS Cost Rule ofThumb for MIDEX (Medium-Class Explorers) missions withdata extracted from NASA’s CADRe. We then compare thismethod’s performance with the classical Rule of Thumb method.

Hooke, Melissa A↗
NASA NTRS2018

Remote Sensing, Uncertainty Quantification, and a Theory of Data Systems; Workshop Report

The purpose of the workshop was to invite statisticians, applied mathematicians, computer scientists, data system architects, experts in remote sensing technology, and Climate and Earth System scientists to review, discuss, and plan research on issues related to large-scale, efficient analysis of distributed data using spatial statistical methods. Our motivation in organizing this event was to catalyze interchange among experts on the fast-emerging problem of analysis of distributed data. As part of SAMSI's 2017-2018 Program on Mathematical and Statistical Methods for Climate and the Earth System, a Working Group on Remote Sensing was established to address statistical and mathematical research problems in the analysis of remote sensing data. The Working Group has five subgroups: 1) Spatial Retrieval Methodology (the so-called \Spatial-X" subgroup); 2) Spatial Analysis for Hyperspectral Data (the so-called \Spatial-Y" subgroup); 3) Emulators for Complex Forward Models; 4) Optimization for Remote Sensing Retrievals; and 5) Theory of Data Systems (ToDS). The ToDS subgroup spent the first half of this academic year formulating a framework in which to consider the joint problem of a) optimizing statistical methods for environments where data are distributed and too large to move to a central location, and b) the design of data system infrastructures within which to implement those statistical methods. To x ideas, the Workshop focused on spatial statistical methods. To date there are many new spatial statistical methods designed with massive data sets in mind, in the literature. However, very few have been implemented for remote sensing data, and none have been implemented in operational settings like those used by NASA and NOAA. A major impediment to their use in these cases is that the data are not only massive, but are stored in different physical locations. These data must be brought together in some way in order to estimate spatial covariance functions, but moving data to a central location for analysis is tedious at best and impossible at worst. Some remote data reduction is almost certainly necessary, but how much? What are the consequences for inference? The fundamental issue underlying these questions is how to navigate the trade-space between costs and uncertainty in the estimates or inferences that are ultimately produced.

Braverman, Amy↗
NASA NTRS2003

Overview of Probabilistic Methods for SAE G-11 Meeting for Reliability and Uncertainty Quantification for DoD TACOM Initiative with SAE G-11 Division

The SAE G-11 RMSL Division and Probabilistic Methods Committee meeting during October 6-8 at the Best Western Sterling Inn, Sterling Heights (Detroit), Michigan is co-sponsored by US Army Tank-automotive & Armaments Command (TACOM). The meeting will provide an industry/government/academia forum to review RMSL technology; reliability and probabilistic technology; reliability-based design methods; software reliability; and maintainability standards. With over 100 members including members with national/international standing, the mission of the G-11's Probabilistic Methods Committee is to "enable/facilitate rapid deployment of probabilistic technology to enhance the competitiveness of our industries by better, faster, greener, smarter, affordable and reliable product development."

Singhal, Surendra N.↗
NASA NTRS2015

Random Predictor Models for Rigorous Uncertainty Quantification: Part 2

This and a companion paper propose techniques for constructing parametric mathematical models describing key features of the distribution of an output variable given input-output data. By contrast to standard models, which yield a single output value at each value of the input, Random Predictors Models (RPMs) yield a random variable at each value of the input. Optimization-based strategies for calculating RPMs having a polynomial dependency on the input and a linear dependency on the parameters are proposed. These formulations yield RPMs having various levels of fidelity in which the mean, the variance, and the range of the model's parameter, thus of the output, are prescribed. As such they encompass all RPMs conforming to these prescriptions. The RPMs are optimal in the sense that they yield the tightest predictions for which all (or, depending on the formulation, most) of the observations are less than a fixed number of standard deviations from the mean prediction. When the data satisfies mild stochastic assumptions, and the optimization problem(s) used to calculate the RPM is convex (or, when its solution coincides with the solution to an auxiliary convex problem), the model's reliability, which is the probability that a future observation would be within the predicted ranges, is bounded rigorously.

Crespo, Luis G.↗