Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “uncertainty”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 919 records · Page 51

Integration of Uncertainty Quantification in a Model-Based Systems Analysis and Engineering Framework

This paper presents a technical approach to improve the confidence in the systems analysis process by integrating Uncertainty Quantification (UQ) techniques within a Model-Based Systems Analysis and Engineering (MBSA&E) framework. The MBSA&E architecture uses system models and multidisciplinary analytical solutions as central artifacts for system design and analysis. The integration of UQ enables engineers to assess and mitigate uncertainties associated with a system model, design parameters, and constraint inputs, leading to more complete design studies and further informed decision-making processes. The proposed approach leverages the strengths of MBSA&E and extends it with a UQ methodology to quantify uncertainties in the input parameters and to trace the uncertainties as they propagate throughout the system model. To demonstrate the effectiveness of an integrated MBSA&E-UQ approach, a case study involving a simplified analysis of a Transonic Truss-Braced Wing (TTBW) concept vehicle is performed. This integration enables a more comprehensive evaluation of system performance and behavior under uncertainty and a more robust approach for system design and analysis. Lastly, the paper addresses the challenges and considerations associated with integrating UQ into an MBSA&E framework.

systems analysis↗

Uncertainty Quantification and Sensitivity Analysis in Process-Structure-Property Simulations for Laser Powder Bed Fusion Additive Manufacturing

Process variations and process-induced defects like porosity cause significant uncertainty in the microstructure and mechanical behavior of additively manufactured metals. Establishing process-structure-property (PSP) relationships and quantifying uncertainty using experiments alone is costly, especially for structural applications where mechanical allowables must be established for qualification and certification. This work presents a PSP simulation framework for laser powder bed fusion with a focus on uncertainty quantification through probabilistic calibration and multi-fidelity uncertainty propagation. Motivated by phenomenological input parameters related to grain nucleation and growth that are difficult to characterize, a global sensitivity analysis (GSA) is completed. Through GSA, the most important input parameters are identified based on their influence on the statistical distributions of microstructural metrics that influence mechanical behavior, including grain size, morphology, and crystallographic texture. The results provide insight on what experiments are necessary to quantify and control PSP uncertainties, particularly those associated with the more challenging input parameters.

additive manufacturing↗

Reduction of Aircraft Noise Uncertainty for a Notional Supersonic Business Jet

NASA supported a study by the International Civil Aviation Organization on the environmental impact of adding supersonic aircraft to the existing global fleet by designing a notional 55-tonne supersonic business jet. The aircraft is referred to as the Supersonic Technology Concept Aeroplane (STCA) and has been used in a multitude of studies over the past few years. One of the many studies on this aircraft was the noise produced by the STCA during landing and take-off (LTO) operations. The LTO noise for the STCA was assessed using contemporary noise prediction tools that have been primarily developed for and utilized by subsonic aircraft, resulting in a high level of uncertainty for the STCA noise predictions. NASA has recently been conducting research and developing tools to reduce the uncertainty for airport noise predictions of supersonic aircraft. The initial focus of the uncertainty reduction was on the jet and inlet-radiated fan noise produced by supersonic engine systems since these sources are dominant during take-off and landing operations for supersonic jets. The results of these efforts are discussed and applied to the STCA model to update the airport noise predictions and associated uncertainty metrics. It is shown that by using new noise source prediction models, the overall system-level cumulative noise uncertainty of the study vehicle is reduced from a standard deviation of 7.8 EPNdB to 2.0 EPNdB.

uncertainty↗

Equipping Neural Network Surrogates with Uncertainty for Propagation in Physical Systems

Coarse-grained or filtered models typically rely on closure models to account for unresolved scales. For instance, large eddy simulation for modeling turbulent fluid flows explicitly resolves the largest scales, but requires modeling closure terms to account for the sub-filter scales. With the vast amount of data available from high-fidelity simulations, there are unique opportunities to leverage data-driven modeling techniques to formulate expressive and flexible closure models. Despite their flexibility, data-driven models struggle in domain shift settings, i.e. when deployed in configurations not captured in the training dataset. In particular, the efficacy of neural network surrogates is difficult to assess a priori due to the deterministic, point-estimate nature of predictions. In high-consequence applications, such models require reliable uncertainty estimates in the data-informed and out-of-distribution regimes. To quantify uncertainties in both regimes, we employ Bayesian neural networks which are able to capture both epistemic and aleatoric uncertainties. We will discuss challenges associated with the training and evaluation of these networks. Furthermore, we will discuss uncertainty embedding strategies to enable efficient sampling and propagation of uncertainty through high-fidelity simulations.

Bayesian neural networks↗

Uncertainty Visualization Challenges in Decision Systems with Ensemble Data & Surrogate Models: Preprint

Uncertainty visualization is a key component in translating important insights from ensemble data into actionable decision-making by visually conveying various aspects of uncertainty within a system. With the recent advent of fast surrogate models for computationally expensive simulations, users can interact with more aspects of data spaces than ever before. However, the integration of ensemble data with surrogate models in a decision-making tool brings up new challenges for uncertainty visualization, namely how to reconcile and communicate the new and different types of uncertainties brought in by surrogates and how to utilize these new data estimates in actionable ways. In this work, we examine these issues as they relate to high-dimensional data visualization, the integration of discrete datasets and the continuous representations of those datasets, and the unique difficulties associated with systems that allow users to iterate between input and output spaces. We assess the role of uncertainty visualization in facilitating intuitive and actionable interaction with ensemble data and surrogate models, and highlight key challenges in this new frontier of computational simulation.

ensemble visualization↗

Correlated Trajectory Uncertainty for Adaptive Sequential Decision Making

One of the great challenges with decision making tasks on real world systems is the fact that data is sparse and acquiring additional data is expensive. In these cases, it is often crucial to make a model of the environment to assist in making decisions. At the same time, limited data means that learned models are erroneous, making it just as important to equip the model with good predictive uncertainties. In the context of learning sequential decision making policies, these uncertainties can prove useful for informing which data to collect for the greatest improvement in policy performance \citep{mehta2021experimental, mehta2022exploration} or informing the policy about unsure regions of state and action space to avoid during test time \citep{yu2020mopo}. Additionally, assuming that realistic samples of the environment can be drawn, an adaptable policy can be trained that attempts to make optimal decisions for any given possible instance of the environment \citep{ghosh2022offline, chen2021offline}. In this work, we examine the so-called ``probabilistic neural network'' (PNN) model that is ubiquitous in model-based reinforcement learning (MBRL) works. We argue that while PNN models may have good marginal uncertainties, they form a distribution of non-smooth transition functions. Not only are these samples unrealistic and may hamper adaptability, but we also assert that this leads to poor uncertainty estimates when predicting multiple step trajectory estimates. To address this issue, we propose a simple sampling method that can be implemented on top of pre-existing models.We evaluate our sampling technique on a number of environments, including a realistic nuclear fusion task, and find that, not only do smooth transition function samples produce more calibrated uncertainties, but they also lead to better downstream performance for an adaptive policy.

Offline Reinforcement Learning↗

Uncertainty quantification of a deep learning fuel property prediction model

Deep learning models are being widely used in the field of combustion. Given the black-box nature of typical neural network based models, uncertainty quantification (UQ) is critical to ensure the reliability of predictions as well as the training datasets, and for a principled quantification of noise and its various sources. Deep learning surrogate models for predicting properties of chemical compounds and mixtures have been recently shown to be promising for enabling data-driven fuel design and optimization, with the ultimate goal of improving efficiency and lowering emissions from combustion engines. In this study, UQ is performed for a multi-task deep learning model that simultaneously predicts the research octane number (RON), Motor Octane Number (MON), and Yield Sooting Index (YSI) of pure components and multicomponent blends. The deep learning model is comprised of three smaller networks: Extractor 1, Extractor 2, and Predictor, and a mixing operator. The molecular fingerprints of individual components are encoded via Extractor 1 and Extractor 2, the mixing operator generates fingerprints for mixtures/blends based on linear mixing operation, and the predictor maps the fingerprint to the target properties. Two different classes of UQ methods, Monte Carlo ensemble methods and Bayesian neural networks (BNNs), are employed for quantifying the epistemic uncertainty. Combinations of Bernoulli and Gaussian distributions with DropConnect and DropOut techniques are explored as ensemble methods. All the DropConnect, DropOut and Bayesian layers are applied to the predictor network. Aleatoric uncertainty is modeled by assuming that each data point has an independent uncertainty associated with it. The results of the UQ study are further analyzed to compare the performance of BNN and ensemble methods. Although this study is confined to UQ of fuel property prediction, the methodologies are applicable to other deep learning frameworks that are being widely used in the combustion community.

33 ADVANCED PROPULSION SYSTEMS↗

Deep Direct-Use Feasibility Study Numerical Modeling and Uncertainty Analysis using iTOUGH2 for West Virginia University

To reduce the geothermal exploration risk, a feasibility study is performed for a deep direct-use system proposed at the West Virginia University (WVU) Morgantown campus. This study applies numerical simulations to investigate reservoir impedance and thermal production. Because of the great depth of the geothermal reservoir, few data are available to characterize reservoir features and properties. As a result, the study focuses on the following three aspects: 1. model choice for predicting reservoir impedance and thermal breakthrough: after investigating three potential models (one single permeability model and two dual permeability models) for flow through fractured rock, it is decided to use single permeability model for further analysis; 2. well placement (horizontal vs. vertical) options: horizontal well placement seems to be more robust to heterogeneity and the impedance is more acceptable; 3. Prediction uncertainty: the most influential parameters are identified using a First-Order-Second-Moment uncertainty propagation analysis, and the uncertain range of the model predictions is estimated by performing a Monte Carlo simulation. Heterogeneity has a large impact on the perdition, therefore, is considered in the predictive model and uncertainty analysis. The numerical model results and uncertainty analysis are used for economic analysis. The dataset submitted here support the described study. Manuscript is submitted to Geothermics, will be linked once paper is accepted.

15 GEOTHERMAL ENERGY↗

Need for Uncertainty Quantification in Predictions [Slides]

Natural and engineered systems are often modeled through mathematical equations using computer codes. Finite resources, inability to accurately control experimental conditions, encounter, or explore the entire parameter space of interest are a few reasons that scientists and engineers rely on numerical simulations for predictive capability. Associated with every prediction is an uncertainty, since our ability to simulate a physical process is limited by our knowledge of the physics, and the parameters. In the last two decades, the need for Uncertainty Quantification (UQ) has been recognized in several fields and we often ask questions about the risk or uncertainty associated with events and make decisions accordingly. In this talk, I will illustrate the effects of model and parametric uncertainties and the need to include them in our predictive capabilities.

97 MATHEMATICS AND COMPUTING↗

Application of Monte Carlo Methods to Perform Uncertainty and Sensitivity Analysis on Inverse Water-Rock Reactions with NETPATH

Methods were developed to quantify uncertainty and sensitivity for NETPATH inverse water-rock reaction models and to calculate dissolved inorganic carbon, carbon-14 groundwater travel times. The NETPATH models calculate upgradient groundwater mixing fractions that produce the downgradient target water chemistry along with amounts of mineral phases that are either precipitated or dissolved. Carbon-14 groundwater travel times are calculated based on the upgradient source-water fractions, carbonate mineral phase changes, and isotopic fractionation. Custom scripts and statistical code were developed for this study to facilitate modifying input parameters, running the NETPATH simulations, extracting relevant output, postprocessing the results, and producing graphs and summaries. The scripts read userspecified values for each constituent’s coefficient of variation, distribution, sensitivity parameter, maximum dissolution or precipitation amounts, and number of Monte Carlo simulations. Monte Carlo methods for analysis of parametric uncertainty assign a distribution to each uncertain variable, sample from those distributions, and evaluate the ensemble output. The uncertainty in input affected the variability of outputs, namely source-water mixing, phase dissolution and precipitation amounts, and carbon-14 travel time. Although NETPATH may provide models that satisfy the constraints, it is up to the geochemist to determine whether the results are geochemically reasonable. Two example water-rock reaction models from previous geochemical reports were considered in this study. Sensitivity analysis was also conducted to evaluate the change in output caused by a small change in input, one constituent at a time. Results were standardized to allow for sensitivity comparisons across all inputs, which results in a representative value for each scenario. The approach yielded insight into the uncertainty in water-rock reactions and travel times. For example, there was little variation in source-water fraction between the deterministic and Monte Carlo approaches, and therefore, little variation in travel times between approaches. Sensitivity analysis proved very useful for identifying the most important input constraints (dissolved-ion concentrations), which can reveal the variables that have the most influence on source-water fractions and carbon-14 travel times. Once these variables are determined, more focused effort can be applied to determining the proper distribution for each constraint. Second, Monte Carlo results for water-rock reaction modeling showed discrete and nonunique results. The NETPATH models provide the solutions that satisfy the constraints of upgradient and downgradient water chemistry. There can exist multiple, discrete solutions for any scenario and these discrete solutions cause grouping of results. As a result, the variability in output may not easily be represented by a single distribution or a mean and variance and care should be taken in the interpretation and reporting of results.

phase dissolution and precipitation amounts↗

Large uncertainty in permafrost carbon stocks due to hillslope soil deposits

Here, northern circumpolar permafrost soils contain more than a third of the global Soil Organic Carbon pool (SOC). The sensitivity of this carbon pool to a changing climate is a primary source of uncertainty in simulationbased climate projections. These projections, however, do not account for the accumulation of soil deposits at the base of hillslopes (hill-toes), and the influence of this accumulation on the distribution, sequestration, and decomposition of SOC in landscapes affected by permafrost. Here we combine topographic models with soil-profile data and topographic analysis to evaluate the quantity and uncertainty of SOC mass stored in perennially frozen hill-toe soil deposits. We show that in Alaska this SOC mass introduces an uncertainty that is > 200% than state-wide estimates of SOC stocks (77 PgC), and that a similarly large uncertainty may also pertain at a circumpolar scale. Soil sampling and geophysical-imaging efforts that target hill-toe deposits can help constrain this large uncertainty.

58 GEOSCIENCES↗

Estimating uncertainty in ambient and saturation nutrient uptake metrics from nutrient pulse releases in stream ecosystems

Nutrient spiraling is an important ecosystem process characterizing nutrient transport and uptake in streams. Various nutrient addition methods are used to estimate uptake metrics; however, uncertainty in the metrics is not often evaluated. A method was developed to quantify uncertainty in ambient and saturation nutrient uptake metrics estimated from saturating pulse nutrient additions (Tracer Additions for Spiraling Curve Characterization; TASCC). Using a Monte Carlo (MC) approach, the 95% confidence interval (CI) was estimated for ambient uptake lengths (S w-amb ) and maximum areal uptake rates (U max ) based on 100,000 datasets generated from each of four nitrogen and five phosphorous TASCC experiments conducted seasonally in a forest stream in eastern Tennessee, U.S.A. Uncertainty estimates from the MC approach were compared to the CIs estimated from ordinary least squares (OLS) and non-linear least squares (NLS) models used to calculate S w-amb and U max , respectively, from the TASCC method. The CIs for Sw-amb and Umax were large, but were not consistently larger using the MC method. Despite the large CIs, significant differences (based on nonoverlapping CIs) in nutrient metrics among seasons were found with more significant differences using the OLS/NLS vs. the MC method. Lastly, we suggest that the MC approach is a robust way to estimate uncertainty, as the calculation of S w-amb and U max violates assumptions of OLS/NLS while the MC approach is free of these assumptions. The MC approach can be applied to other ecosystem metrics that are calculated from multiple parameters, providing a more robust estimate of these metrics and their associated uncertainties.

54 ENVIRONMENTAL SCIENCES↗

Filtering adaptive output feedback control for multivariable nonlinear systems with mismatched uncertainties and unmodeled dynamics

This article synthesizes a filtering adaptive output feedback controller for multivariable nonlinear systems with mismatched uncertainties and unmodeled dynamics. The multivariable nonlinear systems under consideration have both matched and mismatched uncertainties, which satisfy the semiglobal Lipschitz condition. The unmodeled dynamics are bounded-input bounded-output stable. By adopting an estimation/cancellation strategy, a piecewise constant adaptive law drives the estimation error to zero at every time instant, which yields the adaptive parameters; a disturbance rejection control law is designed to compensate the nonlinear uncertainties within the bandwidth of low-pass filters. The matched uncertainties are cancelled directly by adopting their opposite in the control signal, while a dynamic inversion of the system is required to eliminate the effect of the mismatched uncertainties on the output. A feedforward control law is designed to track a given command. Since the virtual reference system defines the best performance that can be achieved by the closed-loop system, the uniform performance bounds are derived for the states and control signals via comparison. Both numerical and practical examples are provided to illustrate the effectiveness of the proposed filtering adaptive output feedback control architecture, comparisons with the model reference adaptive control demonstrates the superiority of the proposed control method.

filtering adaptive control, mismatched uncertainit↗

Reducing the top quark mass uncertainty with jet grooming

The measurement of the top quark mass has large systematic uncertainties coming from the Monte Carlo simulations that are used to match theory and experiment. We explore how much that uncertainty can be reduced by using jet grooming procedures. Using the ATLAS A14 tunes of pythia, we estimate the uncertainty from the choice of tuning parameters in what is meant by the Monte Carlo mass to be around 530 MeV without any corrections. This uncertainty can be reduced by 60% to 200 MeV by calibrating to the W mass and by 70% to 140 MeV by additionally applying soft-drop jet grooming (or to 170 MeV using trimming). At e + e – colliders, the associated uncertainty is around 110 MeV, reducing to 50 MeV after calibrating to the W mass. By analyzing the tuning parameters, we conclude that the importance of jet grooming after calibrating to the W -mass is to reduce sensitivity to the underlying event.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗

Uncertainty amplification due to density/refractive index gradients in background-oriented schlieren experiments

Here, we theoretically analyze the effect of density/refractive index gradients on the measurement precision of background-oriented schlieren (BOS) experiments by deriving the Cramer–Rao lower bound (CRLB) for the 2D centroid estimation process. A model is derived for the diffraction limited image of a dot viewed through a medium containing density gradients that includes the effect of the experimental parameters such as the magnification and f-number. It is shown using the model that nonlinearities in the density gradient field lead to blurring of the dot image. This blurring amplifies the effect of image noise on the centroid estimation process, leading to an increase in the CRLB and a decrease in the measurement precision. The ratio of position uncertainties of a dot in the reference and gradient images is shown to be a function of the ratio of the dot diameters and dot intensities. We termed this parameter the amplification ratio (A F ), and a methodology for reporting position uncertainties in tracking-based BOS measurements is proposed. The theoretical predictions of the dot position estimation variance from the CRLB are compared to ray tracing simulations, and agreement is obtained. The uncertainty amplification is also demonstrated on experimental BOS images of flow induced by a spark discharge, where it is seen that regions of high amplification ratio correspond to regions of density gradients. This analysis elucidates the dependence of the position uncertainty on density and refractive index gradient-induced distortion parameters, provides a methodology for accounting its effect on uncertainty quantification and provides a framework for optimizing experiment design.

42 ENGINEERING↗

Cloud fraction at the ARM SGP site: Reducing uncertainty with self-organizing maps

Instrument downtime leads to uncertainty in the monthly and annual record of cloud fraction (CF), making it difficult to perform time series analyses of cloud properties and perform detailed evaluations of model simulations. As cloud occurrence is partially controlled by the large-scale atmospheric environment, this knowledge is used to reduce uncertainties in the instrument record. Synoptic patterns diagnosed from the North American Regional Reanalysis (NARR) during the period 1997–2010 are classified using a competitive neural network known as the self-organizing map (SOM). The classified synoptic states are then compared to the Atmospheric Radiation Measurement (ARM) Southern Great Plains (SGP) instrument record to determine the expected CF. A number of SOMs are tested to understand how the number of classes and the period of classifications impact the relationship between classified states and CFs. Bootstrapping is utilized to quantify the uncertainty of the instrument record when statistical information from the SOM is included. Although all SOMs significantly reduce the uncertainty of the CF record calculated in Kennedy et al. (Theor Appl Climatol 115:91–105, 2014), SOMs with a large number of classes and separated by month are required to produce the lowest uncertainty and best agreement with the annual cycle of CF. Lastly, this result may be due to a manifestation of seasonally dependent biases in NARR.

54 ENVIRONMENTAL SCIENCES↗

Quantifying Uncertainties in Heat Capacity Measurements of Molten Salts Determined Using Differential Scanning Calorimetry

Uncertainty in specific heat capacity values of a molten salt determined by using differential scanning calorimetry (DSC) was assessed based on the precision of replicate measurements of heat flows used in the calculation and effects of corrections that are commonly made to heat flow measurements. The ratio method of determining heat capacity was applied using the results of replicate heat flow measurements made with two empty cells, a sapphire reference material, and three samples of a doped NaCl-UCl 3 salt mixture. Replicate measurements with empty cells were used to quantify the effects of system instabilities and sensitivities on the measured heat flows of sapphire and salt. The combined effects of uncertainties in individual heat flow measurements made with blank cells using this system were quantified to be 2.6 μV based on isothermal holds before and after the scan, with cell placement adding the greatest uncertainty. This value was used as the tolerance for accepting background-corrected heat flows measured with sapphire and salt to calculate the specific heat capacity. The acceptable heat flows measured for sapphire and salt over the temperature range of 540 to 725 °C resulted in calculated specific heat capacity values ranging from 0.53 to 0.91 J g −1 K −1 with an overall average value of 0.70 J g −1 K −1 and an uncertainty of 0.22 J g −1 K −1 at the 99 % confidence level. The combined uncertainty in the specific heat capacity masked detection of any effect of temperature or salt composition that occurred.

11 NUCLEAR FUEL CYCLE AND FUEL MATERIALS↗

Development Optimization and Uncertainty Analysis Methods for Oil and Gas Reservoirs

Uncertainty complicates the development optimization of oil and gas exploration and production projects, but methods have been devised to analyze uncertainty and its impact on optimal decision-making. This paper compares two methods for development optimization and uncertainty analysis: Monte Carlo (MC) simulation and stochastic programming. Two example problems for a gas field development and an oilfield development are solved and discussed to elaborate the advantages and disadvantages of each method. Development optimization involves decisions regarding the configuration of initial capital investment and subsequent operational decisions. Uncertainty analysis involves the quantification of the impact of uncertain parameters on the optimum design concept. The gas field development problem is designed to highlight the differences in the implementation of the two methods and to show that both methods yield the exact same optimum design. The results show that both MC optimization and stochastic programming provide unique benefits, and that the choice of method depends on the goal of the analysis. While the MC method generates more useful information, along with the optimum design configuration, the stochastic programming method is more computationally efficient in determining the optimal solution. Reservoirs comprise multiple compartments and layers with multiphase flow of oil, water, and gas. We present a workflow for development optimization under uncertainty for these reservoirs, and solve an example on the design optimization of a multicompartment, multilayer oilfield development.

02 PETROLEUM↗