Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “uncertainty estimation”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 253 records · Page 14
All sourcesNASA NTRSDOE OSTI
NASA NTRS2021

Nonlinear Unsteady Aerodynamic Modeling Using Empirical Orthogonal Functions

Empirical orthogonal function modeling is explained and applied to identify compact discrete-time nonlinear unsteady aerodynamic models from data generated by an unsteady three-dimensional compressible Navier-Stokes flow solver for an airfoil undergoing various pitching motions. Model structures, model parameter estimates, and model parameter uncertainty estimates for nondimensional lift, drag, and pitching moment coefficient models were determined autonomously and directly from the data. Prediction tests using data that were not used in the modeling process showed that the identified models exhibited excellent prediction capability, which is a strong indicator of an accurate model.

empirical↗
NASA NTRS2004

Neural network uncertainty assessment using Bayesian statistics: a remote sensing application

Neural network (NN) techniques have proved successful for many regression problems, in particular for remote sensing; however, uncertainty estimates are rarely provided. In this article, a Bayesian technique to evaluate uncertainties of the NN parameters (i.e., synaptic weights) is first presented. In contrast to more traditional approaches based on point estimation of the NN weights, we assess uncertainties on such estimates to monitor the robustness of the NN model. These theoretical developments are illustrated by applying them to the problem of retrieving surface skin temperature, microwave surface emissivities, and integrated water vapor content from a combined analysis of satellite microwave and infrared observations over land. The weight uncertainty estimates are then used to compute analytically the uncertainties in the network outputs (i.e., error bars and correlation structure of these errors). Such quantities are very important for evaluating any application of an NN model. The uncertainties on the NN Jacobians are then considered in the third part of this article. Used for regression fitting, NN models can be used effectively to represent highly nonlinear, multivariate functions. In this situation, most emphasis is put on estimating the output errors, but almost no attention has been given to errors associated with the internal structure of the regression model. The complex structure of dependency inside the NN is the essence of the model, and assessing its quality, coherency, and physical character makes all the difference between a blackbox model with small output errors and a reliable, robust, and physically coherent model. Such dependency structures are described to the first order by the NN Jacobians: they indicate the sensitivity of one output with respect to the inputs of the model for given input data. We use a Monte Carlo integration procedure to estimate the robustness of the NN Jacobians. A regularization strategy based on principal component analysis is proposed to suppress the multicollinearities in order to make these Jacobians robust and physically meaningful.

Neural Networks (Computer)↗
NASA NTRS2013

Adaptive Sampling of Spatiotemporal Phenomena with Optimization Criteria

This work was designed to find a way to optimally (or near optimally) sample spatiotemporal phenomena based on limited sensing capability, and to create a model that can be run to estimate uncertainties, as well as to estimate covariances. The goal was to maximize (or minimize) some function of the overall uncertainty. The uncertainties and covariances were modeled presuming a parametric distribution, and then the model was used to approximate the overall information gain, and consequently, the objective function from each potential sense. These candidate sensings were then crosschecked against operation costs and feasibility. Consequently, an operations plan was derived that combined both operational constraints/costs and sensing gain. Probabilistic modeling was used to perform an approximate inversion of the model, which enabled calculation of sensing gains, and subsequent combination with operational costs. This incorporation of operations models to assess cost and feasibility for specific classes of vehicles is unique.

Chien, Steve A.↗
NASA NTRS2001

Searching for the Beginning of the Ozone Turnaround Using a 22-Year Merged Satellite Data Set

We have used the data from six satellite instruments that measure the total column amount of ozone to construct a consistent merged data set extending from late 1978 into 2000. The keys to constructing a merged data set are to minimize potential drift of individual instruments and to accurately establish instrument-to-instrument offsets. We have used the short-wavelength D-pair measurements (306nm-313nm) of the SBUV and SBUV/2 instruments near the equator to establish a relatively drift-free record for these instruments. We have then used their overlap with the Nimbus 7 and EP TOMS instruments to establish the relative calibration of the various instruments. We have evaluated the drift uncertainty in our merged ozone data (MOD) set by examining both the individual instrument drift uncertainty and the uncertainty in establishing the instrument- to-instrument differences. We conclude that the instrumental drift uncertainty over the 22-year data record is 0.9 %/decade (2-sigma). We have compared our MOD record with 37 ground stations that have a continuous record over that time period. We have a mean drift with respect to the stations of +0.3 %/decade which is within 1-sigma of our uncertainty estimate. Using the satellite record as a transfer standard, we can estimate the capability of the ground instruments to establish satellite calibration. Adding the statistical variability of the station drifts with respect to the satellite to an estimate of the overall drift uncertainty of the world standard instrument, we conclude that the stations should be able to be used to establish the drift of the satellite data record to within and uncertainty of 0.6 %/decade (2-sigma). Adding to this an uncertainty due to the-incomplete global coverage of the stations, we conclude that the station data should be able to establish the global trend with an uncertainty of about 0.7 %/decade, slightly better than for the satellite record. We conclude that merging the two records together gives only a slight improvement in the uncertainty. Keeping them separate gives the greater confidence of two independent measures of the ozone trend and potential recovery. We fit the trend in our MOD record through May of 1991 and then extrapolated forward to see if the data at the end of the record was above the statistical model as a measure of ozone recovery as was done in the last WMO/UNEP assessment report. Because our data set drifts with respect to the ground-stations through May of 1991, we calculated a smaller global trend (-1.1 %/decade) than in the WMO/UNEP report. Our data in 1998 and 1999 was, on average 2 DU above the extrapolated statistical model with a 2-sigma uncertainty of 6 DU. For the combined mid-latitudes of the northern and southern hemispheres, the data was 5 DU above the extrapolated statistical model with a 2-sigma uncertainty of 10 DU. These may be signs of recovery, but they are still statistically insignificant.

Stolarski, Richard S.↗
NASA NTRS2019

Improvements in the Uncertainty Model in the Goddard Institute for Space Studies Surface Temperature (GISTEMP) Analysis

We outline a new and improved uncertainty analysis for the Goddard Institute of Space Studies (GISS) Surface Temperature product version 4 (GISTEMP v4). Historical spatial variations in surface temperature anomalies are derived from historical weather station data and ocean data from ships, buoys and other sensors. Uncertainties arise from measurement uncertainty, changes in spatial coverage of the station record, and systematic biases due to technology shifts and land cover changes. Previously published uncertainty estimates for GISTEMP included only the effect of incomplete station coverage. Here, we update this term using currently available spatial distributions of source data, state‐of‐the‐art reanalyses and incorporate independently derived estimates for ocean data processing, station homogenization and other structural biases. The resulting 95% uncertainties are near 0.05° C in the global annual mean for the last 50 years, and increase going back further in time reaching 0.15° C in 1880. In addition, we quantify the benefits and inherent uncertainty due to the GISTEMP interpolation and averaging method. We use the total uncertainties to estimate the probability for each record year in the GISTEMP to actually be the true record year (to that date), and conclude with 86% likelihood that 2016 was indeed the hottest year of the instrumental period (so far).

Nathan J L Lenssen↗
NASA NTRS1980

Effects of aerodynamic heating and TPS thermal performance uncertainties on the Shuttle Orbiter

A procedure for estimating uncertainties in the aerodynamic-heating and thermal protection system (TPS) thermal-performance methodologies developed for the Shuttle Orbiter is presented. This procedure is used in predicting uncertainty bands around expected or nominal TPS thermal responses for the Orbiter during entry. Individual flowfield and TPS parameters that make major contributions to these uncertainty bands are identified and, by statistical considerations, combined in a manner suitable for making engineering estimates of the TPS thermal confidence intervals and temperature margins relative to design limits. Thus, for a fixed TPS design, entry trajectories for future Orbiter missions can be shaped subject to both the thermal-margin and confidence-interval requirements. This procedure is illustrated by assessing the thermal margins offered by selected areas of the existing Orbiter TPS design for an entry trajectory typifying early flight test missions.

Goodrich, W. D.↗
NASA NTRS2023

Probabilistic Hydrological Estimation of LandSlides (PHELS): Global Ensemble Landslide Hazard Modelling

In this study we present a model for the global Probabilistic Hydrological Estimation of LandSlides (PHELS). PHELS estimates the daily hazard of hydrologically triggered landslides at a coarse spatial resolution of 36 km, by combining landslide susceptibility (LSS) and (percentiles of) hydrological variable(s). The latter include daily rainfall, a 7-day antecedent rainfall index (ARI7) or root-zone soil moisture content (rzmc) as hydrological predictor variables, or the combination of rainfall and rzmc. The hazard estimates with any of these predictor variables have areas under the receiver operating characteristic curve (AUC) above 0.68. The best performance was found with combined rainfall and rzmc predictors (AUC = 0.79), which resulted in the lowest number of missed alarms (especially during spring) and false alarms. Furthermore, PHELS provides hazard uncertainty estimates by generating ensemble simulations based on repeated sampling of LSS and the hydrological predictor variables. The estimated hazard uncertainty follows the behaviour of the input variable uncertainties, is about 13.6 % of the estimated hazard value on average across the globe and in time and is smallest for very low and very high hazard values.

Anne Felsberg↗
NASA NTRS2019

Uncertainties in Evapotranspiration Estimates over West Africa

An evapotranspiration (ET) ensemble composed of 36 land surface model (LSM) experiments and four diagnostic datasets (GLEAM, ALEXI, MOD16, and FLUXNET) is used to investigate uncertainties in ET estimate over five climate regions in West Africa. Diagnostic ET datasets show lower uncertainty estimates and smaller seasonal variations than the LSM-based ET values, particularly in the humid climate regions. Overall, the impact of the choice of LSMs and meteorological forcing datasets on the modeled ET rates increases from north to south. The LSM formulations and parameters have the largest impact on ET in humid regions, contributing to 90% of the ET uncertainty estimates. Precipitation contributes to the ET uncertainty primarily in arid regions. The LSM-based ET estimates are sensitive to the uncertainty of net radiation in arid region and precipitation in humid region. This study serves as support for better determining water availability for agriculture and livelihoods in Africa with earth observations and land surface models.

land surface model↗
NASA NTRS1977

Study of synthesis techniques for insensitive aircraft control systems

Insensitive flight control system design criteria was defined in terms of maximizing performance (handling qualities, RMS gust response, transient response, stability margins) over a defined parameter range. Wing load alleviation for the C-5A was chosen as a design problem. The C-5A model was a 79-state, two-control structure with uncertainties assumed to exist in dynamic pressure, structural damping and frequency, and the stability derivative, M sub w. Five new techniques (mismatch estimation, uncertainty weighting, finite dimensional inverse, maximum difficulty, dual Lyapunov) were developed. Six existing techniques (additive noise, minimax, multiplant, sensitivity vector augmentation, state dependent noise, residualization) and the mismatch estimation and uncertainty weighting techniques were synthesized and evaluated on the design example. Evaluation and comparison of these six techniques indicated that the minimax and the uncertainty weighting techniques were superior to the other six, and of these two, uncertainty weighting has lower computational requirements. Techniques based on the three remaining new concepts appear promising and are recommended for further research.

Harvey, C. A.↗
NASA NTRSDate not supplied

Differential Equation Approximation Using Gradient-Boosted Quantile Regression

The operation of cyber-physical-human (CPH) systems is subject to various epistemic and aleatory uncertainties. Overall trustworthiness of CPH systems relies on the trustworthiness of its components and their interactions. It is important that computational models comprising the cyber component of CPH provide predictions accompanied by a measure of confidence in model outcomes. Uncertainty quantification (UQ) and propagation are especially important in safety critical CPH systems. Gradient-boosted trees is a modeling approach capable both of learning the dynamics of a system and performing UQ. In this paper, we devise a method for using gradient boosting to learn the dynamics of a second order differential equation and estimate uncertainty at the same time. We do this by creating a custom loss function that trains the model to approximate the second derivative of a noisy time series, and to penalize based on a parameter that corresponds to the desired quantile. The resulting gradient boosting model can simulate stochastic trajectories of the system given a single starting point, that is, it can estimate both the expected trajectory and its uncertainty. We show that the uncertainty estimation is well calibrated and that the model can learn the dynamics even in the presence of noise. We demonstrate the approach on a simple cartpole system.

Autonomous systems↗
DOE OSTI2023

Gaussian process hydrodynamics

Abstract We present a Gaussian process (GP) approach, called Gaussian process hydrodynamics (GPH) for approximating the solution to the Euler and Navier-Stokes (NS) equations. Similar to smoothed particle hydrodynamics (SPH), GPH is a Lagrangian particle-based approach that involves the tracking of a finite number of particles transported by a flow. However, these particles do not represent mollified particles of matter but carry discrete/partial information about the continuous flow. Closure is achieved by placing a divergence-free GP prior ξ on the velocity field and conditioning it on the vorticity at the particle locations. Known physics (e.g., the Richardson cascade and velocity increment power laws) is incorporated into the GP prior by using physics-informed additive kernels. This is equivalent to expressing ξ as a sum of independent GPs ξ l , which we call modes, acting at different scales (each mode ξ l self-activates to represent the formation of eddies at the corresponding scales). This approach enables a quantitative analysis of the Richardson cascade through the analysis of the activation of these modes, and enables us to analyze coarse-grain turbulence statistically rather than deterministically. Because GPH is formulated by using the vorticity equations, it does not require solving a pressure equation. By enforcing incompressibility and fluid-structure boundary conditions through the selection of a kernel, GPH requires significantly fewer particles than SPH. Because GPH has a natural probabilistic interpretation, the numerical results come with uncertainty estimates, enabling their incorporation into an uncertainty quantification (UQ) pipeline and adding/removing particles (quanta of information) in an adapted manner. The proposed approach is suitable for analysis because it inherits the complexity of state-of-the-art solvers for dense kernel matrices and results in a natural definition of turbulence as information loss. Numerical experiments support the importance of selecting physics-informed kernels and illustrate the major impact of such kernels on the accuracy and stability. Because the proposed approach uses a Bayesian interpretation, it naturally enables data assimilation and predictions and estimations by mixing simulation data and experimental data.

Mathematics↗
DOE OSTI2022

Aggregation bias and its drivers in large‐scale flood loss estimation: A Massachusetts case study

Abstract Large‐scale estimations of flood losses are often based on spatially aggregated inputs. This makes risk assessments vulnerable to aggregation bias, a well‐studied, sometimes substantial outcome in analyses that model fine‐grained spatial phenomena at coarse spatial units. To evaluate this potential in the context of large‐scale flood risk assessments, we use data from a high‐resolution flood hazard model and structure inventory for over 1.3 million properties in Massachusetts and examine how prominent data aggregation approaches affect the magnitude and spatial distribution of flood loss estimates. All considered aggregation approaches rely on aggregate structure inventories but differ in whether flood hazard is also aggregated. We find that aggregating only structure inventories slightly underestimates overall losses (−10% bias), and when flood hazard data is spatially aggregated to even relatively small spatial units (census block), statewide aggregation bias can reach +366%. All aggregation‐based procedures fail to capture the spatial covariation of inputs distributions in the upper tails that disproportionately generate total expected losses. Our findings are robust to several key assumptions, add important context to published risk assessments and highlight opportunities to improve flood loss estimation uncertainty quantification.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2022

Scalable deep learning for watershed model calibration

Watershed models such as the Soil and Water Assessment Tool (SWAT) consist of high-dimensional physical and empirical parameters. These parameters often need to be estimated/calibrated through inverse modeling to produce reliable predictions on hydrological fluxes and states. Existing parameter estimation methods can be time consuming, inefficient, and computationally expensive for high-dimensional problems. In this paper, we present an accurate and robust method to calibrate the SWAT model (i.e., 20 parameters) using scalable deep learning (DL). We developed inverse models based on convolutional neural networks (CNN) to assimilate observed streamflow data and estimate the SWAT model parameters. Scalable hyperparameter tuning is performed using high-performance computing resources to identify the top 50 optimal neural network architectures. We used ensemble SWAT simulations to train, validate, and test the CNN models. We estimated the parameters of the SWAT model using observed streamflow data and assessed the impact of measurement errors on SWAT model calibration. We tested and validated the proposed scalable DL methodology on the American River Watershed, located in the Pacific Northwest-based Yakima River basin. Our results show that the CNN-based calibration is better than two popular parameter estimation methods (i.e., the generalized likelihood uncertainty estimation [GLUE] and the dynamically dimensioned search [DDS], which is a global optimization algorithm). For the set of parameters that are sensitive to the observations, our proposed method yields narrower ranges than the GLUE method but broader ranges than values produced using the DDS method within the sampling range even under high relative observational errors. The SWAT model calibration performance using the CNNs, GLUE, and DDS methods are compared using R 2 and a set of efficiency metrics, including Nash-Sutcliffe, logarithmic Nash-Sutcliffe, Kling-Gupta, modified Kling-Gupta, and non-parametric Kling-Gupta scores, computed on the observed and simulated watershed responses. The best CNN-based calibrated set has scores of 0.71, 0.75, 0.85, 0.85, 0.86, and 0.91. The best DDS-based calibrated set has scores of 0.62, 0.69, 0.8, 0.77, 0.79, and 0.82. The best GLUE-based calibrated set has scores of 0.56, 0.58, 0.71, 0.7, 0.71, and 0.8. The scores above show that the CNN-based calibration leads to more accurate low and high streamflow predictions than the GLUE and DDS sets. Our research demonstrates that the proposed method has high potential to improve our current practice in calibrating large-scale integrated hydrologic models.

54 ENVIRONMENTAL SCIENCES↗
NASA NTRS2004

Thin-Filament Pyrometry Developed for Measuring Temperatures in Flames

Many valuable advances in combustion science have come from observations of microgravity flames. This research is contributing to the improved efficiency and reduced emissions of practical combustors and is benefiting terrestrial and spacecraft fire safety. Unfortunately, difficulties associated with microgravity have prevented many types of measurements in microgravity flames. In particular, temperature measurements in flames are extremely important but have been limited in microgravity. A novel method of measuring temperatures in microgravity flames is being developed in-house at the National Center for Microgravity Research and the NASA Glenn Research Center and is described here. Called thin-filament pyrometry, it involves using a camera to determine the local gas temperature from the intensity of inserted fibers glowing in a flame. It is demonstrated here to provide accurate measurements of gas temperatures in a flame simultaneously at many locations. The experiment is shown. The flame is a laminar gas jet diffusion flame fueled by methane (CH4) flowing from a 14-mm round burner at a pressure of 1 atm. A coflowing stream of air is used to prevent flame flicker. Nine glowing fibers are visible. These fibers are made of silicon carbide (SiC) and have a diameter of 15 m (for comparison, the average human hair is 75 m in diameter). Because the fibers are so thin, they do little to disturb the flame and their temperature remains close to that of the local gas. The flame and glowing filaments were imaged with a digital black-and-white video camera. This camera has an imaging area of 1000 by 1000 pixels and a wide dynamic range of 12 bits. The resolution of the camera and optics was 0.1 mm. Optical filters were placed in front of the camera to limit incoming light to 750, 850, 950, and 1050 nm. Temperatures were measured in the same flame in the absence of fibers using 50-m Btype thermocouples. These thermocouples provide very accurate temperatures, but they generally are not useful in microgravity tests because they measure temperature at only one location at a time. Thermocouple measurements at a height of 11 mm above the burner were used to calibrate the thin-filament pyrometry system at all four wavelengths. This calibration was used to perform thin-filament pyrometry at other heights above the burner. One such profile is shown in this graph; this is for a height of 21 mm. The agreement between the pyrometry measurements and thermocouple results at this height is excellent in the range of 1000 to 2000 K, with an estimated uncertainty of 50 K and an estimated upper limit of 2500 K. Neither the thermocouple nor the thin-filament pyrometry temperatures have been corrected for radiation, but the correction is expected to be nearly the same for both methods. We anticipate that thin-filament pyrometry similar to that developed here will become an important diagnostic for studies of microgravity flames owing to its accuracy and its ability to simultaneously measure finely spaced temperatures.

Sunderland, Peter B.↗
DOE OSTI2020

Improvement of Quantification in Non-Destructive Characterization of Radioactive Waste Packages -Introduction to the Ideas of a Currently Started Project - 20197

For the characterization of radioactive waste packages (e.g. 200-L drums) with inhomogeneous activity and density distributions preferably non-destructive methods like segmented gamma scanning and transmission measurements are applied. While the first is used for identification of the radioactive inventory, both are necessary for a reliable activity quantification. Although at nuclear facilities, segmented gamma scanner are state-of-the-art and are mostly equipped with a vertically moveable transmission source the combining of this data and further a-priori information improving accuracy in data estimation giving realistic, reliable and traceable uncertainty estimations are improvable. This challenge is tackled by a R and D project funded by the German Federal Ministry of Education and Research (BMBF). The boundary conditions for this project are (i) taking into account the typical instrumentation available at nuclear facilities, (ii) minimize the overall measurement time and (iii) combining data from emission and transmission measurements by Bayesian methods. For the R and D project, being still in its initial stage, the basic ideas, the data used for evaluation and the result of a preliminary investigation demonstrating the proof of principle are presented. Additionally, some first aspects on spatial resolution in emission and transmission measurements are addressed. (authors)

12 MANAGEMENT OF RADIOACTIVE AND NON-RADIOACTIVE W↗
NASA NTRS2021

Validation Assessment for the Soil Moisture Active Passive (SMAP) Level 4 Carbon (L4_C) Data Product Version 5

The post-launch Cal/Val phase of the SMAP mission is guided by two primary objectives for each science product team: 1) to calibrate, verify, and improve the performance of the science algorithms, and 2) validate accuracies of the science data products as specified in the SMAP Level-1 mission science requirements. Algorithm science and product maintenance activities during the SMAP extended mission phase have also involved periodic algorithm calibration and product refinements to maintain or enhance product consistency and performance as well as science utility. This report provides an assessment of the latest (Version 5) SMAP Level 4 Carbon (L4_C) product. The L4_C Version 5 (v5) global record now spans more than six years (March 2015 – present) of SMAP operations and has benefited from five major reprocessing updates to the operational product. These reprocessing events and L4_C product release updates have incorporated various algorithm refinements and calibration adjustments to account for similar refinements to the upstream GEOS land model assimilation system, SMAP brightness temperatures, and MODIS vegetation inputs used for L4_C processing. The SMAP L4_C algorithms utilize a terrestrial carbon flux model informed by daily surface and root zone soil moisture information contributed from the SMAP Level 4 Soil Moisture (L4_SM) product along with optical remote sensing-based (e.g. MODIS-based) land cover and canopy fractional photosynthetic active radiation (fPAR), and other ancillary biophysical data. The carbon flux model estimates global daily net ecosystem CO2 exchange (NEE) and the component carbon fluxes, namely, vegetation gross primary production (GPP) and soil heterotrophic respiration (Rh). Other L4_C product elements include surface (~0-5 cm depth) soil organic carbon (SOC) stocks and associated environmental constraints to these processes, including soil moisture-related controls on GPP and ecosystem respiration (Kimball et al. 2014, Jones et al. 2017). The L4_C product addresses SMAP carbon cycle science objectives by: 1) providing a direct link between terrestrial carbon fluxes and underlying freeze/thaw and soil moisture-related constraints to these processes, 2) documenting primary connections between terrestrial water, energy and carbon cycles, and 3) improving understanding of terrestrial carbon sink activity. The SMAP L4_C algorithms and operational product are mature and at a CEOS Validation Stage 4 level (Jackson et al. 2012) based on extensive validation of the multi-year record against a diverse array of independent benchmarks, well characterized global performance, and systematic refinements gained from five major reprocessing events. There are no Level-1 mission science requirements for the L4_C product; however, self-imposed requirements have been established focusing on NEE as the primary product field for validation, and on demonstrating L4_C accuracy and success in meeting product science requirements (Jackson et al. 2012). The other L4_C product fields also have strong utility for carbon science applications (e.g., Liu et al. 2019, Endsley et al. 2020); however, analysis of these other fields is considered secondary relative to primary validation activities focusing on NEE. The L4_C targeted accuracy requirements are to meet or exceed a mean unbiased root-mean-square error (ubRMSE, or standard deviation of the error) for NEE of 1.6 g C m-2 d-1 and 30 g C m-2 yr-1, emphasizing northern (≥45°N) boreal and arctic ecosystems; this accuracy is similar to that of tower eddy covariance measurement-based observations (Baldocchi 2008). Methods used for the latest v5 L4_C product performance and validation assessment have been established from the SMAP Cal/Val plan and previous studies (Jackson et al. 2012, Jones et al. 2017) and include: 1) consistency evaluations of the product fields against earlier product releases (version 4 or earlier); 2) comparisons of daily carbon flux estimates with independent tower eddy 4covariance measurement-based daily carbon (CO2) flux observations from core tower validation sites (CVS); and 3) consistency checks against other global carbon products, including soil carbon inventory records, global GPP records derived from tower observation upscaling methods, and satellite-based observations of canopy solar induced chlorophyll fluorescence (SIF) as a surrogate for GPP. Metrics used to evaluate relative agreement between L4_C product fields and observational benchmarks include correlation (r-value), RMSE differences, bias and model sensitivity diagnostics. Following these validation criteria, the present report provides a validation assessment of the latest L4_C product release (v5). Detailed descriptions of the L4_C algorithm and additional global product accuracy and performance results are given elsewhere (Jones et al. 2017, Endsley et al. 2020). The v5 L4_C product replaces earlier product versions and continues to show: (i) accuracy and performance levels meeting or exceeding SMAP L4_C science requirements; (ii) improvement over the previous product version (version 4); and (iii) suitability for a diversity of science applications. Example L4_C applications from the recent literature include clarifying environmental trends and controls on the northern terrestrial carbon sink (Liu et al. 2019), diagnosing drought-related impacts on ecosystem productivity (Li et al. 2020), and regional monitoring of cropland conditions for projecting annual yields (Wurster et al. 2020). 2EXPECTED L4_C ALGORITHM AND PRODUCTPERFORMANCE The L4_C algorithm performance, including variance and uncertainty estimates of model outputs, was determined during the mission pre-launch phase through spatially explicit model sensitivity studies using available model inputs similar to those currently being used for operational production and evaluating the resulting model simulations over the observed range of northern (≥45 °N) and global conditions (Kimball et al. 2012, Entekhabi et al. 2014). The L4_C algorithm options were also evaluated during the mission prelaunch phase, including deriving canopy fPAR from lower order NDVI (Normalized Difference Vegetation Index) inputs in lieu of using MODIS (MOD15) fPAR; and including an explicit model representation of boreal fire disturbance recovery impacts. These results indicated that the L4_C accuracy requirements (i.e., NEE ubRMSE ≤ 30 g C m-2 yr-1or ≤ 1.6 g C m-2 d-1) could be met from the baseline algorithms over more than 82% and 89% of global and northern vegetated land areas, respectively (Yi et al. 2013, Kimball et al. 2014). The global L4_C algorithm error budget for NEE derived during the mission prelaunch phase indicated that the estimated NEE ubRMSE uncertainty is proportional to GPP and is therefore larger in higher biomass productivity areas, including forests and croplands (Kimball et al. 2014). Likewise, NEE ubRMSE uncertainty is expected to be lower in less-productive areas, including grasslands and shrublands. Expected model NEE ubRMSE levels were also generally within targeted accuracy levels for characteristically less-productive boreal and Arctic biomes, even though relative model error as a proportion of total productivity (NEE RMSE / GPP) may be large in these areas. The estimated NEE uncertainty was lower than expected in some warmer tropical high biomass productivity areas (e.g. Amazon rainforest) because of reduced low temperature and moisture constraints to the L4_C respiration calculations so that the bulk of model uncertainty is contributed by GPP in these areas. Model NEE uncertainty in the African Congo was estimated to

SMAP↗
DOE OSTI2023

Quantifying uncertainty for deep learning based forecasting and flow-reconstruction using neural architecture search ensembles

Classical problems in computational physics such as data-driven forecasting and signal reconstruction from sparse sensors have recently seen an explosion in deep neural network (DNN) based algorithmic approaches. However, most DNN models do not provide uncertainty estimates, which are crucial for establishing the trustworthiness of these techniques in downstream decision making tasks and scenarios. In recent years, ensemble-based methods have achieved significant success for the uncertainty quantification in DNNs on a number of benchmark problems. However, their performance on real-world applications remains under-explored. In this work, we present an automated approach to DNN discovery and demonstrate how this may also be utilized for ensemble-based uncertainty quantification. Specifically, we propose the use of a scalable neural and hyperparameter architecture search for discovering an ensemble of DNN models for complex dynamical systems. We highlight how the proposed method not only discovers high-performing neural network ensembles for our tasks, but also quantifies uncertainty seamlessly. This is achieved by using genetic algorithms and Bayesian optimization for sampling the search space of neural network architectures and hyperparameters. Subsequently, a model selection approach is used to identify candidate models for an ensemble set construction. Afterwards, a variance decomposition approach is used to estimate the uncertainty of the predictions from the ensemble. We demonstrate the feasibility of this framework for two tasks — forecasting from historical data and flow reconstruction from sparse sensors for the sea-surface temperature. In conclusion, we demonstrate superior performance from the ensemble in contrast with individual high-performing models and other benchmarks.

Deep ensembles↗