Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “ensemble Kalman filter”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 55 records · Page 3
All sourcesNASA NTRSDOE OSTI
DOE OSTI2020

A Novel Modeling Framework for Computationally Efficient and Accurate Real-Time Ensemble Flood Forecasting With Uncertainty Quantification

A novel modeling framework that simultaneously improves accuracy, predictability, and computational efficiency is presented. It embraces the benefits of three modeling techniques integrated together for the first time: surrogate modeling, parameter inference, and data assimilation. The use of polynomial chaos expansion (PCE) surrogates significantly decreases computational time. Parameter inference allows for model faster convergence, reduced uncertainty, and superior accuracy of simulated results. Ensemble Kalman filters assimilate errors that occur during forecasting. To examine the applicability and effectiveness of the integrated framework, we developed 18 approaches according to how surrogate models are constructed, what type of parameter distributions are used as model inputs, and whether model parameters are updated during the data assimilation procedure. We conclude that (1) PCE must be built over various forcing and flow conditions, and in contrast to previous studies, it does not need to be rebuilt at each time step; (2) model parameter specification that relies on constrained, posterior information of parameters (so-called Selected specification) can significantly improve forecasting performance and reduce uncertainty bounds compared to Random specification using prior information of parameters; and (3) no substantial differences in results exist between single and dual ensemble Kalman filters, but the latter better simulates flood peaks. The use of PCE effectively compensates for the computational load added by the parameter inference and data assimilation (up to ~80 times faster). Therefore, the presented approach contributes to a shift in modeling paradigm arguing that complex, high-fidelity hydrologic and hydraulic models should be increasingly adopted for real-time and ensemble flood forecasting.

54 ENVIRONMENTAL SCIENCES↗
NASA NTRSDate not supplied

Assimilation of Satellite Sea Surface Temperature and Salinity Using the Local Ensemble Transform Kalman Filter Applied to the NASA GEOS Earth System Model

The NASA GEOS-S2S-2 system (Molod et al., 2020) uses the SST relaxation with the Level-4 SST products while the latest GEOS GEOS-S2S-3 system directly assimilates the Level-4 SST in the data assimilation step, yet no study has been conducted to understand the impact of this switch on the GEOS coupled analysis and short-range forecast. In addition, the GEOS-S2S-2/3 systems use the ocean model MOM5. Recently, the ocean model MOM6 is added to the GEOS ESM. This poster presents results of the first coupling of the ensemble data assimilation system to the GEOS MOM6.

Cheng Da↗
NASA NTRSDate not supplied

Examination of L-Band Brightness Temperature Forecasts in the SMAP Level-4 Soil Moisture Analysis

The NASA Soil Moisture Active Passive (SMAP) mission [4] has been providing L-band (1.4 GHz) passive microwave brightness temperature (Tb) observations since April 2015. By assimilating the Tb observations into the NASA Catchment land surface model [5] using a spatially distributed ensemble Kalman filter, the NASA Global Modeling and Assimilation Office generates the SMAP Level-4 Soil Moisture (L4_SM) product, which provides global, 3-hourly, 9-km resolution estimates of surface (0-5 cm) and root-zone (0-100 cm) soil moisture with ~2.5-day latency for use in research and applications [6]. The L4_SM product also includes estimates of soil temperature, land surface fluxes, and assimilation diagnostics such as the model forecast and observed Tb values [7]. The output from the L4_SM system is routinely monitored by the L4_SM team. Such monitoring provides valuable information; instances of unusually large Tb observation-minus-forecast (O-F) residuals can indicate events during which soil moisture conditions are poorly described in the land modeling system [7,8]. For example, repeated occurrences of very large Tb O-F values in central Australia were traced back to deficiencies in the gauge-based precipitation product used in the land modeling system through L4_SM Version 5; this discovery prompted the use, beginning in Version 6, of satellite- and gauge-based precipitation observations outside of North America [9]. Recently, a statistical analysis of the Tb O-F residuals revealed systematic errors in the “tau-omega” L-band radiative transfer model that converts the land-model simulated soil moisture and temperature into the forecast Tb prior to the L4_SM analysis. The L4_SM analysis is built on the ensemble Kalman filter and assumes unbiased forecast errors. The presence of systematic Tb forecast errors could thus adversely impact the quality of the analyzed soil moisture. In this paper, we examine the Tb O-F residuals of the latest Version 7 L4_SM data (Science Version ID Vv7030 and Vv7032) [10,11,12]. In the L4_SM Version 7 algorithm, key parameters of the L-band radiative transfer model, including soil roughness, scattering albedo, and a (seasonally varying) climatology of vegetation opacity, are obtained from the SMAP Level-2 Radiometer retrieval product (Version 5) [1]. Additionally, we also use ground measurements of surface soil moisture and soil temperature from the SMAP core validation sites as in situ reference of soil conditions [2,3]. As part of the L4_SM system calibration, the seasonally-varying bias between the model forecast Tb and the observed values is removed prior to the assimilation of the SMAP Tb observations [7]. That is, the L4_SM system is designed to only correct errors in synoptic-scale and interannual variations from the long-term mean seasonal cycle while maintaining the model’s (potentially erroneous) climatology. The Tb O-F residuals examined here are thus computed using SMAP Tb observations after they are rescaled to the mean seasonal cycle of Tb from the modeling system without SMAP data assimilation. Consequently, the long-term average of the Tb O-F residuals has a global mean of only 0.13 K and locally small values, ranging from -1 to 3 K. Despite the small time-average values of the Tb O-F residuals, the model forecast Tb was nevertheless found to exhibit undesirable systematic errors. At some locations, the time-average Tb O-F values strongly depend on surface soil moisture (SM). At the Yanco SMAP core validation site, for example, the Tb O-F residuals typically range from 5 to 15 K under dry soil moisture conditions (SM < 0.15 m3 m-3) yet are predominantly negative under wet soil moisture conditions (SM > 0.25 m3 m-3), with values ranging from 0 to -40 K. This results in soil moisture analysis increments that persistently make the soil drier under dry SM conditions and persistently make the soil wetter under wet SM conditions, suggesting an error in the dynamic range of the simulated Tb, soil moisture or soil temperature. In this paper, we describe the higher-order systematic Tb forecast errors in more detail, examine their impact on the L4_SM product quality, and explore potential avenues to further improve the L4_SM algorithm.

Rolf Reichle↗
NASA NTRSDate not supplied

Non-Gaussian Ensemble Filtering and Adaptive Inflation for Soil Moisture Data Assimilation

The rank histogram filter (RHF) and the ensemble Kalman filter (EnKF) are assessed for soil moisture estimation using perfect model (identical twin) synthetic data assimilation experiments. The primary motivation is to gauge the impact on analysis quality attributable to the consideration of non-Gaussian forecast error distributions. Using the NASA Catchment land surface model, the two filters are compared at 18 globally distributed single-catchment locations for a 10-yr experiment period. It is shown that both filters yield adequate estimates of soil moisture, with the RHF having a small but significant performance advantage. Most notably, the RHF consistently increases the normalized information contribution (NIC) score of the mean absolute bias by 0.05 over that of the EnKF for surface, root-zone, and profile soil moisture. The RHF also increases the NIC score for the anomaly correlation of surface soil moisture by 0.02 over that of the EnKF (at a 5% significance level). Results additionally demonstrate that the performance of both filters is somewhat improved when the ensemble priors are adaptively inflated to offset the negative effects of systematic errors.

Rolf Reichle↗
NASA NTRS2023

Non-Gaussian Ensemble Filtering and Adaptive Inflation for Soil Moisture Data Assimilation

The rank histogram filter (RHF) and the ensemble Kalman filter (EnKF) are assessed for soil moisture estimation using perfect model (identical twin) synthetic data assimilation experiments. The primary motivation is to gauge the impact on analysis quality attributable to the consideration of non-Gaussian forecast error distributions. Using the NASA Catchment land surface model, the two filters are compared at 18 globally-distributed single-catchment locations for a 10-year experiment period. It is shown that both filters yield adequate estimates of soil moisture, with the RHF having a small but significant performance advantage. Most notably, the RHF systematically increases the normalized information contribution (NIC) score of the mean absolute bias by 0.05 over that of the EnKF for surface, root-zone and profile soil moisture. The RHF also increases the NIC score for the anomaly correlation of surface soil moisture by 0.02 over that of the EnKF (at a 5% significance level). Results also demonstrate that the performance of both filters is somewhat improved when the ensemble priors are adaptively inflated to offset the negative effects of systematic errors.

Soil moisture↗
NASA NTRS2016

Ensemble-Based Assimilation of Aerosol Observations in GEOS-5

MERRA-2 is the latest Aerosol Reanalysis produced at NASA's Global Modeling Assimilation Office (GMAO) from 1979 to present. This reanalysis is based on a version of the GEOS-5 model radiatively coupled to GOCART aerosols and includes assimilation of bias corrected Aerosol Optical Depth (AOD) from AVHRR over ocean, MODIS sensors on both Terra and Aqua satellites, MISR over bright surfaces and AERONET data. In order to assimilate lidar profiles of aerosols, we are updating the aerosol component of our assimilation system to an Ensemble Kalman Filter (EnKF) type of scheme using ensembles generated routinely by the meteorological assimilation. Following the work performed with the first NASA's aerosol reanalysis (MERRAero), we first validate the vertical structure of MERRA-2 aerosol assimilated fields using CALIOP data over regions of particular interest during 2008.

LDE↗
DOE OSTI2022

Physics-Informed Gaussian Process Regression for States Estimation and Forecasting in Power Grids

Real-time state estimation and forecasting are critical for the efficient operation of power grids. In this paper, a physics-informed Gaussian process regression (PhI-GPR) method is presented and used for forecasting and estimating the phase angle, angular speed, and wind mechanical power of a three-generator power grid system using sparse measurements. In standard data-driven Gaussian process regression (GPR), parameterized models for the prior statistics are fit by maximizing the marginal likelihood of observed data. In the PhI-GPR method, we propose to compute the prior statistics offline by solving stochastic differential equations (SDEs) governing the power grid dynamics. The short-term forecast of a power grid system dominated by wind generation is complicated by the stochastic nature of the wind and the resulting uncertainty in wind mechanical power. Here, we assume that the power grid dynamics are governed by swing equations, with the wind mechanical power fluctuating randomly in time. We solve these equations for the mean and covariances of the power grid states using the Monte Carlo simulation method. We demonstrate that the proposed PhI-GPR method can accurately forecast and estimate observed and unobserved states. For the considered problem, PhI-GPR has computational advantages over the ensemble Kalman filter (EnKF) method: In PhI-GPR, ensembles are computed offline and independently of the data acquisition process, whereas for EnFK, ensembles are computed online with data acquisition, rendering real-time forecast more challenging. We also demonstrate that the PhI-GPR forecast is more accurate than the EnKF forecast when the random mechanical wind power is non-Markovian. In contrast, the two methods produce similar forecasts for the Markovian mechanical wind power. For observed states, we show that PhI-GPR provides a forecast comparable to the standard data-driven GPR; both forecasts are significantly more accurate than the autoregressive integrated moving average (ARIMA) forecast. We also show that the ARIMA forecast is more sensitive to observation frequency and measurement errors than the PhI-GPR forecast.

24 POWER TRANSMISSION AND DISTRIBUTION↗
NASA NTRS2011

Reduced Kalman Filters for Clock Ensembles

This paper summarizes the author's work ontimescales based on Kalman filters that act upon the clock comparisons. The natural Kalman timescale algorithm tends to optimize long-term timescale stability at the expense of short-term stability. By subjecting each post-measurement error covariance matrix to a non-transparent reduction operation, one obtains corrected clocks with improved short-term stability and little sacrifice of long-term stability.

Kalman filter↗
DOE OSTI2021

Assimilation of X-band Phased-Array Radar Data with EnKF for the Analysis and Warning Forecast of a Tornadic storm

The impact of assimilating China's operational X-band Phased-Array radar's (X-PAR) data on the analysis and warning forecast of the vortex structure and intensity of the June 8, 2018 Foshan, Guangdong province, tornadic storm was investigated for the first time using an Ensemble Kalman Filter (EnKF) data assimilation system. Both radar radial velocity (Vr) and reflectivity (Z) from two S-band operational radars and one X-PAR were assimilated. Deterministic forecasts were launched every 6 min from 05:42 UTC (20 min before the tornado touched down) to 06:00 UTC from the EnKF mean analysis field. Five experiments were conducted to examine the added capability of Z assimilation of the EnKF system, and to investigate the impact of assimilating X-PAR data on the analysis and prediction of the tornadic storm. Compared to the experiment without Z assimilation, the assimilation of Z reduced the analysis error and greatly reduced the forecast error of Z. Furthermore, the assimilation of X-PAR data greatly improved the vortex structure of the tornadic storm at low levels, and improved the intensity of the rear inflow of the tornadic storm, especially with a higher assimilation frequency. Compared to the experiments without X-PAR data assimilation, assimilating X-PAR data improved the predictability of tornadic storm.

54 ENVIRONMENTAL SCIENCES↗
NASA NTRS2022

Assimilation of SMAP Products for Improving Streamflow Simulations over Tropical Climate Region—Is Spatial Information More Important Than Temporal Information?

Streamflow is one of the key variables in the hydrological cycle. Simulation and forecasting of streamflow are challenging tasks for hydrologists, especially in sparsely gauged areas. Coarse spatial resolution remote sensing soil moisture products (equal to or larger than 9 km) are often assimilated into hydrological models to improve streamflow simulation in large catchments. This study uses the Ensemble Kalman Filter (EnKF) technique to assimilate SMAP soil moisture products at the coarse spatial resolution of 9 km (SMAP 9 km), and downscaled SMAP soil moisture product at the higher spatial resolution of 1 km (SMAP 1 km), into the Soil and Water Assessment Tool (SWAT) to investigate the usefulness of different spatial and temporal resolutions of remotely sensed soil moisture products in streamflow simulation and forecasting. The experiment was set up for eight catchments across the tropical climate of Vietnam, with varying catchment areas from 267 to 6430 km^2 during the period 2017–2019. We comprehensively evaluated the EnKF-based SWAT model in simulating streamflow at low, average, and high flow. Our results indicated that high-spatial resolution of downscaled SMAP 1 km is more beneficial in the data assimilation framework in aiding the accuracy of streamflow simulation, as compared to that of SMAP 9 km, especially for the small catchments. Our analysis on the impact of observation resolution also indicates that the improvement in the streamflow simulation with data assimilation is more significant at catchments where downscaled SMAP 1 km has fewer missing observations. This study is helpful for adding more understanding of performances of soil moisture data assimilation based hydrological modelling over the tropical climate region, and exhibits the potential use of remote sensing data assimilation in hydrology.

soil moisture↗
NASA NTRS2022

The Joint Assimilation of Remotely Sensed Leaf Area Index and Surface Soil Moisture into a Land Surface Model

This work tests the hypothesis that jointly assimilating satellite observations of leaf area index and surface soil moisture into a land surface model improves the estimation of land vegetation and water variables. An Ensemble Kalman Filter is used to test such hypothesis across the Contiguous United States during April 2015 – December 2018. The performance of the proposed methodology is assessed for several modeled vegetation and water variables (evapotranspiration, net ecosystem exchange, and soil moisture) in terms of random errors and anomaly correlation coefficients against a set of independent validation datasets (i.e., Global Land Evaporation Amsterdam Model, FLUXCOM, and International Soil Moisture Network). Results show that the assimilation of leaf area index mostly improves the estimation of evapotranspiration and net ecosystem exchange, whereas the assimilation of surface soil moisture alone improves surface soil moisture content, especially in the western US, in terms of both root mean squared error and anomaly correlation coefficient. The joint assimilation of vegetation and soil moisture information combines the results of individual vegetation and soil moisture assimilations and reduces errors (and increases correlations with the reference datasets) in evapotranspiration, net ecosystem exchange, and surface soil moisture simulated by the land surface model. However, because soil moisture satellite observations only provide information on the water content in the top 5 cm of the soil column, the impact of the proposed data assimilation technique on root zone soil moisture is limited. This work moves one step forward in the direction of improving our estimation and understanding of land surface interactions using a multi-variate data assimilation approach, which can be particularly useful in regions of the world where ground observations are sparse or missing altogether.

Data assimilation↗
NASA NTRS2009

The Principle of Energetic Consistency

A basic result in estimation theory is that the minimum variance estimate of the dynamical state, given the observations, is the conditional mean estimate. This result holds independently of the specifics of any dynamical or observation nonlinearity or stochasticity, requiring only that the probability density function of the state, conditioned on the observations, has two moments. For nonlinear dynamics that conserve a total energy, this general result implies the principle of energetic consistency: if the dynamical variables are taken to be the natural energy variables, then the sum of the total energy of the conditional mean and the trace of the conditional covariance matrix (the total variance) is constant between observations. Ensemble Kalman filtering methods are designed to approximate the evolution of the conditional mean and covariance matrix. For them the principle of energetic consistency holds independently of ensemble size, even with covariance localization. However, full Kalman filter experiments with advection dynamics have shown that a small amount of numerical dissipation can cause a large, state-dependent loss of total variance, to the detriment of filter performance. The principle of energetic consistency offers a simple way to test whether this spurious loss of variance limits ensemble filter performance in full-blown applications. The classical second-moment closure (third-moment discard) equations also satisfy the principle of energetic consistency, independently of the rank of the conditional covariance matrix. Low-rank approximation of these equations offers an energetically consistent, computationally viable alternative to ensemble filtering. Current formulations of long-window, weak-constraint, four-dimensional variational methods are designed to approximate the conditional mode rather than the conditional mean. Thus they neglect the nonlinear bias term in the second-moment closure equation for the conditional mean. The principle of energetic consistency implies that, to precisely the extent that growing modes are important in data assimilation, this term is also important.

Cohn, Stephen E.↗
NASA NTRS2012

A review of reduced Kalman filters for clock ensembles

This paper reviews the author’s previous work on free-running timescales based on Kalman filters that act upon clock comparisons. The natural Kalman clock ensemble algorithm tends to optimize long-term timescale stability at the expense of short-term stability. By subjecting each postmeasurement error covariance matrix to a non-transparent reduction operation, one obtains corrected clocks with improved short-term stability and little sacrifice of long-term stability. A new result on covariance matrix reduction is also stated.

Greenhall, Charles A.↗
DOE OSTI2025

Joint state-parameter estimation for the reduced fracture model via the united filter

Here, in this paper, we introduce an effective United Filter method for jointly estimating the solution state and physical parameters in flow and transport problems within fractured porous media. Fluid flow and transport in fractured porous media are critical in subsurface hydrology, geophysics, and reservoir geomechanics. Reduced fracture models, which represent fractures as lower-dimensional interfaces, enable efficient multi-scale simulations. However, reduced fracture models also face accuracy challenges due to modeling errors and uncertainties in physical parameters such as permeability and fracture geometry. To address these challenges, we propose a United Filter method, which integrates the Ensemble Score Filter (EnSF) for state estimation with the Direct Filter for parameter estimation. EnSF, based on a score-based diffusion model framework, produces ensemble representations of the state distribution without deep learning. Meanwhile, the Direct Filter, a recursive Bayesian inference method, estimates parameters directly from state observations. The United Filter combines these methods iteratively: EnSF estimates are used to refine parameter values, which are then fed back to improve state estimation. Numerical experiments demonstrate that the United Filter method surpasses the state-of-the-art Augmented Ensemble Kalman Filter, delivering more accurate state and parameter estimation for reduced fracture models. This framework also provides a robust and efficient solution for PDE-constrained inverse problems with uncertainties and sparse observations.

Bayesian inference↗
DOE OSTI2022

Development of an open-source regional data assimilation system in PEcAn v. 1.7.2: application to carbon cycle reanalysis across the contiguous US using SIPNET

Abstract. The ability to monitor, understand, and predict the dynamics of the terrestrial carbon cycle requires the capacity to robustly and coherently synthesize multiple streams of information that each provide partial information about different pools and fluxes. In this study, we introduce a new terrestrial carbon cycle data assimilation system, built on the PEcAn model–data eco-informatics system, and its application for the development of a proof-of-concept carbon “reanalysis” product that harmonizes carbon pools (leaf, wood, soil) and fluxes (GPP, Ra, Rh, NEE) across the contiguous United States from 1986–2019. We first calibrated this system against plant trait and flux tower net ecosystem exchange (NEE) using a novel emulated hierarchical Bayesian approach. Next, we extended the Tobit–Wishart ensemble filter (TWEnF) state data assimilation (SDA) framework, a generalization of the common ensemble Kalman filter which accounts for censored data and provides a fully Bayesian estimate of model process error, to a regional-scale system with a calibrated localization. Combined with additional workflows for propagating parameter, initial condition, and driver uncertainty, this represents the most complete and robust uncertainty accounting available for terrestrial carbon models. Our initial reanalysis was run on an irregular grid of ∼ 500 points selected using a stratified sampling method to efficiently capture environmental heterogeneity. Remotely sensed observations of aboveground biomass (Landsat LandTrendr) and leaf area index (LAI) (MODIS MOD15) were sequentially assimilated into the SIPNET model. Reanalysis soil carbon, which was indirectly constrained based on modeled covariances, showed general agreement with SoilGrids, an independent soil carbon data product. Reanalysis NEE, which was constrained based on posterior ensemble weights, also showed good agreement with eddy flux tower NEE and reduced root mean square error (RMSE) compared to the calibrated forecast. Ultimately, PEcAn's new open-source regional data assimilation framework provides a scalable workflow for harmonizing multiple data constraints and providing a uniform synthetic platform for carbon monitoring, reporting, and verification (MRV) as well as accelerating terrestrial carbon cycle research.

54 ENVIRONMENTAL SCIENCES↗
NASA NTRS2014

Hybrid Data Assimilation without Ensemble Filtering

The Global Modeling and Assimilation Office is preparing to upgrade its three-dimensional variational system to a hybrid approach in which the ensemble is generated using a square-root ensemble Kalman filter (EnKF) and the variational problem is solved using the Grid-point Statistical Interpolation system. As in most EnKF applications, we found it necessary to employ a combination of multiplicative and additive inflations, to compensate for sampling and modeling errors, respectively and, to maintain the small-member ensemble solution close to the variational solution; we also found it necessary to re-center the members of the ensemble about the variational analysis. During tuning of the filter we have found re-centering and additive inflation to play a considerably larger role than expected, particularly in a dual-resolution context when the variational analysis is ran at larger resolution than the ensemble. This led us to consider a hybrid strategy in which the members of the ensemble are generated by simply converting the variational analysis to the resolution of the ensemble and applying additive inflation, thus bypassing the EnKF. Comparisons of this, so-called, filter-free hybrid procedure with an EnKF-based hybrid procedure and a control non-hybrid, traditional, scheme show both hybrid strategies to provide equally significant improvement over the control; more interestingly, the filter-free procedure was found to give qualitatively similar results to the EnKF-based procedure.

Kalman Filter↗
NASA NTRS2018

Insight and Evidence Motivating the Simplification of Dual-Analysis Hybrid Systems into Single-Analysis Hybrid Systems

Many hybrid data assimilation systems currently used for NWP employ some form of dual-analysis system approach. Typically a hybrid variational analysis is responsible for creating initial conditions for high-resolution forecasts, and an ensemble analysis system is responsible for creating sample perturbations used to form the flow-dependent part of the background error covariance required in the hybrid analysis component. In many of these, the two analysis components employ different methodologies, e.g., variational and ensemble Kalman filter. In such cases, it is not uncommon to have observations treated rather differently between the two analyses components; recentering of the ensemble analysis around the hybrid analysis is used to compensated for such differences. Furthermore, in many cases, the hybrid variational high-resolution system implements some type of four-dimensional approach, whereas the underlying ensemble system relies on a three-dimensional approach, which again introduces discrepancies in the overall system. Connected to these is the expectation that one can reliably estimate observation impact on forecasts issued from hybrid analyses by using an ensemble approach based on the underlying ensemble strategy of dual-analysis systems. Just the realization that the ensemble analysis makes substantially different use of observations as compared to their hybrid counterpart should serve as enough evidence of the implausibility of such expectation. This presentation assembles numerous anecdotal evidence to illustrate the fact that hybrid dual-analysis systems must, at the very minimum, strive for consistent use of the observations in both analysis sub-components. Simpler than that, this work suggests that hybrid systems can reliably be constructed without the need to employ a dual-analysis approach. In practice, the idea of relying on a single analysis system is appealing from a cost-maintenance perspective. More generally, single-analysis systems avoid contradictions such as having to choose one sub-component to generate performance diagnostics to another, possibly not fully consistent, component.

Todling, Ricardo↗
NASA NTRS2006

An Ensemble-Based Smoother with Retrospectively Updated Weights for Highly Nonlinear Systems

Monte Carlo computational methods have been introduced into data assimilation for nonlinear systems in order to alleviate the computational burden of updating and propagating the full probability distribution. By propagating an ensemble of representative states, algorithms like the ensemble Kalman filter (EnKF) and the resampled particle filter (RPF) rely on the existing modeling infrastructure to approximate the distribution based on the evolution of this ensemble. This work presents an ensemble-based smoother that is applicable to the Monte Carlo filtering schemes like EnKF and RPF. At the minor cost of retrospectively updating a set of weights for ensemble members, this smoother has demonstrated superior capabilities in state tracking for two highly nonlinear problems: the double-well potential and trivariate Lorenz systems. The algorithm does not require retrospective adaptation of the ensemble members themselves, and it is thus suited to a streaming operational mode. The accuracy of the proposed backward-update scheme in estimating non-Gaussian distributions is evaluated by comparison to the more accurate estimates provided by a Markov chain Monte Carlo algorithm.

Monte Carlo↗