SEARCH · Engineering Papers
Results for “ensemble Kalman filters”
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
Simultaneous assimilation of AIRS Xco 2 and meteorological observations in a carbon climate model with an ensemble Kalman filter
Explore the source record for details and available documents.
Results from Assimilating AMSR-E Soil Moisture Estimates into a Land Surface Model Using an Ensemble Kalman Filter in the Land Information System
Improve simulations of soil moisture/temperature, and consequently boundary layer states and processes, by assimilating AMSR-E soil moisture estimates into a coupled land surface-mesoscale model Provide a new land surface model as an option in the Land Information System (LIS)
Building Physics-Based Solar Cycle Forecasts Using the Ensemble Kalman Filter Method
Explore the source record for details and available documents.
Physics-Based Solar Cycle Forecast Using the Ensemble Kalman Filter Method
Explore the source record for details and available documents.
Geomagnetic secular variation forecast using the NASA GEMS ensemble Kalman filter: A candidate SV model for IGRF-13
Explore the source record for details and available documents.
Estimation of Power System Frequency Dynamics Using Cubature and Ensemble Kalman Filters
Explore the source record for details and available documents.
Mass Conservation and Positivity Preservation with Ensemble-type Kalman Filter Algorithms
Maintaining conservative physical laws numerically has long been recognized as being important in the development of numerical weather prediction (NWP) models. In the broader context of data assimilation, concerted efforts to maintain conservation laws numerically and to understand the significance of doing so have begun only recently. In order to enforce physically based conservation laws of total mass and positivity in the ensemble Kalman filter, we incorporate constraints to ensure that the filter ensemble members and the ensemble mean conserve mass and remain nonnegative through measurement updates. We show that the analysis steps of ensemble transform Kalman filter (ETKF) algorithm and ensemble Kalman filter algorithm (EnKF) can conserve the mass integral, but do not preserve positivity. Further, if localization is applied or if negative values are simply set to zero, then the total mass is not conserved either. In order to ensure mass conservation, a projection matrix that corrects for localization effects is constructed. In order to maintain both mass conservation and positivity preservation through the analysis step, we construct a data assimilation algorithms based on quadratic programming and ensemble Kalman filtering. Mass and positivity are both preserved by formulating the filter update as a set of quadratic programming problems that incorporate constraints. Some simple numerical experiments indicate that this approach can have a significant positive impact on the posterior ensemble distribution, giving results that are more physically plausible both for individual ensemble members and for the ensemble mean. The results show clear improvements in both analyses and forecasts, particularly in the presence of localized features. Behavior of the algorithm is also tested in presence of model error.
Conservation of Mass and Preservation of Positivity with Ensemble-Type Kalman Filter Algorithms
This paper considers the incorporation of constraints to enforce physically based conservation laws in the ensemble Kalman filter. In particular, constraints are used to ensure that the ensemble members and the ensemble mean conserve mass and remain nonnegative through measurement updates. In certain situations filtering algorithms such as the ensemble Kalman filter (EnKF) and ensemble transform Kalman filter (ETKF) yield updated ensembles that conserve mass but are negative, even though the actual states must be nonnegative. In such situations if negative values are set to zero, or a log transform is introduced, the total mass will not be conserved. In this study, mass and positivity are both preserved by formulating the filter update as a set of quadratic programming problems that incorporate non-negativity constraints. Simple numerical experiments indicate that this approach can have a significant positive impact on the posterior ensemble distribution, giving results that are more physically plausible both for individual ensemble members and for the ensemble mean. In two examples, an update that includes a non-negativity constraint is able to properly describe the transport of a sharp feature (e.g., a triangle or cone). A number of implementation questions still need to be addressed, particularly the need to develop a computationally efficient quadratic programming update for large ensemble.
An Ensemble Score Filter for Tracking High-Dimensional Nonlinear Dynamical Systems
We propose an ensemble score filter (EnSF) for solving high-dimensional nonlinear filtering problems with superior accuracy. A major drawback of existing filtering methods, e.g., particle filters or ensemble Kalman filters, is the low accuracy in handling high-dimensional and highly nonlinear problems. EnSF attacks this challenge by exploiting the score-based diffusion model, defined in a pseudo-temporal domain, to characterizing the evolution of the filtering density. EnSF stores the information of the recursively updated filtering density function in the score function, in stead of storing the information in a set of finite Monte Carlo samples (used in particle filters and ensemble Kalman filters). Unlike existing diffusion models that train neural networks to approximate the score function, we develop a training-free score estimation that uses mini-batch-based Monte Carlo estimator to directly approximate the score function at any pseudo-spatial-temporal location, which provides sufficient accuracy in solving high-dimensional nonlinear problems as well as saves tremendous amount of time spent on training neural networks. Another essential aspect of EnSF is its analytical update step, gradually incorporating data information into the score function, which is crucial in mitigating the degeneracy issue faced when dealing with very high-dimensional nonlinear filtering problems. High-dimensional Lorenz systems are used to demonstrate the performance of our method. EnSF provides surprisingly impressive performance in reliably tracking extremely high-dimensional Lorenz systems (up to 1,000,000 dimension) with highly nonlinear observation processes, which is a well-known challenging problem for existing filtering methods.
An Ensemble Recentering Kalman Filter with an Application to Argo Temperature Data Assimilation into the NASA GEOS-5 Coupled Model
A two-step ensemble recentering Kalman filter (ERKF) analysis scheme is introduced. The algorithm consists of a recentering step followed by an ensemble Kalman filter (EnKF) analysis step. The recentering step is formulated such as to adjust the prior distribution of an ensemble of model states so that the deviations of individual samples from the sample mean are unchanged but the original sample mean is shifted to the prior position of the most likely particle, where the likelihood of each particle is measured in terms of closeness to a chosen subset of the observations. The computational cost of the ERKF is essentially the same as that of a same size EnKF. The ERKF is applied to the assimilation of Argo temperature profiles into the OGCM component of an ensemble of NASA GEOS-5 coupled models. Unassimilated Argo salt data are used for validation. A surprisingly small number (16) of model trajectories is sufficient to significantly improve model estimates of salinity over estimates from an ensemble run without assimilation. The two-step algorithm also performs better than the EnKF although its performance is degraded in poorly observed regions.
Rock Physics-Based Data Assimilation of Integrated Continuous Active-Source Seismic and Pressure Monitoring Data during Geological Carbon Storage
Summary There has been substantial controversy concerning the role of geological carbon storage (GCS) in sequestering anthropogenic carbon emissions to mitigate climate change and global warming. Arguments center on the inability to monitor a geological storage site precisely and continuously, especially highlighting the associated costs and spatiotemporal trade-offs when using conventional subsurface monitoring techniques (well logs, core samples, chemical tracers, and 4D seismics). Active surveillance of GCS sites is essential for managing and mitigating potential leaks but is also required by regulation. With the goal of enhancing the monitoring capability at GCS sites, we present a rock physics-based joint data assimilation model to study a popular GCS site at Cranfield, Mississippi, USA. Synthetic continuous active-source seismic monitoring (CASSM) data (in the form of Vp and Qp measurements) and wellbore pressure monitoring data are assimilated with an ensemble of reservoir realizations to monitor gas saturation and reservoir pressure changes over a period of 100 years. Synthetic seismic attributes are generated using rock physics models (RPMs) and wellbore pressure monitoring data are extracted from the ground truth. Two assimilation methods, ensemble Kalman filter (EnKF) and ensemble Kalman smoother (EnKS), are tested in an observation system simulation experiment (OSSE) environment to assess the prediction accuracy of the individual and composite observation systems. The joint monitoring system achieves more accurate estimates of gas saturation and pressure, across the time span from start of injection to end of forecast, as compared to a single type of monitoring tool and irrespective of data assimilation algorithm choice. These results indicate that jointly assimilated data from two types of sensors (in this case, crosswell seismic and downhole pressure) may lead to a more risk-reducing monitoring design. One would expect that more data, vis-à-vis inclusion of a new sensor type, will improve the accuracy of any GCS monitoring system. However, from a practical standpoint, one important question is whether such a gain in accuracy is worth the additional cost associated with the new sensor. This paper focuses on quantifying the gain in accuracy, such that a practitioner can answer this question.
Simultaneous Estimation of Model State Variables and Observation and Forecast Biases Using a Two-Stage Hybrid Kalman Filter
In this paper, we present a two-stage hybrid Kalman filter to estimate both observation and forecast bias in hydrologic models, in addition to state variables. The biases are estimated using the discrete Kalman filter, and the state variables using the ensemble Kalman filter. A key issue in this multi-component assimilation scheme is the exact partitioning of the difference between observation and forecasts into state, forecast bias and observation bias updates. Here, the error covariances of the forecast bias and the unbiased states are calculated as constant fractions of the biased state error covariance, and the observation bias error covariance is a function of the observation prediction error covariance. In a series of synthetic experiments, focusing on the assimilation of discharge into a rainfall-runoff model, it is shown that both static and dynamic observation and forecast biases can be successfully estimated. The results indicate a strong improvement in the estimation of the state variables and resulting discharge as opposed to the use of a bias-unaware ensemble Kalman filter. Furthermore, minimal code modification in existing data assimilation software is needed to implement the method. The results suggest that a better performance of data assimilation methods should be possible if both forecast and observation biases are taken into account.
Assimilation of SMAP and ASCAT Soil Moisture Retrievals into the JULES Land Surface Model Using the Local Ensemble Transform Kalman Filter
A land data assimilation system is developed to merge satellite soil moisture retrievals into the Joint U.K. Land Environment Simulator (JULES) land surface model (LSM) using the Local Ensemble Transform Kalman Filter (LETKF). The system assimilates microwave soil moisture retrievals from the Soil Moisture Active Passive (SMAP) radiometer and the Advanced Scatterometer (ASCAT) after bias correction based on cumulative distribution function fitting. The soil moisture assimilation estimates are evaluated with ground-based soil moisture measurements over the continental U.S. for five consecutive warm seasons (May–September of 2015–2019). The result shows that both SMAP and ASCAT retrievals improve the accuracy of soil moisture estimates. Especially, the SMAP single-sensor assimilation experiment shows the best performance with the increase of temporal anomaly correlation by ΔR ~ 0.05 for surface soil moisture and ΔR ~ 0.03 for root-zone soil moisture compared with the LSM simulation without satellite data assimilation. SMAP assimilation is more skillful than ASCAT assimilation primarily because of the greater skill of the assimilated SMAP retrievals compared to the ASCAT retrievals. The skill improvement also depends significantly on the region; the higher skill improvement in the western U.S. compared to the eastern U.S. is explained by the Kalman gain in the two experiments. Additionally, the regional skill differences in the single-sensor assimilation experiments are attributed to the number of assimilated observations. Finally, the soil moisture assimilation estimates provide more realistic land surface information than model-only simulations for the 2015 and the 2016 western U.S. droughts, suggesting the advantage of using satellite soil moisture retrievals in the current drought monitoring system.
The Potential Benefits of Handling Mixture Statistics via a Bi-Gaussian EnKF: Tests With All-Sky Satellite Infrared Radiances
The meteorological characteristics of cloudy atmospheric columns can be very different from their clear counterparts. Thus, when a forecast ensemble is uncertain about the presence/absence of clouds at a specific atmospheric column (i.e., some members are clear while others are cloudy), that column's ensemble statistics will contain a mixture of clear and cloudy statistics. Such mixtures are inconsistent with the ensemble data assimilation algorithms currently used in numerical weather prediction. Hence, ensemble data assimilation algorithms that can handle such mixtures can potentially outperform currently used algorithms. In this study, we demonstrate the potential benefits of addressing such mixtures through a bi-Gaussian extension of the ensemble Kalman filter (BGEnKF). The BGEnKF is compared against the commonly used ensemble Kalman filter (EnKF) using perfect model observing system simulated experiments (OSSEs) with a realistic weather model (the Weather Research and Forecast model). Synthetic all-sky infrared radiance observations are assimilated in this study. In these OSSEs, the BGEnKF outperforms the EnKF in terms of the horizontal wind components, temperature, specific humidity, and simulated upper tropospheric water vapor channel infrared brightness temperatures. This study is one of the first to demonstrate the potential of a Gaussian mixture model EnKF with a realistic weather model. Our results thus motivate future research toward improving numerical Earth system predictions though explicitly handling mixture statistics.
Feasibility of DEIM for retrieving the initial field via dimensionality reduction
When parameter estimation is solved in a high-dimensional space, the dimensionality reduction strategy becomes the primary consideration for alleviating the tremendous computational cost. Here, the discrete empirical interpolation method (DEIM) is explored to retrieve the initial condition (IC) by combining the polynomial chaos (PC) based ensemble Kalman filter (i.e. PC-EnKF), where a non-intrusive PC expansion is considered as a surrogate model in place of the forward model in the prediction step of the ensemble Kalman filter, resulting in fewer forward model integrations but with a comparable accuracy as Monte Carlo-based approaches. The DEIM acts as a hyper-reduction tool to provide the low-dimensional input for the high-dimensional initial field, which can be reconstructed using the information on the sparse interpolation grid points that is adaptively obtained through PC-EnKF data assimilation method. Thus an innovative framework to reconstruct the IC is developed. The detailed procedure at each assimilation iteration includes: the determination of the spatial interpolation points, the estimation of the initial values on the interpolation locations using the optimal observations, and the reconstruction of IC in the full space. The current study uses the reconstruction field of initial conditions of the Navier-Stokes equations as an example to illustrate the efficacy of our method. The experimental results demonstrate the proposed algorithm achieves a satisfactory reconstruction for the initial field. The proposed method helps to extend the applicable area of DEIM in solving inverse problems.
Assimilation of Satellite Sea Surface Temperature and Salinity Using the Local Ensemble Transform Kalman Filter Applied to the NASA GEOS Earth System Model
The NASA GEOS-S2S-2 system (Molod et al., 2020) uses the SST relaxation with the Level-4 SST products while the latest GEOS GEOS-S2S-3 system directly assimilates the Level-4 SST in the data assimilation step, yet no study has been conducted to understand the impact of this switch on the GEOS coupled analysis and short-range forecast. In addition, the GEOS-S2S-2/3 systems use the ocean model MOM5. Recently, the ocean model MOM6 is added to the GEOS ESM. This poster presents results of the first coupling of the ensemble data assimilation system to the GEOS MOM6.