Abstract:Abstract This article shows that increasing the observation variance at small scales can reduce the ensemble size required to avoid collapse in particle filtering of spatially extended dynamics and improve the resulting uncertainty quantification at large scales. Particle filter weights depend on how well ensemble members agree with observations, and collapse occurs when a few ensemble members receive most of the weight. Collapse causes catastrophic variance underestimation. Increasing small-scale variance in the observation error model reduces the incidence of collapse by de-emphasizing small-scale differences between the ensemble members and the observations. Doing so smooths the posterior mean, though it does not smooth the individual ensemble members. Two options for implementing the proposed observation error model are described. Taking a discretized elliptic differential operator as an observation error covariance matrix provides the desired property of a spectrum that grows in the approach to small scales. This choice also introduces structure exploitable by scalable computation techniques, including multigrid solvers and multiresolution approximations to the corresponding integral operator. Alternatively the observations can be smoothed and then assimilated under the assumption of independent errors, which is equivalent to assuming large errors at small scales. The method is demonstrated on a linear stochastic partial differential equation, where it significantly reduces the occurrence of particle filter collapse while maintaining accuracy. It also improves continuous ranked probability scores by as much as 25%, indicating that the weighted ensemble more accurately represents the true distribution. The method is compatible with other techniques for improving the performance of particle filters.

What the collapse of the ensemble Kalman filter tells us about particle filters

Particle Kalman Filtering: A Nonlinear Bayesian Framework for Ensemble Kalman Filters

Localizing the Ensemble Kalman Particle Filter

Nonlinear stability and ergodicity of ensemble based Kalman filters

A hybrid ensemble transform filter for nonlinear and spatially extended dynamical systems

Particle Network Ensemble Kalman Filter

Analysis of the Ensemble Kalman Filter for Inverse Problems

Accuracy of the Ensemble Kalman Filter in the Near-Linear Setting

Asymptotic Convergence of the Ensemble Kalman Filter.

An Iterative Ensemble Kalman Filter for Data Assimilation

Adaptive Multi-Fidelity Probabilistic Collocation-Based Kalman Filter for Subsurface Flow Data Assimilation: Numerical Modeling and Real-World Experiment

A Local Ensemble Kalman Filter for Atmospheric Data Assimilation

Curse-of-dimensionality revisited: Collapse of the particle filter in very large scale systems

Particle filters for high-dimensional geoscience applications: a review

On the stability and the uniform propagation of chaos properties of Ensemble Kalman-Bucy filters

Data Assimilation for Unsaturated Flow Models with Restart Adaptive Probabilistic Collocation Based Kalman Filter

Data assimilation for nonlinear problems by ensemble Kalman filter with reparameterization

Ensemble Kalman Methods for High-Dimensional Hierarchical Dynamic Space-Time Models

Improving Particle Filter Performance by Smoothing Observations

A hybrid particle-ensemble Kalman filter for problems with medium nonlinearity

Particle Filtering for System Identification in Civil Engineering