Abstract:Data assimilation methods aim at estimating the state of a system by combining observations with a physical model. When sequential data assimilation is considered, the joint distribution of the latent state and the observations is described mathematically using a state-space model, and filtering or smoothing algorithms are used to approximate the conditional distribution of the state given the observations. The most popular algorithms in the data assimilation community are based on the Ensemble Kalman Filter and Smoother (EnKF/EnKS) and its extensions. In this paper we investigate an alternative approach where a Conditional Particle Filter (CPF) is combined with Backward Simulation (BS). This allows to explore efficiently the latent space and simulate quickly relevant trajectories of the state conditionally to the observations. We also tackle the difficult problem of parameter estimation. Indeed, the models generally involve statistical parameters in the physical models and/or in the stochastic models for the errors. These parameters strongly impact the results of the data assimilation algorithm and there is a need for an efficient method to estimate them. Expectation-Maximization (EM) is the most classical algorithm in the statistical literature to estimate the parameters in models with latent variables. It consists in updating sequentially the parameters by maximizing a likelihood function where the state is approximated using a smoothing algorithm. In this paper, we propose an original Stochastic Expectation-Maximization (SEM) algorithm combined to the CPF-BS smoother to estimate the statistical parameters. We show on several toy models that this algorithm provides, with reasonable computational cost, accurate estimations of the statistical parameters and the state in highly nonlinear state-space models, where the application of EM algorithms using EnKS is limited. We also provide a Python source code of the algorithm.

Estimation of State Space Models Using Particle Filters : Applications to Economics and Finance

Estimation with particle filter under model uncertainty

Estimating mixed-effects state-space models via particle filters and the EM algorithm

A Stable Particle Filter for a Class of High-Dimensional State-Space Models

Improved Particle Filter Using Clustering Similarity of the State Trajectory with Application to Nonlinear Estimation: Theory, Modeling, and Applications

Particle Filter Based On Partial Accurate Observation For High Dimensional Stochastic Dynamic System

Optimization-based Particle Filter for State and Parameter Estimation

Hybrid Sampling-Based Particle Filtering with Temporal Constraints

An Improved Particle Filter Based on Space Distribution and Time Series Analysis

Particle Filtering for Dynamic Systems with Future Constraints

Resampling-free Particle Filters in High-dimensions

An efficient particle-based method for maximum likelihood estimation in nonlinear state-space models

Bayesian Filtering for High-Dimensional State-Space Models With State Partition and Error Compensation

State estimation of switched finite-field networks: A multi-valued particle filter approach

The Extended State Particle Filter for Unknown Process Models

A simple modification for improving inference of non-linear dynamical systems

Particle Gibbs for Likelihood-Free Inference of State Space Models with Application to Stochastic Volatility

Particle Filtering with Applications in Networked Systems: a Survey

A variance-adaptive particle filter with application to time-varying parameter estimation

Traffic Estimation Based on Particle Filtering with Stochastic State Reconstruction Using Mobile Network Data

Reduced Credit Risk Measurement Model with Particle Filter Approach