Abstract:Data assimilation methods aim at estimating the state of a system by combining observations with a physical model. When sequential data assimilation is considered, the joint distribution of the latent state and the observations is described mathematically using a state-space model, and filtering or smoothing algorithms are used to approximate the conditional distribution of the state given the observations. The most popular algorithms in the data assimilation community are based on the Ensemble Kalman Filter and Smoother (EnKF/EnKS) and its extensions. In this paper we investigate an alternative approach where a Conditional Particle Filter (CPF) is combined with Backward Simulation (BS). This allows to explore efficiently the latent space and simulate quickly relevant trajectories of the state conditionally to the observations. We also tackle the difficult problem of parameter estimation. Indeed, the models generally involve statistical parameters in the physical models and/or in the stochastic models for the errors. These parameters strongly impact the results of the data assimilation algorithm and there is a need for an efficient method to estimate them. Expectation-Maximization (EM) is the most classical algorithm in the statistical literature to estimate the parameters in models with latent variables. It consists in updating sequentially the parameters by maximizing a likelihood function where the state is approximated using a smoothing algorithm. In this paper, we propose an original Stochastic Expectation-Maximization (SEM) algorithm combined to the CPF-BS smoother to estimate the statistical parameters. We show on several toy models that this algorithm provides, with reasonable computational cost, accurate estimations of the statistical parameters and the state in highly nonlinear state-space models, where the application of EM algorithms using EnKS is limited. We also provide a Python source code of the algorithm.

Data assimilation for online model calibration in discrete event simulation

Data assimilation approach for addressing imperfections in people flow measurement techniques using particle filter

Maintaining the long-term accuracy of water distribution models with data assimilation methods: A comparative study

Data Assimilation in Air Contaminant Dispersion Using a Particle Filter and Expectation-Maximization Algorithm

A Mesoscopic Traffic Data Assimilation Framework for Vehicle Density Estimation on Urban Traffic Networks Based on Particle Filters

Improving the Accuracy and Efficiency of Online Calibration for Simulation-based Dynamic Traffic Assignment

Modeling Study of a Severe Aerosol Pollution Event in December 2013over Shanghai China:An Application of Chemical Data Assimilation

Data assimilation and agent-based modelling: towards the incorporation of categorical agent parameters

Research on water level forecasting and hydraulic parameter calibration in the 1D open channel hydrodynamic model using data assimilation

Adaptive Multi-Fidelity Probabilistic Collocation-Based Kalman Filter for Subsurface Flow Data Assimilation: Numerical Modeling and Real-World Experiment

Adaptive Multifidelity Data Assimilation for Nonlinear Subsurface Flow Problems

Off-line Calibration of Dynamic Traffic Assignment System Based on FlowSIM

Exploring the Performance of Spatio-Temporal Assimilation in an Urban Cellular Automata Model

Statistical-Learning-Based Ensemble Data Assimilation Methods for Parameter Estimation in Hydrodynamic Models

Flexible Iterative Ensemble Smoother for Calibration of Perfect and Imperfect Models

Model error covariance estimation in particle and ensemble Kalman filters using an online expectation-maximization algorithm

Assimilating UAV-Based Observation Data for Wildfire Spread Simulation

Sequential data assimilation with multiple nonlinear models and applications to subsurface flow

An efficient particle-based method for maximum likelihood estimation in nonlinear state-space models

Data assimilation method for improving the global spatiotemporal predictions of epidemic dynamics yielded by an ensemble Kalman filter and Metropolis–Hastings sampling

Introduction to data assimilation for parameter estimation