Abstract:Data assimilation methods aim at estimating the state of a system by combining observations with a physical model. When sequential data assimilation is considered, the joint distribution of the latent state and the observations is described mathematically using a state-space model, and filtering or smoothing algorithms are used to approximate the conditional distribution of the state given the observations. The most popular algorithms in the data assimilation community are based on the Ensemble Kalman Filter and Smoother (EnKF/EnKS) and its extensions. In this paper we investigate an alternative approach where a Conditional Particle Filter (CPF) is combined with Backward Simulation (BS). This allows to explore efficiently the latent space and simulate quickly relevant trajectories of the state conditionally to the observations. We also tackle the difficult problem of parameter estimation. Indeed, the models generally involve statistical parameters in the physical models and/or in the stochastic models for the errors. These parameters strongly impact the results of the data assimilation algorithm and there is a need for an efficient method to estimate them. Expectation-Maximization (EM) is the most classical algorithm in the statistical literature to estimate the parameters in models with latent variables. It consists in updating sequentially the parameters by maximizing a likelihood function where the state is approximated using a smoothing algorithm. In this paper, we propose an original Stochastic Expectation-Maximization (SEM) algorithm combined to the CPF-BS smoother to estimate the statistical parameters. We show on several toy models that this algorithm provides, with reasonable computational cost, accurate estimations of the statistical parameters and the state in highly nonlinear state-space models, where the application of EM algorithms using EnKS is limited. We also provide a Python source code of the algorithm.

Data Assimilation for Models with Parametric Uncertainty

Fast Uncertainty Quantification by Sparse Data Learning from Multiphysics Systems

Resource-Constrained Model Selection for Uncertainty Propagation and Data Assimilation.

Model uncertainty estimation using the expectation maximization algorithm and a particle flow filter

Combining Stochastic Parameterized Reduced‐Order Models With Machine Learning for Data Assimilation and Uncertainty Quantification With Partial Observations

Adaptive Multifidelity Data Assimilation for Nonlinear Subsurface Flow Problems

Adaptive Multi-Fidelity Probabilistic Collocation-Based Kalman Filter for Subsurface Flow Data Assimilation: Numerical Modeling and Real-World Experiment

Data Assimilation for Unsaturated Flow Models with Restart Adaptive Probabilistic Collocation Based Kalman Filter

Model error covariance estimation in particle and ensemble Kalman filters using an online expectation-maximization algorithm

Sequential data assimilation with multiple nonlinear models and applications to subsurface flow

A New Data Assimilation Method Based On Robust Filter Technique

An efficient particle-based method for maximum likelihood estimation in nonlinear state-space models

Estimation with particle filter under model uncertainty

Data assimilation for the stochastic Camassa-Holm equation using particle filtering: a numerical investigation

Uncertainty quantification of nonlinear Lagrangian data assimilation using linear stochastic forecast models

Efficient Sequetial Data Assimilation for the Joint Distributios of System States

A Defensive Marginal Particle Filtering Method for Data Assimilation

Gaussian Process Regression and Conditional Polynomial Chaos for Parameter Estimation

Bayesian inference for geophysical fluid dynamics using generative models

Data assimilation for online model calibration in discrete event simulation

Parametric Uncertainty Assessment in Hydrological Modeling Using the Generalized Polynomial Chaos Expansion