Abstract:The use of the ensemble smoother (ES) instead of the ensemble Kalman filter increases the nonlinearity of the update step during data assimilation and the need for iterative assimilation methods. A previous version of the iterative ensemble smoother based on Gauss–Newton formulation was able to match data relatively well but only after a large number of iterations. A multiple data assimilation method (MDA) was generally more efficient for large problems but lacked ability to continue “iterating” if the data mismatch was too large. In this paper, we develop an efficient, iterative ensemble smoother algorithm based on the Levenberg–Marquardt (LM) method of regularizing the update direction and choosing the step length. The incorporation of the LM damping parameter reduces the tendency to add model roughness at early iterations when the update step is highly nonlinear, as it often is when all data are assimilated simultaneously. In addition, the ensemble approximation of the Hessian is modified in a way that simplifies computation and increases stability. We also report on a simplified algorithm in which the model mismatch term in the updating equation is neglected. We thoroughly evaluated the new algorithm based on the modified LM method, LM-ensemble randomized maximum likelihood (LM-EnRML), and the simplified version of the algorithm, LM-EnRML (approx), on three test cases. The first is a highly nonlinear single-variable problem for which results can be compared against the true conditional pdf. The second test case is a one-dimensional two-phase flow problem in which the permeability of 31 grid cells is uncertain. In this case, Markov chain Monte Carlo results are available for comparison with ensemble-based results. The third test case is the Brugge benchmark case with both 10 and 20 years of history. The efficiency and quality of results of the new algorithms were compared with the standard ES (without iteration), the ensemble-based Gauss–Newton formulation, the standard ensemble-based LM formulation, and the MDA. Because of the high level of nonlinearity, the standard ES performed poorly on all test cases. The MDA often performed well, especially at early iterations where the reduction in data mismatch was quite rapid. The best results, however, were always achieved with the new iterative ensemble smoother algorithms, LM-EnRML and LM-EnRML (approx).

Dynamical Gaussian, lognormal, and reverse lognormal maximum‐likelihood ensemble filter

The Maximum Likelihood Ensemble Filter with State Space Localization

A competitive baseline for deep learning enhanced data assimilation using conditional Gaussian ensemble Kalman filtering

An Adaptive Covariance Parameterization Technique for the Ensemble Gaussian Mixture Filter

A comparison of nonlinear extensions to the ensemble Kalman filter: Gaussian Anamorphosis and Two-Step Ensemble Filters

The Mean Field Ensemble Kalman Filter: Near-Gaussian Setting

Model error covariance estimation in particle and ensemble Kalman filters using an online expectation-maximization algorithm

A hybrid particle-ensemble Kalman filter for problems with medium nonlinearity

Model uncertainty estimation using the expectation maximization algorithm and a particle flow filter

Ensemble-localized Kernel Density Estimation with Applications to the Ensemble Gaussian Mixture Filter

Bayesian Ensemble Kalman Filter for Gaussian Mixture Models

A Gaussian mixture ensemble transform filter

Nonlinear Bayesian Filtering with Natural Gradient Gaussian Approximation

Gaussian Mixture Multiple-Model Multi-Bernoulli Filters For Nonlinear Models Via Unscented Transforms

Evaluation of Gaussian approximations for data assimilation in reservoir models

Approximate MMSE Estimator for Linear Dynamic Systems With Gaussian Mixture Noise

Levenberg–Marquardt forms of the iterative ensemble smoother for efficient history matching and uncertainty quantification

Exact nonlinear state estimation

Ensemble Kalman Filtering for Online Gaussian Process Regression and Learning

The Ensemble Epanechnikov Mixture Filter

A unified framework for the analysis of accuracy and stability of a class of approximate Gaussian filters for the Navier-Stokes Equations