Abstract:Many data-science problems can be formulated as an inverse problem, where the parameters are estimated by minimizing a proper loss function. When complicated black-box models are involved, derivative-free optimization tools are often needed. The ensemble Kalman filter (EnKF) is a particle-based derivative-free Bayesian algorithm originally designed for data assimilation. Recently, it has been applied to inverse problems for computational efficiency. The resulting algorithm, known as ensemble Kalman inversion (EKI), involves running an ensemble of particles with EnKF update rules so they can converge to a minimizer. In this article, we investigate EKI convergence in general nonlinear settings. To improve convergence speed and stability, we consider applying EKI with non-constant step-sizes and covariance inflation. We prove that EKI can hit critical points with finite steps in non-convex settings. We further prove that EKI converges to the global minimizer polynomially fast if the loss function is strongly convex. We verify the analysis presented with numerical experiments on two inverse problems.

What problem does this paper attempt to address?

### What problems does this paper attempt to solve? This paper aims to explore and accelerate the convergence speed and stability of Ensemble Kalman Inversion (EKI) in a nonlinear environment. Specifically, the main research problems include: 1. **Convergence analysis of EKI**: - The paper studies the convergence behavior of EKI in a general nonlinear setting, especially how to improve its convergence speed and stability by introducing non - constant step sizes and covariance inflation. - The author proves that EKI can reach a critical point within a finite number of steps, and if the loss function is strongly convex, EKI can converge to the global minimum at a polynomial rate. 2. **Improvement of the optimization algorithm**: - To improve the performance of EKI, the paper proposes a new Tikhonov - regularized EKI algorithm, which applies non - constant step sizes and covariance inflation. - These modifications not only theoretically improve the performance of the algorithm, but also are verified in numerical experiments. 3. **Comparison with the Gauss - Newton method**: - The paper compares EKI with the Gauss - Newton method and explains why EKI can be used as an optimization tool. - Although EKI does not require gradient information, its update rule is similar to that of the Gauss - Newton method in the long - term operation. 4. **Verification of practical applications**: - The author verifies their theoretical analysis through numerical experiments. They implement the new version of EKI on the one - dimensional Lorenz 96 model and one - dimensional and two - dimensional nonlinear elliptic partial differential equations respectively, and compare it with the standard Tikhonov EKI. ### Summary of mathematical formulas - **Data - mismatch function**: \[ \ell_0(u)=\|G(u) - y\|^2_{\Gamma} \] where \(\|v\|^2_{\Gamma}=v^T\Gamma^{-1}v\) represents the weighted Mahalanobis distance. - **Regularized loss function**: \[ \ell(u)=\|G(u) - y\|^2_{\Gamma}+\lambda\|u\|^2_{\Sigma} \] - **EKI mean update formula**: \[ m_{n + 1}=m_n+C^{pu}_n(C^{pp}_n+h^{-1}_n\Gamma^+)^{-1}(z - H(m_n)) \] - **EKI covariance update formula**: \[ C^{uu}_{n + 1}=C^{uu}_n-C^{up}_n(C^{pp}_n+h^{-1}_n\Gamma^+)^{-1}C^{pu}_n+\alpha^2_n\Sigma \] - **Gauss - Newton update formula**: \[ G_n=C^{uu}_nJ_n(J_nC^{uu}_nJ_n^T+h^{-1}_n\Gamma^+)^{-1}(z - H(m_n)) \] These formulas show the key steps of EKI in the optimization process and its similarity to the Gauss - Newton method. Through these analyses, the paper provides a solid theoretical basis and practical improvement suggestions for the application of EKI in a nonlinear environment.

Convergence Acceleration of Ensemble Kalman Inversion in Nonlinear Settings

The Ensemble Kalman Filter for Dynamic Inverse Problems

Gradient flow structure and convergence analysis of the ensemble Kalman inversion for nonlinear forward models

Iterated Kalman Methodology For Inverse Problems

Adaptive Tikhonov strategies for stochastic ensemble Kalman inversion

Ensemble Kalman inversion: mean-field limit and convergence analysis

Sparsity-promoting Elastic Net Method with Rotations for High-Dimensional Nonlinear Inverse Problem

The Fundamental Subspaces of Ensemble Kalman Inversion

Dropout Ensemble Kalman inversion for high dimensional inverse problems

Flow Annealed Kalman Inversion for Gradient-Free Inference in Bayesian Inverse Problems

Analysis of the Ensemble Kalman Filter for Inverse Problems

Improve Unscented Kalman Inversion With Low-Rank Approximation and Reduced-Order Model

Component-wise iterative ensemble Kalman inversion for static Bayesian models with unknown measurement error covariance

EnKSGD: A Class Of Preconditioned Black Box Optimization And Inversion Algorithms

Nonlinear Kalman Filtering with Divergence Minimization

Interacting Langevin Diffusions: Gradient Structure And Ensemble Kalman Sampler

Gradient-free training of neural ODEs for system identification and control using ensemble Kalman inversion

Ensemble Kalman Diffusion Guidance: A Derivative-free Method for Inverse Problems

Asymptotic Convergence of the Ensemble Kalman Filter.

Inverse Extended Kalman Filter -- Part I: Fundamentals

Nonlinear stability and ergodicity of ensemble based Kalman filters