Abstract:In this paper, we study efficient approximate sampling for probability distributions known up to normalization constants. We specifically focus on a problem class arising in Bayesian inference for large-scale inverse problems in science and engineering applications. The computational challenges we address with the proposed methodology are: (i) the need for repeated evaluations of expensive forward models; (ii) the potential existence of multiple modes; and (iii) the fact that gradient of, or adjoint solver for, the forward model might not be feasible. While existing Bayesian inference methods meet some of these challenges individually, we propose a framework that tackles all three systematically. Our approach builds upon the Fisher-Rao gradient flow in probability space, yielding a dynamical system for probability densities that converges towards the target distribution at a uniform exponential rate. This rapid convergence is advantageous for the computational burden outlined in (i). We apply Gaussian mixture approximations with operator splitting techniques to simulate the flow numerically; the resulting approximation can capture multiple modes thus addressing (ii). Furthermore, we employ the Kalman methodology to facilitate a derivative-free update of these Gaussian components and their respective weights, addressing the issue in (iii). The proposed methodology results in an efficient derivative-free posterior approximation method, flexible enough to handle multi-modal distributions: Gaussian Mixture Kalman Inversion (GMKI). The effectiveness of GMKI is demonstrated both theoretically and numerically in several experiments with multimodal target distributions, including proof-of-concept and two-dimensional examples, as well as a large-scale application: recovering the Navier-Stokes initial condition from solution data at positive times.

Particle Flow Gaussian Sum Particle Filter

Particle Flow Gaussian Particle Filter

Fast Gaussian particle filtering algorithm

Particle Gaussian Mixture (PGM) Filters

Contour tracking using Gaussian particle filter

GREY PREDICTION BASED PARTICLE FILTER FOR MANEUVERING TARGET TRACKING

Gaussian Particle Filter Based Pose and Motion Estimation

Particle Filter with Hybrid Proposal Distribution for Nonlinear State Estimation.

Gaussian Condensation Filter Based on Cooperative Constrained Particle Flow

A General-Purpose Fixed-Lag No-U-Turn Sampler for Nonlinear Non-Gaussian State Space Models

A Fast Estimating Linear/Nonlinear Condition Particle Filter

GAPSLAM: Blending Gaussian Approximation and Particle Filters for Real-Time Non-Gaussian SLAM

Improved Particle Filter Based on Fine Resampling Algorithm

Flexible and Efficient Inference with Particles for the Variational Gaussian Approximation

The Gaussian Sum Convolution Probability Hypothesis Density Filter

Variational Gaussian filtering via Wasserstein gradient flows

A Non-Gaussian Bayesian Filter Using Power and Generalized Logarithmic Moments

Particle Filter Based On Partial Accurate Observation For High Dimensional Stochastic Dynamic System

Drift Homotopy Methods for A Non-Gaussian Filter

Efficient, multimodal, and derivative-free Bayesian inference with Fisher-Rao gradient flows

Particle filters for positioning, navigation, and tracking