Abstract:In this paper, we study efficient approximate sampling for probability distributions known up to normalization constants. We specifically focus on a problem class arising in Bayesian inference for large-scale inverse problems in science and engineering applications. The computational challenges we address with the proposed methodology are: (i) the need for repeated evaluations of expensive forward models; (ii) the potential existence of multiple modes; and (iii) the fact that gradient of, or adjoint solver for, the forward model might not be feasible. While existing Bayesian inference methods meet some of these challenges individually, we propose a framework that tackles all three systematically. Our approach builds upon the Fisher-Rao gradient flow in probability space, yielding a dynamical system for probability densities that converges towards the target distribution at a uniform exponential rate. This rapid convergence is advantageous for the computational burden outlined in (i). We apply Gaussian mixture approximations with operator splitting techniques to simulate the flow numerically; the resulting approximation can capture multiple modes thus addressing (ii). Furthermore, we employ the Kalman methodology to facilitate a derivative-free update of these Gaussian components and their respective weights, addressing the issue in (iii). The proposed methodology results in an efficient derivative-free posterior approximation method, flexible enough to handle multi-modal distributions: Gaussian Mixture Kalman Inversion (GMKI). The effectiveness of GMKI is demonstrated both theoretically and numerically in several experiments with multimodal target distributions, including proof-of-concept and two-dimensional examples, as well as a large-scale application: recovering the Navier-Stokes initial condition from solution data at positive times.

Fast Posterior Probability Sampling with Normalizing Flows and Its Applicability in Bayesian analysis in Particle Physics

Fast posterior probability sampling with normalizing flows and its applicability in Bayesian analysis in particle physics

Normalizing Flows for Bayesian Posteriors: Reproducibility and Deployment

ν-Flows: Conditional Neutrino Regression

Understanding posterior projection effects with normalizing flows

Fast and improved neutrino reconstruction in multineutrino final states with conditional normalizing flows

$ν^2$-Flows: Fast and improved neutrino reconstruction in multi-neutrino final states with conditional normalizing flows

Inference and De-Noising of Non-Gaussian Particle Distribution Functions: A Generative Modeling Approach

Accelerated nested sampling with $β$-flows for gravitational waves

Comparative Study of Coupling and Autoregressive Flows through Robust Statistical Tests

Flow Matching for Scalable Simulation-Based Inference

Chained Quantile Morphing with Normalizing Flows

Marginal Post Processing of Bayesian Inference Products with Normalizing Flows and Kernel Density Estimators

End-to-end simulation of particle physics events with Flow Matching and generator Oversampling

Faster Uncertainty Quantification for Inverse Problems with Conditional Normalizing Flows

Bayesian inference for finite mixtures of univariate and multivariate skew-normal and skew-t distributions

Stochastic Normalizing Flows

Efficient, multimodal, and derivative-free Bayesian inference with Fisher-Rao gradient flows

Flow Annealed Importance Sampling Bootstrap meets Differentiable Particle Physics

Fast Inference Using Automatic Differentiation and Neural Transport in Astroparticle Physics

Fast Parameter Inference on Pulsar Timing Arrays with Normalizing Flows