Abstract:In this paper, we study efficient approximate sampling for probability distributions known up to normalization constants. We specifically focus on a problem class arising in Bayesian inference for large-scale inverse problems in science and engineering applications. The computational challenges we address with the proposed methodology are: (i) the need for repeated evaluations of expensive forward models; (ii) the potential existence of multiple modes; and (iii) the fact that gradient of, or adjoint solver for, the forward model might not be feasible. While existing Bayesian inference methods meet some of these challenges individually, we propose a framework that tackles all three systematically. Our approach builds upon the Fisher-Rao gradient flow in probability space, yielding a dynamical system for probability densities that converges towards the target distribution at a uniform exponential rate. This rapid convergence is advantageous for the computational burden outlined in (i). We apply Gaussian mixture approximations with operator splitting techniques to simulate the flow numerically; the resulting approximation can capture multiple modes thus addressing (ii). Furthermore, we employ the Kalman methodology to facilitate a derivative-free update of these Gaussian components and their respective weights, addressing the issue in (iii). The proposed methodology results in an efficient derivative-free posterior approximation method, flexible enough to handle multi-modal distributions: Gaussian Mixture Kalman Inversion (GMKI). The effectiveness of GMKI is demonstrated both theoretically and numerically in several experiments with multimodal target distributions, including proof-of-concept and two-dimensional examples, as well as a large-scale application: recovering the Navier-Stokes initial condition from solution data at positive times.

A Flow Model with a More-Added Coupling Function

A Tale of Two Flows: Cooperative Learning of Langevin Flow and Normalizing Flow Toward Energy-Based Model

Flow Matching for Generative Modeling

Neural Conjugate Flows: Physics-informed architectures with flow structure

SE(3) Equivariant Augmented Coupling Flows

C-Flow: Conditional Generative Flow Models for Images and 3D Point Clouds

ContextFlow++: Generalist-Specialist Flow-based Generative Models with Mixed-Variable Context Encoding

Generative Flows with Matrix Exponential

Computing high-dimensional optimal transport by flow neural networks

Flow Matching for Scalable Simulation-Based Inference

MCFlow: Monte Carlo Flow Models for Data Imputation

MH-DCNet: an Improved Flow Field Prediction Framework Coupling Neural Network with Physics Solver

Noise Flow: Noise Modeling with Conditional Normalizing Flows

A Flow Model of Neural Networks

Generative Assignment Flows for Representing and Learning Joint Distributions of Discrete Data

On the Universality of Coupling-based Normalizing Flows

Improving and generalizing flow-based generative models with minibatch optimal transport

Efficient, multimodal, and derivative-free Bayesian inference with Fisher-Rao gradient flows

Flow Map Matching

CasFlow: Exploring Hierarchical Structures and Propagation Uncertainty for Cascade Prediction

RG-Flow: A hierarchical and explainable flow model based on renormalization group and sparse prior