Abstract:The purpose of this paper is to answer a few open questions in the interface of kernel methods and PDE gradient flows. Motivated by recent advances in machine learning, particularly in generative modeling and sampling, we present a rigorous investigation of Fisher-Rao and Wasserstein type gradient flows concerning their gradient structures, flow equations, and their kernel approximations. Specifically, we focus on the Fisher-Rao (also known as Hellinger) geometry and its various kernel-based approximations, developing a principled theoretical framework using tools from PDE gradient flows and optimal transport theory. We also provide a complete characterization of gradient flows in the maximum-mean discrepancy (MMD) space, with connections to existing learning and inference algorithms. Our analysis reveals precise theoretical insights linking Fisher-Rao flows, Stein flows, kernel discrepancies, and nonparametric regression. We then rigorously prove evolutionary $\Gamma$-convergence for kernel-approximated Fisher-Rao flows, providing theoretical guarantees beyond pointwise convergence. Finally, we analyze energy dissipation using the Helmholtz-Rayleigh principle, establishing important connections between classical theory in mechanics and modern machine learning practice. Our results provide a unified theoretical foundation for understanding and analyzing approximations of gradient flows in machine learning applications through a rigorous gradient flow and variational method perspective.

Variational Gaussian filtering via Wasserstein gradient flows

Learning Gaussian Mixtures Using the Wasserstein-Fisher-Rao Gradient Flow

Mean-field Variational Inference via Wasserstein Gradient Flow

Variational Gaussian approximation of the Kushner optimal filter

Flexible and Efficient Inference with Particles for the Variational Gaussian Approximation

Bridging the Gap Between Variational Inference and Wasserstein Gradient Flows

Gradient Flows in Filtering and Fisher-Rao Geometry

A Bayesian Filtering Algorithm for Gaussian Mixture Models

Linear Gaussian Regression Filter Based on Variational Bayes.

Variational Adaptive Kalman Filter With Gaussian-Inverse-Wishart Mixture Distribution

Drift Homotopy Methods for A Non-Gaussian Filter

Kernel Approximation of Fisher-Rao Gradient Flows

Approximate inference with Wasserstein gradient flows

Adaptive Maximum Correntropy Gaussian Filter Based on Variational Bayes

Wasserstein Gradient Flow over Variational Parameter Space for Variational Inference

Variational Bayesian Kalman Filter Using Natural Gradient

Gaussian/Gaussian‐mixture Filters for Non‐linear Stochastic Systems with Delayed States

Large-Scale Wasserstein Gradient Flows

Recursive Nonlinear Filtering Via Gaussian Approximation with Minimized Kullback–Leibler Divergence

Particle Flow Gaussian Particle Filter

Solving Fredholm Integral Equations of the First Kind via Wasserstein Gradient Flows