Abstract:We provide finite-particle convergence rates for the Stein Variational Gradient Descent (SVGD) algorithm in the Kernelized Stein Discrepancy ($\mathsf{KSD}$) and Wasserstein-2 metrics. Our key insight is that the time derivative of the relative entropy between the joint density of $N$ particle locations and the $N$-fold product target measure, starting from a regular initial distribution, splits into a dominant `negative part' proportional to $N$ times the expected $\mathsf{KSD}^2$ and a smaller `positive part'. This observation leads to $\mathsf{KSD}$ rates of order $1/\sqrt{N}$, in both continuous and discrete time, providing a near optimal (in the sense of matching the corresponding i.i.d. rates) double exponential improvement over the recent result by Shi and Mackey (2024). Under mild assumptions on the kernel and potential, these bounds also grow polynomially in the dimension $d$. By adding a bilinear component to the kernel, the above approach is used to further obtain Wasserstein-2 convergence in continuous time. For the case of `bilinear + Matérn' kernels, we derive Wasserstein-2 rates that exhibit a curse-of-dimensionality similar to the i.i.d. setting. We also obtain marginal convergence and long-time propagation of chaos results for the time-averaged particle laws.

An Exposition of Götze's Estimation of the Rate of Convergence in the Multivariate Central Limit Theorem

The Multivariate Rate of Convergence for Selberg's Central Limit Theorem

The Rate of Convergence for Selberg's Central Limit Theorem under the Riemann Hypothesis

Central limit theorems with a rate of convergence for sequences of transformations

Kac's Central Limit Theorem by Stein's Method

Multidimensional Stein's method for Gamma approximation

Stein kernels for $q$-moment measures and new bounds for the rate of convergence in the central limit theorem

Bounds for distributional approximation in the multivariate delta method by Stein's method

On Stein's method and mod-* convergence

Bounding the -Distance Between One-Dimensional Continuous and Discrete Distributions via Stein's Method

Convergence rates for estimating multivariate scale mixtures of uniform densities

Stein's Method for Spatial Random Graphs

Moment estimates, exponential integrability, concentration inequalities and exit times estimates on evolving manifolds

A Stein characterisation of the distribution of the product of correlated normal random variables

On the speed of convergence in the strong density theorem

Moderate deviation principles and Mod-Gaussian convergence for lacunary trigonometric sums

Convergence and concentration properties of constant step-size SGD through Markov chains

Improved rates of convergence for the multivariate Central Limit Theorem in Wasserstein distance

A monotone scheme for G-equations with application to the explicit convergence rate of robust central limit theorem

On Stein factors in Stein's method for normal approximation

Improved Finite-Particle Convergence Rates for Stein Variational Gradient Descent