Abstract:Understanding stochastic gradient descent (SGD) and its variants is essential for machine learning. However, most of the preceding analyses are conducted under amenable conditions such as unbiased gradient estimator and bounded objective functions, which does not encompass many sophisticated applications, such as variational Monte Carlo, entropy-regularized reinforcement learning and variational inference. In this paper, we consider the SGD algorithm that employ the Markov Chain Monte Carlo (MCMC) estimator to compute the gradient, called MCMC-SGD. Since MCMC reduces the sampling complexity significantly, it is an asymptotically convergent biased estimator in practice. Moreover, by incorporating a general class of unbounded functions, it is much more difficult to analyze the MCMC sampling error. Therefore, we assume that the function is sub-exponential and use the Bernstein inequality for non-stationary Markov chains to derive error bounds of the MCMC estimator. Consequently, MCMC-SGD is proven to have a first order convergence rate $O(\log K/\sqrt{n K})$ with $K$ iterations and a sample size $n$. It partially explains how MCMC influences the behavior of SGD. Furthermore, we verify the correlated negative curvature condition under reasonable assumptions. It is shown that MCMC-SGD escapes from saddle points and reaches $(\epsilon,\epsilon^{1/4})$ approximate second order stationary points or $\epsilon^{1/2}$-variance points at least $O(\epsilon^{-11/2}\log^{2}(1/\epsilon) )$ steps with high probability. Our analysis unveils the convergence pattern of MCMC-SGD across a broad class of stochastic optimization problems, and interprets the convergence phenomena observed in practical applications.

On the Convergence of Gradient Descent in GANs: MMD GAN As a Gradient Flow

On gradient regularizers for MMD GANs

Local Convergence of Gradient Descent-Ascent for Training Generative Adversarial Networks

Convergence Dynamics of Generative Adversarial Networks: The Dual Metric Flows

(De)-regularized Maximum Mean Discrepancy Gradient Flow

On Gradient Descent Ascent for Nonconvex-Concave Minimax Problems

Generative Sliced MMD Flows with Riesz Kernels

Sobolev Descent

Convergence of flow-based generative models via proximal gradient descent in Wasserstein space

Convergence of Non-Convex Non-Concave GANs Using Sinkhorn Divergence

Discretized Gradient Flow for Manifold Learning in the Space of Embeddings

Global Convergence to the Equilibrium of GANs using Variational Inequalities

A Concentration Inequality for Maximum Mean Discrepancy (MMD)-based Statistics and Its Application in Generative Models

On Convergence and Stability of GANs

On the Convergence and Robustness of Training GANs with Regularized Optimal Transport

Convergence of Policy Gradient for Entropy Regularized MDPs with Neural Network Approximation in the Mean-Field Regime

Posterior Sampling Based on Gradient Flows of the MMD with Negative Distance Kernel

Deep Generative Learning via Variational Gradient Flow

Convergence Analysis of Stochastic Gradient Descent with MCMC Estimators

Convergence of GANs Training: A Game and Stochastic Control Methodology

Train simultaneously, generalize better: Stability of gradient-based minimax learners