Abstract:Understanding stochastic gradient descent (SGD) and its variants is essential for machine learning. However, most of the preceding analyses are conducted under amenable conditions such as unbiased gradient estimator and bounded objective functions, which does not encompass many sophisticated applications, such as variational Monte Carlo, entropy-regularized reinforcement learning and variational inference. In this paper, we consider the SGD algorithm that employ the Markov Chain Monte Carlo (MCMC) estimator to compute the gradient, called MCMC-SGD. Since MCMC reduces the sampling complexity significantly, it is an asymptotically convergent biased estimator in practice. Moreover, by incorporating a general class of unbounded functions, it is much more difficult to analyze the MCMC sampling error. Therefore, we assume that the function is sub-exponential and use the Bernstein inequality for non-stationary Markov chains to derive error bounds of the MCMC estimator. Consequently, MCMC-SGD is proven to have a first order convergence rate $O(\log K/\sqrt{n K})$ with $K$ iterations and a sample size $n$. It partially explains how MCMC influences the behavior of SGD. Furthermore, we verify the correlated negative curvature condition under reasonable assumptions. It is shown that MCMC-SGD escapes from saddle points and reaches $(\epsilon,\epsilon^{1/4})$ approximate second order stationary points or $\epsilon^{1/2}$-variance points at least $O(\epsilon^{-11/2}\log^{2}(1/\epsilon) )$ steps with high probability. Our analysis unveils the convergence pattern of MCMC-SGD across a broad class of stochastic optimization problems, and interprets the convergence phenomena observed in practical applications.

Variance reduction for dependent sequences with applications to Stochastic Gradient MCMC

Accelerated Stochastic ADMM with Variance Reduction

Variance Reduction for the Independent Metropolis Sampler

Stochastic Sub-Sampled Newton Method with Variance Reduction

Stochastic Gradient Hamiltonian Monte Carlo with Variance Reduction for Bayesian Inference.

Diffusion Approximations and Control Variates for MCMC

Theoretical guarantees for neural control variates in MCMC

Neural variance reduction for stochastic differential equations

Stochastic viscosity approximations of Hamilton-Jacobi equations and variance reduction

Variance Reduction in Black-box Variational Inference by Adaptive Importance Sampling

Stochastic Variance-Reduced Majorization-Minimization Algorithms

Variance reduction techniques for stochastic proximal point algorithms

Variance reduction for sequential sampling in stochastic programming

A Vine-Copula Based Adaptive Mcmc Sampler for Efficient Inference of Dynamical Systems

Convergence Analysis of Stochastic Gradient Descent with MCMC Estimators

Improved Variance Reduction Extragradient Method with Line Search for Stochastic Variational Inequalities.

Markov Chain Variance Estimation: A Stochastic Approximation Approach

Stochastic Variance-Reduced Newton: Accelerating Finite-Sum Minimization with Large Batches

Variational mcmc