Abstract:Understanding stochastic gradient descent (SGD) and its variants is essential for machine learning. However, most of the preceding analyses are conducted under amenable conditions such as unbiased gradient estimator and bounded objective functions, which does not encompass many sophisticated applications, such as variational Monte Carlo, entropy-regularized reinforcement learning and variational inference. In this paper, we consider the SGD algorithm that employ the Markov Chain Monte Carlo (MCMC) estimator to compute the gradient, called MCMC-SGD. Since MCMC reduces the sampling complexity significantly, it is an asymptotically convergent biased estimator in practice. Moreover, by incorporating a general class of unbounded functions, it is much more difficult to analyze the MCMC sampling error. Therefore, we assume that the function is sub-exponential and use the Bernstein inequality for non-stationary Markov chains to derive error bounds of the MCMC estimator. Consequently, MCMC-SGD is proven to have a first order convergence rate $O(\log K/\sqrt{n K})$ with $K$ iterations and a sample size $n$. It partially explains how MCMC influences the behavior of SGD. Furthermore, we verify the correlated negative curvature condition under reasonable assumptions. It is shown that MCMC-SGD escapes from saddle points and reaches $(\epsilon,\epsilon^{1/4})$ approximate second order stationary points or $\epsilon^{1/2}$-variance points at least $O(\epsilon^{-11/2}\log^{2}(1/\epsilon) )$ steps with high probability. Our analysis unveils the convergence pattern of MCMC-SGD across a broad class of stochastic optimization problems, and interprets the convergence phenomena observed in practical applications.

An Improved Analysis of Stochastic Gradient Descent with Momentum

Convergence of SGD with momentum in the nonconvex case: A time window-based analysis

Acceleration of stochastic gradient descent with momentum by averaging: finite-sample rates and asymptotic normality

The Anytime Convergence of Stochastic Gradient Descent with Momentum: From a Continuous-Time Perspective

A Unified Analysis of Stochastic Momentum Methods for Deep Learning

Stochastic Momentum Method with Double Acceleration for Regularized Empirical Risk Minimization

A Unified Analysis of Stochastic Momentum Methods for Deep Learning.

On the Convergence of Memory-Based Distributed SGD.

When and Why Momentum Accelerates SGD:An Empirical Study

A Unified Analysis of AdaGrad With Weighted Aggregation and Momentum Acceleration

Convergence rates of stochastic gradient method with independent sequences of step-size and momentum weight

Stochastic Gradient Descent with Nonlinear Conjugate Gradient-Style Adaptive Momentum

Momentum Ensures Convergence of SIGNSGD under Weaker Assumptions.

Convergence Analysis of Stochastic Gradient Descent with MCMC Estimators

$μ^2$-SGD: Stable Stochastic Optimization via a Double Momentum Mechanism

A Diffusion Approximation Theory of Momentum SGD in Nonconvex Optimization

On the Generalization of Stochastic Gradient Descent with Momentum

Convergence and Stability of the Stochastic Proximal Point Algorithm with Momentum

Optimal Adaptive and Accelerated Stochastic Gradient Descent

Bound Analysis of Natural Gradient Descent in Stochastic Optimization Setting

Convergence Analysis of Accelerated Stochastic Gradient Descent under the Growth Condition