Abstract:Understanding stochastic gradient descent (SGD) and its variants is essential for machine learning. However, most of the preceding analyses are conducted under amenable conditions such as unbiased gradient estimator and bounded objective functions, which does not encompass many sophisticated applications, such as variational Monte Carlo, entropy-regularized reinforcement learning and variational inference. In this paper, we consider the SGD algorithm that employ the Markov Chain Monte Carlo (MCMC) estimator to compute the gradient, called MCMC-SGD. Since MCMC reduces the sampling complexity significantly, it is an asymptotically convergent biased estimator in practice. Moreover, by incorporating a general class of unbounded functions, it is much more difficult to analyze the MCMC sampling error. Therefore, we assume that the function is sub-exponential and use the Bernstein inequality for non-stationary Markov chains to derive error bounds of the MCMC estimator. Consequently, MCMC-SGD is proven to have a first order convergence rate $O(\log K/\sqrt{n K})$ with $K$ iterations and a sample size $n$. It partially explains how MCMC influences the behavior of SGD. Furthermore, we verify the correlated negative curvature condition under reasonable assumptions. It is shown that MCMC-SGD escapes from saddle points and reaches $(\epsilon,\epsilon^{1/4})$ approximate second order stationary points or $\epsilon^{1/2}$-variance points at least $O(\epsilon^{-11/2}\log^{2}(1/\epsilon) )$ steps with high probability. Our analysis unveils the convergence pattern of MCMC-SGD across a broad class of stochastic optimization problems, and interprets the convergence phenomena observed in practical applications.

Application of Stochastic Gradient Descent Technique for Method of Moments

ACMo: Angle-Calibrated Moment Methods for Stochastic Optimization

Multi-stage stochastic gradient method with momentum acceleration

Stochastic Sub-Sampled Newton Method with Variance Reduction

Stochastic Gradient Descent in the Viewpoint of Graduated Optimization

Stochastic Generalized Method of Moments.

Convergence Analysis of Stochastic Gradient Descent with MCMC Estimators

A Stochastic Alternating Direction Method of Multipliers for Non-smooth and Non-convex Optimization

A self-scaling memoryless BFGS based conjugate gradient method using multi-step secant condition for unconstrained minimization

Mirror descent method for stochastic multi-objective optimization

Continuation Newton methods with deflation techniques for global optimization problems

A new non-adaptive optimization method: Stochastic gradient descent with momentum and difference

Optimal Adaptive and Accelerated Stochastic Gradient Descent

Combining Conjugate Gradient and Momentum for Unconstrained Stochastic Optimization With Applications to Machine Learning

A Diffusion Approximation Theory of Momentum SGD in Nonconvex Optimization

A three-term conjugate gradient descent method with some applications

Application of Compressed Sensing Theory in the Method of Moments

The Anytime Convergence of Stochastic Gradient Descent with Momentum: From a Continuous-Time Perspective

On Markov Chain Gradient Descent

Stochastic Gradient Made Stable: A Manifold Propagation Approach for Large-Scale Optimization