Abstract:Understanding stochastic gradient descent (SGD) and its variants is essential for machine learning. However, most of the preceding analyses are conducted under amenable conditions such as unbiased gradient estimator and bounded objective functions, which does not encompass many sophisticated applications, such as variational Monte Carlo, entropy-regularized reinforcement learning and variational inference. In this paper, we consider the SGD algorithm that employ the Markov Chain Monte Carlo (MCMC) estimator to compute the gradient, called MCMC-SGD. Since MCMC reduces the sampling complexity significantly, it is an asymptotically convergent biased estimator in practice. Moreover, by incorporating a general class of unbounded functions, it is much more difficult to analyze the MCMC sampling error. Therefore, we assume that the function is sub-exponential and use the Bernstein inequality for non-stationary Markov chains to derive error bounds of the MCMC estimator. Consequently, MCMC-SGD is proven to have a first order convergence rate $O(\log K/\sqrt{n K})$ with $K$ iterations and a sample size $n$. It partially explains how MCMC influences the behavior of SGD. Furthermore, we verify the correlated negative curvature condition under reasonable assumptions. It is shown that MCMC-SGD escapes from saddle points and reaches $(\epsilon,\epsilon^{1/4})$ approximate second order stationary points or $\epsilon^{1/2}$-variance points at least $O(\epsilon^{-11/2}\log^{2}(1/\epsilon) )$ steps with high probability. Our analysis unveils the convergence pattern of MCMC-SGD across a broad class of stochastic optimization problems, and interprets the convergence phenomena observed in practical applications.

Convergence of Riemannian Stochastic Gradient Descent on Hadamard Manifold

Convergence Analysis of Gradient Algorithms on Riemannian Manifolds Without Curvature Constraints and Application to Riemannian Mass

Riemannian Stochastic Hybrid Gradient Algorithm for Nonconvex Optimization

Convergence Analysis of Stochastic Gradient Descent with MCMC Estimators

Convergence and concentration properties of constant step-size SGD through Markov chains

Convergence of Hyperbolic Neural Networks under Riemannian Stochastic Gradient Descent

A general framework of Riemannian adaptive optimization methods with a convergence analysis

Demystifying SGD with Doubly Stochastic Gradients

Gradient descent for unbounded convex functions on Hadamard manifolds and its applications to scaling problems

Learning-Rate-Free Stochastic Optimization over Riemannian Manifolds

Convergence rates of stochastic gradient method with independent sequences of step-size and momentum weight

Randomized Gradient Descents on Riemannian Manifolds: Almost Sure Convergence to Global Minima in and beyond Quantum Optimization

Convergence of Markov Chains for Constant Step-size Stochastic Gradient Descent with Separable Functions

Stochastic Gradient Made Stable: A Manifold Propagation Approach for Large-Scale Optimization

Demystifying the Myths and Legends of Nonconvex Convergence of SGD

On the Convergence of A Data-Driven Regularized Stochastic Gradient Descent for Nonlinear Ill-Posed Problems

Stochastic Modified Flows for Riemannian Stochastic Gradient Descent

Riemannian stochastic optimization methods avoid strict saddle points

The Anytime Convergence of Stochastic Gradient Descent with Momentum: From a Continuous-Time Perspective

Decentralized Online Riemannian Optimization with Dynamic Environments

Painless Stochastic Conjugate Gradient for Large-Scale Machine Learning