Abstract:In this paper, we study the convergence properties of the Stochastic Gradient Descent (SGD) method for finding a stationary point of a given objective function $J(\cdot)$. The objective function is not required to be convex. Rather, our results apply to a class of ``invex'' functions, which have the property that every stationary point is also a global minimizer. First, it is assumed that $J(\cdot)$ satisfies a property that is slightly weaker than the Kurdyka-Lojasiewicz (KL) condition, denoted here as (KL'). It is shown that the iterations $J(\boldsymbol{\theta}_t)$ converge almost surely to the global minimum of $J(\cdot)$. Next, the hypothesis on $J(\cdot)$ is strengthened from (KL') to the Polyak-Lojasiewicz (PL) condition. With this stronger hypothesis, we derive estimates on the rate of convergence of $J(\boldsymbol{\theta}_t)$ to its limit. Using these results, we show that for functions satisfying the PL property, the convergence rate of both the objective function and the norm of the gradient with SGD is the same as the best-possible rate for convex functions. While some results along these lines have been published in the past, our contributions contain two distinct improvements. First, the assumptions on the stochastic gradient are more general than elsewhere, and second, our convergence is almost sure, and not in expectation. We also study SGD when only function evaluations are permitted. In this setting, we determine the ``optimal'' increments or the size of the perturbations. Using the same set of ideas, we establish the global convergence of the Stochastic Approximation (SA) algorithm under more general assumptions on the measurement error, compared to the existing literature. We also derive bounds on the rate of convergence of the SA algorithm under appropriate assumptions.

Sufficient and necessary condition for the convergence of stochastic approximation algorithms

ON THE COMPLETE CONVERGENCE OF U-STATISTICS

A Unified Convergence Theorem

Conditions for uniform in time convergence: applications to averaging, numerical discretisations and mean-field systems

Central Limit Theorems of a Recursive Stochastic Algorithm with Applications to Adaptive Designs

Strong Approximation Theorems for Sums of Random Variables when Extreme Terms Are Excluded

The Convergence of the Sums of Independent Random Variables under the Sub-Linear Expectations

Criterion for convergence almost everywhere, with applications

Stochastic Successive Convex Approximation for Non-Convex Constrained Stochastic Optimization

Formalization of a Stochastic Approximation Theorem

Finite-Time Decoupled Convergence in Nonlinear Two-Time-Scale Stochastic Approximation

Convergence of Density Approximations for Stochastic Heat Equation

Sufficient and Necessary Condition for the Asymptotic Stability of Stochastic Systems With Discrete Time Feedbacks and Applications

On the complete convergence for uncertain random variables

Convergence Rates for Stochastic Approximation: Biased Noise with Unbounded Variance, and Applications

Necessary and sufficient convergence conditions for algebraic image reconstruction algorithms.

Stochastic Approximation of Smooth and Strongly Convex Functions: Beyond the Convergence Rate

The Stability and Convergence of Computing Long-Time Behaviour

Stochastic Convergence of Regularized Solutions and Their Finite Element Approximations to Inverse Source Problems

Asymptotic Time-Uniform Inference for Parameters in Averaged Stochastic Approximation

New Sufficient Conditions for Finite Time Stability of Nonlinear Time Delay Systems