Abstract:This paper studies the asymptotic behavior of the constant step Stochastic Gradient Descent for the minimization of an unknown function, defined as the expectation of a non convex, non smooth, locally Lipschitz random function. As the gradient may not exist, it is replaced by a certain operator: a reasonable choice is to use an element of the Clarke subdifferential of the random function; another choice is the output of the celebrated backpropagation algorithm, which is popular amongst practioners, and whose properties have recently been studied by Bolte and Pauwels. Since the expectation of the chosen operator is not in general an element of the Clarke subdifferential of the mean function, it has been assumed in the literature that an oracle of the Clarke subdifferential of the mean function is available. As a first result, it is shown in this paper that such an oracle is not needed for almost all initialization points of the algorithm. Next, in the small step size regime, it is shown that the interpolated trajectory of the algorithm converges in probability (in the compact convergence sense) towards the set of solutions of a particular differential inclusion: the subgradient flow. Finally, viewing the iterates as a Markov chain whose transition kernel is indexed by the step size, it is shown that the invariant distribution of the kernel converge weakly to the set of invariant distribution of this differential inclusion as the step size tends to zero. These results show that when the step size is small, with large probability, the iterates eventually lie in a neighborhood of the critical points of the mean function.

Gradient Estimation for Smooth Stopping Criteria

Pathwise Derivatives for Multivariate Distributions

Gradient Estimation via Differentiable Metropolis-Hastings

Convergence of Constant Step Stochastic Gradient Descent for Non-Smooth Non-Convex Functions

A probabilistic approach to continuous differentiability of optimal stopping boundaries

A Gradient Smoothed Functional Algorithm with Truncated Cauchy Random Perturbations for Stochastic Optimization

Stopping Criteria for, and Strong Convergence of, Stochastic Gradient Descent on Bottou-Curtis-Nocedal Functions

Gradient Estimation Using Stochastic Computation Graphs

Generalizing Stochastic Smoothing for Differentiation and Gradient Estimation

Drift Control with Discretionary Stopping for a Diffusion Process

Understanding the unstable convergence of gradient descent.

Early stopping for conjugate gradients in statistical inverse problems

Likelihood Ratio Gradient Estimation for Steady-State Parameters

Parametric estimation of stochastic differential equations via online gradient descent

Truncated Cauchy random perturbations for smoothed functional-based stochastic optimization

The Anytime Convergence of Stochastic Gradient Descent with Momentum: From a Continuous-Time Perspective

Asymptotic error analysis for stochastic gradient optimization schemes with first and second order modified equations

On the Monotonicity of the Stopping Boundary for Time-Inhomogeneous Optimal Stopping Problems

Convergence of Markov Chains for Constant Step-size Stochastic Gradient Descent with Separable Functions

A Central Limit Theorem for Algorithmic Estimator of Saddle Point

Conditional Monte Carlo Gradient Estimation