Abstract:We study high-probability convergence guarantees of learning on streaming data in the presence of heavy-tailed noise. In the proposed scenario, the model is updated in an online fashion, as new information is observed, without storing any additional data. To combat the heavy-tailed noise, we consider a general framework of nonlinear stochastic gradient descent (SGD), providing several strong results. First, for non-convex costs and component-wise nonlinearities, we establish a convergence rate arbitrarily close to $\mathcal{O}\left(t^{-\frac{1}{4}}\right)$, whose exponent is independent of noise and problem parameters. Second, for strongly convex costs and component-wise nonlinearities, we establish a rate arbitrarily close to $\mathcal{O}\left(t^{-\frac{1}{2}}\right)$ for the weighted average of iterates, with exponent again independent of noise and problem parameters. Finally, for strongly convex costs and a broader class of nonlinearities, we establish convergence of the last iterate, with a rate $\mathcal{O}\left(t^{-\zeta} \right)$, where $\zeta \in (0,1)$ depends on problem parameters, noise and nonlinearity. As we show analytically and numerically, $\zeta$ can be used to inform the preferred choice of nonlinearity for given problem settings. Compared to state-of-the-art, who only consider clipping, require bounded noise moments of order $\eta \in (1,2]$, and establish convergence rates whose exponents go to zero as $\eta \rightarrow 1$, we provide high-probability guarantees for a much broader class of nonlinearities and symmetric density noise, with convergence rates whose exponents are bounded away from zero, even when the noise has finite first moment only. Moreover, in the case of strongly convex functions, we demonstrate analytically and numerically that clipping is not always the optimal nonlinearity, further underlining the value of our general framework.

Almost Sure Convergence Rates and Concentration of Stochastic Approximation and Reinforcement Learning with Markovian Noise

Concentration of Contractive Stochastic Approximation: Additive and Multiplicative Noise

On Almost Sure Convergence Rates of Stochastic Gradient Methods.

On Almost Sure Convergence Rates of Stochastic Gradient Methods

Almost Sure Convergence Rates Analysis and Saddle Avoidance of Stochastic Gradient Methods

Finite-Time Convergence Rates of Decentralized Local Markovian Stochastic Approximation

The ODE Method for Stochastic Approximation and Reinforcement Learning with Markovian Noise

Almost sure convergence rates of stochastic gradient methods under gradient domination

Stochastic Approximation with Unbounded Markovian Noise: A General-Purpose Theorem

High Probability Convergence Bounds for Non-convex Stochastic Gradient Descent with Sub-Weibull Noise

Convergence Rates for Stochastic Approximation: Biased Noise with Unbounded Variance, and Applications

High-probability Convergence Bounds for Nonlinear Stochastic Gradient Descent Under Heavy-tailed Noise

Almost Sure Synchronization for Nonlinear Complex Stochastic Networks with Lévy Noise

Stochastic Gradient and Langevin Processes

Tight Finite Time Bounds of Two-Time-Scale Linear Stochastic Approximation with Markovian Noise

Almost Sure Convergence of Average Reward Temporal Difference Learning

Markovian Foundations for Quasi-Stochastic Approximation with Applications to Extremum Seeking Control

Finite-Time Error Bounds for Distributed Linear Stochastic Approximation

Concentration Bounds for Co-occurrence Matrices of Markov Chains.

Stochastic Nonsmooth Convex Optimization with Heavy-Tailed Noises: High-Probability Bound, In-Expectation Rate and Initial Distance Adaptation

Almost Sure Convergence of the Numerical Discretization of Stochastic Jump Diffusions