Abstract:Chung's lemma is a classical tool for establishing asymptotic convergence rates of (stochastic) optimization methods under strong convexity-type assumptions and appropriate polynomial diminishing step sizes. In this work, we develop a generalized version of Chung's lemma, which provides a simple non-asymptotic convergence framework for a more general family of step size rules. We demonstrate broad applicability of the proposed generalized Chung's lemma by deriving tight non-asymptotic convergence rates for a large variety of stochastic methods. In particular, we obtain partially new non-asymptotic complexity results for stochastic optimization methods, such as stochastic gradient descent and random reshuffling, under a general $(\theta,\mu)$-Polyak-Lojasiewicz (PL) condition and for various step sizes strategies, including polynomial, constant, exponential, and cosine step sizes rules. Notably, as a by-product of our analysis, we observe that exponential step sizes can adapt to the objective function's geometry, achieving the optimal convergence rate without requiring exact knowledge of the underlying landscape. Our results demonstrate that the developed variant of Chung's lemma offers a versatile, systematic, and streamlined approach to establish non-asymptotic convergence rates under general step size rules.

Two Generalizations of Stampacchia Lemma and Applications

Variations of a Class of Monge–Ampère-type Functionals and Their Applications

A generalization of a lemma of Boccardo and Orsina and application

A Summation Theorem and Its Applications

On the Functional ∫ωf + ∫Ω*g

A Generalized Mountain Pass Lemma with a Closed Subset for Locally Lipschitz Functionals

Local Regularity for Minimizers of Integral Functionals

A Generalized Version of Chung's Lemma and its Applications

Miscellaneous applications of certain minimax theorems II

Local boundedness for minimizers of variational integrals under anisotropic nonstandard growth conditions

The Method of Invariant Sets of Descending Flow for Locally Lipschitz Functionals

Some Applications of Generalized Mountain Pass Lemma

Fatou's Lemma in Its Classic Form and Lebesgue's Convergence Theorems for Varying Measures with Applications to MDPs

Regularity results for minimizers of non-autonomous integral functionals

On De Giorgi's lemma for variational interpolants in metric and Banach spaces

An expanded analysis of local fractional integral inequalities via generalized -convexity

New generalization of perturbed Ostrowski type inequalities and applications

The G Class of Functions and Its Applications

Local behaviour of the mixed local and nonlocal problems with nonstandard growth

Compactness results for a Dirichlet energy of nonlocal gradient with applications

ON THE GENERALIZED NONLINEAR VARIATIONAL INEQUALITY IN BANACH SPACES