Abstract:Chung's lemma is a classical tool for establishing asymptotic convergence rates of (stochastic) optimization methods under strong convexity-type assumptions and appropriate polynomial diminishing step sizes. In this work, we develop a generalized version of Chung's lemma, which provides a simple non-asymptotic convergence framework for a more general family of step size rules. We demonstrate broad applicability of the proposed generalized Chung's lemma by deriving tight non-asymptotic convergence rates for a large variety of stochastic methods. In particular, we obtain partially new non-asymptotic complexity results for stochastic optimization methods, such as stochastic gradient descent and random reshuffling, under a general $(\theta,\mu)$-Polyak-Lojasiewicz (PL) condition and for various step sizes strategies, including polynomial, constant, exponential, and cosine step sizes rules. Notably, as a by-product of our analysis, we observe that exponential step sizes can adapt to the objective function's geometry, achieving the optimal convergence rate without requiring exact knowledge of the underlying landscape. Our results demonstrate that the developed variant of Chung's lemma offers a versatile, systematic, and streamlined approach to establish non-asymptotic convergence rates under general step size rules.

A Generalization of Müller’s Iteration Method Based on Standard Information

An Iteration Method with Maximal Order Based on Standard Information.

An Iteration Method with Generally Convergent Property for Cubic Polynomials.

Extraneous Fixed Points of Euler Iteration and Corresponding Sullivan’s Basin

Some Substantial Modifications and Improvements for Derivative-Free Iterative Methods and Derivative-Free Transformation for Multiple Zeros

Improved Muller Method and Bisection Method with Global and Asymptotic Superlinear Convergence of Both Point and Interval for Solving Nonlinear Equations

ON THE CONVERGENT ITERATION METHOD OF ORDER TWO FOR FINDING ZEROS OF THE DERIVATIVE

A Simultaneous Iteration with High Order Convergence.

Convergence of an iterative algorithm for Teichm\"uller maps via generalized harmonic maps

Generalized high-order iterative methods for solutions of nonlinear systems and their applications

Fifth-order Iterative Method for Finding Multiple Roots of Nonlinear Equations

A Novel Iterative Method for Computing Generalized Inverse

Two effective inexact iteration methods for solving the generalized absolute value equations

A generalization of the Newton-based matrix splitting iteration method for generalized absolute value equations

Memory-Accelerating Methods for One-Step Iterative Schemes with Lie Symmetry Method Solving Nonlinear Boundary-Value Problem

Two‐step iterative methods for multiple roots and their applications for solving several physical and chemical problems

Numerical Solution of the Absolute Value Equation Using Modified Iteration Methods

Design of Generalized Fractional Order Gradient Descent Method

A Free-Derivative Iteration Method of Order Three Having Convergence of Both Point and Interval for Nonlinear Equations

Generalization of the Gradient Method with Fractional Order Gradient Direction

A Generalized Version of Chung's Lemma and its Applications