Abstract:The subgradient method is one of the most fundamental algorithmic schemes for nonsmooth optimization. The existing complexity and convergence results for this method are mainly derived for Lipschitz continuous objective functions. In this work, we first extend the typical iteration complexity results for the subgradient method to cover non-Lipschitz convex and weakly convex minimization. Specifically, for the convex case, we can drive the suboptimality gap to below $\varepsilon$ in $\mathcal{O}( \varepsilon^{-2} )$ iterations; for the weakly convex case, we can drive the gradient norm of the Moreau envelope of the objective function to below $\varepsilon$ in $\mathcal{O}( \varepsilon^{-4} )$ iterations. Then, we provide convergence results for the subgradient method in the non-Lipschitz setting when proper diminishing rules on the step size are used. In particular, when $f$ is convex, we establish an $\mathcal{O}(\log(k)/\sqrt{k})$ rate of convergence in terms of the suboptimality gap, where $k$ represents the iteration count. With an additional quadratic growth property, the rate is improved to $\mathcal{O}(1/k)$ in terms of the squared distance to the optimal solution set. When $f$ is weakly convex, asymptotic convergence is established. Our results neither require any modification to the subgradient method nor impose any growth condition on the subgradients, while our analysis is surprisingly simple. To further illustrate the wide applicability of our framework, we extend the aforementioned iteration complexity results to cover the truncated subgradient, the stochastic subgradient, and the proximal subgradient methods for non-Lipschitz convex / weakly convex objective functions.

Linear Convergence of Subgradient Algorithm for Convex Feasibility on Riemannian Manifolds

Subgradient Projection Algorithms for Convex Feasibility on Riemannian Manifolds with Lower Bounded Curvatures

Convergence Analysis of Gradient Algorithms on Riemannian Manifolds Without Curvature Constraints and Application to Riemannian Mass

Convergence of Inexact Steepest Descent Algorithm for Multiobjective Optimizations on Riemannian Manifolds Without Curvature Constraints

Subgradient Algorithms on Riemannian Manifolds of Lower Bounded Curvatures

Equilibrium Problems on Riemannian Manifolds with Applications

PROJECTION ALGORITHMS FOR CONVEX FEASIBILITY PROBLEMS ON HADAMARD MANIFOLDS

On Convergence Rates of Linearized Proximal Algorithms for Convex Composite Optimization with Applications.

On Linear Convergence of Non-Euclidean Gradient Methods Without Strong Convexity and Lipschitz Gradient Continuity.

Convergence analysis of the Gauss–Newton method for convex inclusion and convex-composite optimization problems

On the linear convergence rate of Riemannian proximal gradient method

Weakly Convex Optimization over Stiefel Manifold Using Riemannian Subgradient-Type Methods

Variational Analysis Perspective on Linear Convergence of Some First Order Methods for Nonsmooth Convex Optimization Problems

Gauss–Newton Method for Convex Composite Optimizations on Riemannian Manifolds

Proximal Point Algorithms on Hadamard Manifolds: Linear Convergence and Finite Termination

Revisiting Subgradient Method: Complexity and Convergence Beyond Lipschitz Continuity

Linear Regularity and Linear Convergence of Projection-Based Methods for Solving Convex Feasibility Problems

Convergence Rate of Descent Method with New Inexact Line-Search on Riemannian Manifolds

Convergence of a Subgradient Extragradient Algorithm for Solving Monotone Variational Inequalities

An Incremental Subgradient Method on Riemannian Manifolds

Nonsmooth Optimization over Stiefel Manifold: Riemannian Subgradient Methods