Abstract:Coordinate-type subgradient methods for addressing nonsmooth optimization problems are relatively underexplored due to the set-valued nature of the subdifferential. In this work, our study focuses on nonsmooth composite optimization problems, encompassing a wide class of convex and weakly convex (nonconvex nonsmooth) problems. By utilizing the chain rule of the composite structure properly, we introduce the Randomized Coordinate Subgradient method (RCS) for tackling this problem class. To the best of our knowledge, this is the first coordinate subgradient method for solving general nonsmooth composite optimization problems. In theory, we consider the linearly bounded subgradients assumption for the objective function, which is more general than the traditional Lipschitz continuity assumption, to account for practical scenarios. We then conduct convergence analysis for RCS in both convex and weakly convex cases based on this generalized Lipschitz-type assumption. Specifically, we establish the 𝒪(1/√(k)) convergence rate in expectation and the õ(1/√(k)) almost sure asymptotic convergence rate in terms of the suboptimality gap when f is convex. For the case when f is weakly convex and its subdifferential satisfies the global metric subregularity property, we derive the 𝒪(ε^-4) iteration complexity in expectation. We also establish an asymptotic convergence result. To justify the global metric subregularity property utilized in the analysis, we establish this error bound condition for the concrete (real-valued) robust phase retrieval problem. We also provide a convergence lemma and the relationship between the global metric subregularity properties of a weakly convex function and its Moreau envelope. Finally, we conduct several experiments to demonstrate the possible superiority of RCS over the subgradient method.

Coordinate descent methods beyond smoothness and separability

Coordinate descent methods beyond smoothness and separability

Randomized sketch descent methods for non-separable linearly constrained optimization

Convex and Non-convex Optimization Under Generalized Smoothness

First Order Methods beyond Convexity and Lipschitz Gradient Continuity with Applications to Quadratic Inverse Problems

Smooth over-parameterized solvers for non-smooth structured optimization

Smooth Distance Approximation

Iteration complexity analysis of random coordinate descent methods for $\ell_0$ regularized convex problems

Randomized Coordinate Subgradient Method for Nonsmooth Composite Optimization

Extended convexity and smoothness and their applications in deep learning

Constrained Global Optimization by Smoothing

Riemannian Smoothing Gradient Type Algorithms]{Riemannian Smoothing Gradient Type Algorithms for Nonsmooth Optimization Problem on Compact Riemannian Submanifold Embedded in Euclidean Space

Efficiency of Stochastic Coordinate Proximal Gradient Methods on Nonseparable Composite Optimization

A TRUST-REGION METHOD FOR NONSMOOTH NONCONVEX OPTIMIZATION

Optimizing $(L_0, L_1)$-Smooth Functions by Gradient Methods

Generalized-Smooth Nonconvex Optimization is As Efficient As Smooth Nonconvex Optimization

A Smoothing Stochastic Gradient Method for Composite Optimization

A stochastic moving ball approximation method for smooth convex constrained minimization

Approximation of shape optimization problems with non-smooth PDE constraints

Adaptive Algorithms for Relatively Lipschitz Continuous Convex Optimization Problems