Abstract: We develop a new variational approach on level sets aiming towards convergence rate analysis of a variable Bregman proximal gradient (VBPG) method for a broad class of nonsmooth and nonconvex optimization problems. With this new approach, we are able to extend the concepts of Bregman proximal mapping and their corresponding Bregman proximal envelops, Bregman proximal gap function to nonconvex setting. Properties of these mappings and functions are examined. An aim of this work is to provide a solid foundation on which further design and analysis of VBPG for more general nonconvex optimization problems are possible. Another aim is to provide a unified theory on linear convergence of VBPG with a particular interest towards proximal gradient methods. Centrol to our analysis for achieving the above goals is an error bound in terms of level sets and subdifferentials (level-set subdifferential error bound) along with its links to other level-set error bounds. As a consequence, we have established a number of positive results. These newly established results not only enable us to show that any accumulation of the sequence generated by VBPG is at least a critical point of the limiting subdifferential or even a critical point of the proximal subdifferential with a fixed Bregman function in each iteration, but also provide a fresh perspective that allows us to explore inner-connections among many known sufficient conditions for linear convergence of various first-order methods. Along the way, we are able to derive a number of verifiable conditions for level-set error bounds to hold, obtain linear convergence of VBPG, and derive necessary conditions and sufficient conditions for linear convergence relative to a level set for nonsmooth and nonconvex optimization problems.

On Linear Convergence of Non-Euclidean Gradient Methods Without Strong Convexity and Lipschitz Gradient Continuity.

The convergence of conjugate gradient method with nonmonotone line search

Convergence Analysis of Gradient Algorithms on Riemannian Manifolds Without Curvature Constraints and Application to Riemannian Mass

Linear Convergence of Subgradient Algorithm for Convex Feasibility on Riemannian Manifolds

Convergence analysis of the Gauss–Newton method for convex inclusion and convex-composite optimization problems

Convergence Properties of Nonlinear Conjugate Gradient Methods.

Linear Convergence of Gradient and Proximal-Gradient Methods Under the Polyak-Łojasiewicz Condition

Linear convergence of first order methods for non-strongly convex optimization

Linear Convergence of Adaptive Stochastic Gradient Descent

Linear convergence of forward-backward accelerated algorithms without knowledge of the modulus of strong convexity

On the Linear Convergence of a Proximal Gradient Method for a Class of Nonsmooth Convex Minimization Problems

Conditions for linear convergence of the gradient method for non-convex optimization

A Local Nearly Linearly Convergent First-Order Method for Nonsmooth Functions with Quadratic Growth

A nearly linearly convergent first-order method for nonsmooth functions with quadratic growth

A Variational Approach on Level sets and Linear Convergence of Variable Bregman Proximal Gradient Method for Nonconvex Optimization Problems

On the Global and Linear Convergence of the Generalized Alternating Direction Method of Multipliers

Linear Convergence Rate in Convex Setup is Possible! Gradient Descent Method Variants under $(L_0,L_1)$-Smoothness

Restricted strong convexity and its applications to convergence analysis of gradient-type methods in convex optimization

Convergence of Nonmonotone Proximal Gradient Methods under the Kurdyka-Lojasiewicz Property without a Global Lipschitz Assumption

Characterization of Linear Convergence of Gradient Descent