Abstract: We develop a new variational approach on level sets aiming towards convergence rate analysis of a variable Bregman proximal gradient (VBPG) method for a broad class of nonsmooth and nonconvex optimization problems. With this new approach, we are able to extend the concepts of Bregman proximal mapping and their corresponding Bregman proximal envelops, Bregman proximal gap function to nonconvex setting. Properties of these mappings and functions are examined. An aim of this work is to provide a solid foundation on which further design and analysis of VBPG for more general nonconvex optimization problems are possible. Another aim is to provide a unified theory on linear convergence of VBPG with a particular interest towards proximal gradient methods. Centrol to our analysis for achieving the above goals is an error bound in terms of level sets and subdifferentials (level-set subdifferential error bound) along with its links to other level-set error bounds. As a consequence, we have established a number of positive results. These newly established results not only enable us to show that any accumulation of the sequence generated by VBPG is at least a critical point of the limiting subdifferential or even a critical point of the proximal subdifferential with a fixed Bregman function in each iteration, but also provide a fresh perspective that allows us to explore inner-connections among many known sufficient conditions for linear convergence of various first-order methods. Along the way, we are able to derive a number of verifiable conditions for level-set error bounds to hold, obtain linear convergence of VBPG, and derive necessary conditions and sufficient conditions for linear convergence relative to a level set for nonsmooth and nonconvex optimization problems.

Variable Projection Algorithms: Theoretical Insights and A Novel Approach for Problems with Large Residual

An Iterative Implementation of Variable Projection for Separable Nonlinear Optimization Problems

Modified Gram–Schmidt Method-Based Variable Projection Algorithm for Separable Nonlinear Models

Robust Variable Projection Algorithm for the Identification of Separable Nonlinear Models

Variable projection algorithms with sparse constraint for separable nonlinear models

A Regularized Variable Projection Algorithm for Separable Nonlinear Least-Squares Problems.

Joint Variable and Variable Projection Algorithms for Separable Nonlinear Models Using Aitken Acceleration Technique

Separable nonlinear least squares: the variable projection method and its applications

Accelerating Variable Projection Algorithm by Aitken Technique.

Convergence Analysis of a Variable Projection Method for Regularized Separable Nonlinear Inverse Problems

An efficient variable projection formulation for separable nonlinear least squares problems.

Variable Programming: A Generalized Minimax Problem. Part Ii: Algorithms

Variable projection methods for separable nonlinear inverse problems with general-form Tikhonov regularization

On Variable-Step Relaxed Projection Algorithm for Variational Inequalities

Nonmonotone variable projection algorithms for matrix decomposition with missing data

Variable Programming: A Generalized Minimax Problem. Part I: Models and Theory

On Convergence Rates of Linearized Proximal Algorithms for Convex Composite Optimization with Applications.

Recursive Variable Projection Algorithm for a Class of Separable Nonlinear Models

Convergence Properties of Projection and Contraction Methods for Variational Inequality Problems

A Variational Approach on Level sets and Linear Convergence of Variable Bregman Proximal Gradient Method for Nonconvex Optimization Problems

Variable Projection Support Vector Machines and Some Applications Using Adaptive Hermite Expansions