Abstract:The proximal gradient method is a standard approach to solve the composite minimization problems where the objective function is the sum of a continuously differentiable function and a lower semicontinuous, extended-valued function. For both monotone and nonmonotone proximal gradient methods, the convergence theory has traditionally replied heavily on the assumption of global Lipschitz continuity. Recent works have shown that the monotone proximal gradient method, even when the local Lipschitz continuity (rather than global) is assumed, converges to the stationarity globally in the presence of Kurdyka-Łojasiewicz Property. However, how to extend these results from monotone proximal gradient method to nonmonotone proximal gradient method (NPG) remains an open question. In this manuscript, we consider two types of NPG: those combined with average line search and max line search, respectively. By partitioning of indices into two subsets, one of them aims to achieve a decrease in the functional sequence, we establish the global convergence and rate-of-convergence (same as the monotone version) results under the KL property, merely requiring the local Lipschitz assumption, and without an a priori knowledge of the iterative sequence being bounded. When our work is almost done, we noticed that [17] presented the analogous results for the NPG with average line search, whose partitioning of index set is totally different with ours. Drawing upon the findings in this manuscript and [17], we confidently conclude that the convergence theory of NPG is independent on the specific partitioning of the index set.

Proximal Gradient Method with Extrapolation and Line Search for a Class of Nonconvex and Nonsmooth Problems

A class of modified accelerated proximal gradient methods for nonsmooth and nonconvex minimization problems

Bregman Proximal Gradient Algorithm with Extrapolation for a Class of Nonconvex Nonsmooth Minimization Problems

Projective Proximal Gradient Descent for A Class of Nonconvex Nonsmooth Optimization Problems: Fast Convergence Without Kurdyka-Lojasiewicz (KL) Property

Proximal Gradient Method with Automatic Selection of the Parameter by Automatic Differentiation.

Some Accelerated Alternating Proximal Gradient Algorithms for a Class of Nonconvex Nonsmooth Problems

On the Linear Convergence of a Proximal Gradient Method for a Class of Nonsmooth Convex Minimization Problems

Advances in Nonmonotone Proximal Gradient Methods merely with Local Lipschitz Assumptions in the Presense of Kurdyka-Łojasiewicz Property: A Study of Average and Max Line Search

Linear Convergence of Inexact Descent Method and Inexact Proximal Gradient Algorithms for Lower-Order Regularization Problems

A Bregman Proximal Stochastic Gradient Method with Extrapolation for Nonconvex Nonsmooth Problems

Accelerated Proximal Gradient Methods for Nonconvex Programming

Proximal-Proximal-Gradient Method

Riemannian Proximal Gradient Methods (extended Version)

On Convergence Rates of Linearized Proximal Algorithms for Convex Composite Optimization with Applications.

A Simple Proximal Stochastic Gradient Method for Nonsmooth Nonconvex Optimization.

More Efficient Accelerated Proximal Algorithm for Nonconvex Problems

Stochastic Gauss–Seidel type inertial proximal alternating linearized minimization and its application to proximal neural networks

Adaptive proximal algorithms for convex optimization under local Lipschitz continuity of the gradient

Riemannian proximal gradient methods

MGProx: A nonsmooth multigrid proximal gradient method with adaptive restriction for strongly convex optimization

Nonconvex Stochastic Bregman Proximal Gradient Method with Application to Deep Learning