Abstract:In this paper, we consider the problem that minimizing the sum of a nonsmooth function with a smooth one in the nonconvex setting, which arising in many contemporary applications such as machine learning, statistics, and signal/image processing. To solve this problem, we propose a new nonmonotone accelerated proximal gradient method with variable stepsize strategy. Note that incorporating inertial term into proximal gradient method is a simple and efficient acceleration technique, while the descent property of the proximal gradient algorithm will lost. In our algorithm, the iterates generated by inertial proximal gradient scheme are accepted when the objective function values decrease or increase appropriately; otherwise, the iteration point is generated by proximal gradient scheme, which makes the function values on a subset of iterates are decreasing. We also introduce a variable stepsize strategy, which does not need a line search or does not need to know the Lipschitz constant and makes the algorithm easy to implement. We show that the sequence of iterates generated by the algorithm converges to a critical point of the objective function. Further, under the assumption that the objective function satisfies the Kurdyka–Łojasiewicz inequality, we prove the convergence rates of the objective function values and the iterates. Moreover, numerical results on both convex and nonconvex problems are reported to demonstrate the effectiveness and superiority of the proposed method and stepsize strategy.

Smoothing Accelerated Proximal Gradient Method with Fast Convergence Rate for Nonsmooth Multi-objective Optimization

Smoothing Accelerated Proximal Gradient Method with Fast Convergence Rate for Nonsmooth Convex Optimization Beyond Differentiability

Scaled Proximal Gradient Methods for Multiobjective Optimization: Improved Linear Convergence and Nesterov's Acceleration

The convergence rate of the accelerated proximal gradient algorithm for Multiobjective Optimization is faster than $O(1/k^2)$

Barzilai-Borwein Proximal Gradient Methods for Multiobjective Composite Optimization Problems with Improved Linear Convergence

Accelerated Quasi-Newton Proximal Extragradient: Faster Rate for Smooth Convex Optimization

A class of modified accelerated proximal gradient methods for nonsmooth and nonconvex minimization problems

A feasible smoothing accelerated projected gradient method for nonsmooth convex optimization

Accelerated Proximal Gradient Methods for Nonconvex Programming

A Bregman Proximal Stochastic Gradient Method with Extrapolation for Nonconvex Nonsmooth Problems

More Efficient Accelerated Proximal Algorithm for Nonconvex Problems

Smoothing accelerated algorithm for constrained nonsmooth convex optimization problems

On the Convergence of Newton-type Proximal Gradient Method for Multiobjective Optimization Problems

Stochastic smoothing accelerated gradient method for nonsmooth convex composite optimization

A nonmonotone accelerated proximal gradient method with variable stepsize strategy for nonsmooth and nonconvex minimization problems

A Proximal Gradient Method with an Explicit Line search for Multiobjective Optimization

Accelerated Proximal Coordinate Gradient Methods

Fast Multiobjective Gradient Methods with Nesterov Acceleration via Inertial Gradient-Like Systems

Accelerated gradient methods for nonconvex nonlinear and stochastic programming

Projective Proximal Gradient Descent for A Class of Nonconvex Nonsmooth Optimization Problems: Fast Convergence Without Kurdyka-Lojasiewicz (KL) Property

A Trust Region Proximal Gradient Method for Nonlinear Multi-objective Optimization Problems