Abstract:In this paper, we consider the problem that minimizing the sum of a nonsmooth function with a smooth one in the nonconvex setting, which arising in many contemporary applications such as machine learning, statistics, and signal/image processing. To solve this problem, we propose a new nonmonotone accelerated proximal gradient method with variable stepsize strategy. Note that incorporating inertial term into proximal gradient method is a simple and efficient acceleration technique, while the descent property of the proximal gradient algorithm will lost. In our algorithm, the iterates generated by inertial proximal gradient scheme are accepted when the objective function values decrease or increase appropriately; otherwise, the iteration point is generated by proximal gradient scheme, which makes the function values on a subset of iterates are decreasing. We also introduce a variable stepsize strategy, which does not need a line search or does not need to know the Lipschitz constant and makes the algorithm easy to implement. We show that the sequence of iterates generated by the algorithm converges to a critical point of the objective function. Further, under the assumption that the objective function satisfies the Kurdyka–Łojasiewicz inequality, we prove the convergence rates of the objective function values and the iterates. Moreover, numerical results on both convex and nonconvex problems are reported to demonstrate the effectiveness and superiority of the proposed method and stepsize strategy.

Three-point Step Size Gradient Method with Relaxed Generalized Armijo Step Size Rule

New Two - Point Stepsize Gradient Methods for Solving Unconstrained Optimization Problems

Two efficient gradient methods with approximately optimal stepsizes based on regularization models for unconstrained optimization

On the acceleration of the Barzilai-Borwein method

An Efficient Gradient Method with Approximate Optimal Stepsize for the Strictly Convex Quadratic Minimization Problem

An Efficient Gradient Method with Approximate Optimal Stepsize for Large-Scale Unconstrained Optimization

New Gradient Methods with Adaptive Stepsizes by Approximate Models

An Efficient Gradient Method with Approximately Optimal Stepsizes Based on Regularization Models for Unconstrained Optimization

Gradient Methods with Adaptive Step-Sizes

New Scaling on the Gradient Method

A New Dai-Liao Conjugate Gradient Method based on Approximately Optimal Stepsize for Unconstrained Optimization

Equipping Barzilai-Borwein method with two dimensional quadratic termination property

Some generalized three-term conjugate gradient methods based on CD approach for unconstrained optimization problems

A mechanism of three-dimensional quadratic termination for the gradient method with applications

A New Hybrid Three-Term Conjugate Gradient Algorithm for Large-Scale Unconstrained Problems

A three-term conjugate gradient descent method with some applications

A class of new three-term descent conjugate gradient algorithms for large-scale unconstrained optimization and applications to image restoration problems

A nonmonotone accelerated proximal gradient method with variable stepsize strategy for nonsmooth and nonconvex minimization problems

Two Modified Three-Term Type Conjugate Gradient Methods And Their Global Convergence For Unconstrained Optimization

An Improved Gradient Method with Approximately Optimal Stepsize Based on Conic model for Unconstrained Optimization

Efficient methods for convex problems with bregman barzilai-borwein step sizes