Abstract:In this paper, we consider the problem that minimizing the sum of a nonsmooth function with a smooth one in the nonconvex setting, which arising in many contemporary applications such as machine learning, statistics, and signal/image processing. To solve this problem, we propose a new nonmonotone accelerated proximal gradient method with variable stepsize strategy. Note that incorporating inertial term into proximal gradient method is a simple and efficient acceleration technique, while the descent property of the proximal gradient algorithm will lost. In our algorithm, the iterates generated by inertial proximal gradient scheme are accepted when the objective function values decrease or increase appropriately; otherwise, the iteration point is generated by proximal gradient scheme, which makes the function values on a subset of iterates are decreasing. We also introduce a variable stepsize strategy, which does not need a line search or does not need to know the Lipschitz constant and makes the algorithm easy to implement. We show that the sequence of iterates generated by the algorithm converges to a critical point of the objective function. Further, under the assumption that the objective function satisfies the Kurdyka–Łojasiewicz inequality, we prove the convergence rates of the objective function values and the iterates. Moreover, numerical results on both convex and nonconvex problems are reported to demonstrate the effectiveness and superiority of the proposed method and stepsize strategy.

A New Proximal Iterative Hard Thresholding Method with Extrapolation for $$\ell _0$$ Minimization

An Efficient Interior-Point Algorithm with New Non-Monotone Line Search Filter Method for Nonlinear Constrained Programming

Iterative algorithms for equilibrium problems based on proximal-like methods

Scaled Proximal Gradient Methods for Sparse Optimization Problems

A nonmonotone accelerated proximal gradient method with variable stepsize strategy for nonsmooth and nonconvex minimization problems

Linearized Proximal Algorithms with Adaptive Stepsizes for Convex Composite Optimization with Applications

Global convergence of block proximal iteratively reweighted algorithm with extrapolation

Smoothing fast iterative hard thresholding algorithm for $\ell_0$ regularized nonsmooth convex regression problem

Finite Termination of Inexact Proximal Point Algorithms in Hilbert Spaces

Smoothed Proximal Lagrangian Method for Nonlinear Constrained Programs

Linear Convergence of Inexact Descent Method and Inexact Proximal Gradient Algorithms for Lower-Order Regularization Problems

Optimal $k$-thresholding algorithms for sparse optimization problems

An interior proximal gradient method for nonconvex optimization

Dual Iterative Hard Thresholding: from Non-convex Sparse Minimization to Non-smooth Concave Maximization.

Proximal Point Algorithms on Hadamard Manifolds: Linear Convergence and Finite Termination

Iteratively reweighted $\ell_1$ algorithms with extrapolation

N ov 2 01 6 A Proximal Stochastic Quasi-Newton Algorithm

A Proximal-Gradient Method for Constrained Optimization

An inexact regularized proximal Newton method for nonconvex and nonsmooth optimization

A nested primal–dual iterated Tikhonov method for regularized convex optimization