Abstract:In this paper, we consider the problem that minimizing the sum of a nonsmooth function with a smooth one in the nonconvex setting, which arising in many contemporary applications such as machine learning, statistics, and signal/image processing. To solve this problem, we propose a new nonmonotone accelerated proximal gradient method with variable stepsize strategy. Note that incorporating inertial term into proximal gradient method is a simple and efficient acceleration technique, while the descent property of the proximal gradient algorithm will lost. In our algorithm, the iterates generated by inertial proximal gradient scheme are accepted when the objective function values decrease or increase appropriately; otherwise, the iteration point is generated by proximal gradient scheme, which makes the function values on a subset of iterates are decreasing. We also introduce a variable stepsize strategy, which does not need a line search or does not need to know the Lipschitz constant and makes the algorithm easy to implement. We show that the sequence of iterates generated by the algorithm converges to a critical point of the objective function. Further, under the assumption that the objective function satisfies the Kurdyka–Łojasiewicz inequality, we prove the convergence rates of the objective function values and the iterates. Moreover, numerical results on both convex and nonconvex problems are reported to demonstrate the effectiveness and superiority of the proposed method and stepsize strategy.

Scaled Proximal Gradient Methods for Sparse Optimization Problems

Convergence of Projected Subgradient Method with Sparse or Low-Rank Constraints

A Proximal-Gradient Homotopy Method for the Sparse Least-Squares Problem

A Proximal-Gradient Method for Constrained Optimization

Linearized Proximal Algorithms with Adaptive Stepsizes for Convex Composite Optimization with Applications

SignProx: One-bit Proximal Algorithm for Nonconvex Stochastic Optimization

Scaled Proximal Gradient Methods for Multiobjective Optimization: Improved Linear Convergence and Nesterov's Acceleration

Multidimensional extrapolated global proximal gradient and applications for image processing

A Feasible Level Proximal Point Method for Nonconvex Sparse Constrained Optimization

Optimal $k$-thresholding algorithms for sparse optimization problems

Gradient Hard Thresholding Pursuit for Sparsity-Constrained Optimization.

A General Iterative Shrinkage and Thresholding Algorithm for Non-convex Regularized Optimization Problems

An Alternating Proximal Splitting Method With Global Convergence For Nonconvex Structured Sparsity Optimization

A proximal-gradient method for problems with overlapping group-sparse regularization: support identification complexity

An Alternating Manifold Proximal Gradient Method for Sparse Principal Component Analysis and Sparse Canonical Correlation Analysis

A nonmonotone accelerated proximal gradient method with variable stepsize strategy for nonsmooth and nonconvex minimization problems

Proximal methods for structured nonsmooth optimization over Riemannian submanifolds

Smoothing accelerated algorithm for constrained nonsmooth convex optimization problems

Efficient Stochastic Gradient Hard Thresholding

An interior proximal gradient method for nonconvex optimization

On the Suboptimality of Proximal Gradient Descent for $\ell^{0}$ Sparse Approximation