Abstract:In this paper, we consider the problem that minimizing the sum of a nonsmooth function with a smooth one in the nonconvex setting, which arising in many contemporary applications such as machine learning, statistics, and signal/image processing. To solve this problem, we propose a new nonmonotone accelerated proximal gradient method with variable stepsize strategy. Note that incorporating inertial term into proximal gradient method is a simple and efficient acceleration technique, while the descent property of the proximal gradient algorithm will lost. In our algorithm, the iterates generated by inertial proximal gradient scheme are accepted when the objective function values decrease or increase appropriately; otherwise, the iteration point is generated by proximal gradient scheme, which makes the function values on a subset of iterates are decreasing. We also introduce a variable stepsize strategy, which does not need a line search or does not need to know the Lipschitz constant and makes the algorithm easy to implement. We show that the sequence of iterates generated by the algorithm converges to a critical point of the objective function. Further, under the assumption that the objective function satisfies the Kurdyka–Łojasiewicz inequality, we prove the convergence rates of the objective function values and the iterates. Moreover, numerical results on both convex and nonconvex problems are reported to demonstrate the effectiveness and superiority of the proposed method and stepsize strategy.

A New Globally Convergent Algorithm for Non-Lipschitz ℓp-ℓq Minimization

A GLOBALLY AND SUPERLIEARLY CONVERGENT FEASIBLE SQP ALGORITHM FOR DEGENERATE CONSTRAINED OPTIMIZATION

The convergence of conjugate gradient method with nonmonotone line search

Linear Convergence of Inexact Descent Method and Inexact Proximal Gradient Algorithms for Lower-Order Regularization Problems

On Convergence Rates of Linearized Proximal Algorithms for Convex Composite Optimization with Applications.

Revisiting $L_q(0\leq q<1)$ Norm Regularized Optimization

Linearized Proximal Algorithms with Adaptive Stepsizes for Convex Composite Optimization with Applications

Sparse Estimation Via ℓ_q Optimization Method in High-Dimensional Linear Regression

An Aggressive Reduction on the Complexity of Optimization for Non-Strongly Convex Objectives

A nonmonotone accelerated proximal gradient method with variable stepsize strategy for nonsmooth and nonconvex minimization problems

On the Convergence of an Active-Set Method for ℓ1minimization

Group Sparse Optimization Via Lp, Q Regularization.

An inexact regularized proximal Newton method for nonconvex and nonsmooth optimization

Smoothed Proximal Lagrangian Method for Nonlinear Constrained Programs

A Gradient Descent Based Algorithm for ℓ Minimization.

Lp quasi-norm minimization: algorithm and applications

A regularized Newton method for $\ell_q$-norm composite optimization problems

A globally convergent proximal Newton-type method in nonsmooth convex optimization

On Fast Convergence of Proximal Algorithms for SQRT-Lasso Optimization: Don't Worry About Its Nonsmooth Loss Function

A modified inertial proximal minimization algorithm for structured nonconvex and nonsmooth problem