Abstract:In this paper, we consider the problem that minimizing the sum of a nonsmooth function with a smooth one in the nonconvex setting, which arising in many contemporary applications such as machine learning, statistics, and signal/image processing. To solve this problem, we propose a new nonmonotone accelerated proximal gradient method with variable stepsize strategy. Note that incorporating inertial term into proximal gradient method is a simple and efficient acceleration technique, while the descent property of the proximal gradient algorithm will lost. In our algorithm, the iterates generated by inertial proximal gradient scheme are accepted when the objective function values decrease or increase appropriately; otherwise, the iteration point is generated by proximal gradient scheme, which makes the function values on a subset of iterates are decreasing. We also introduce a variable stepsize strategy, which does not need a line search or does not need to know the Lipschitz constant and makes the algorithm easy to implement. We show that the sequence of iterates generated by the algorithm converges to a critical point of the objective function. Further, under the assumption that the objective function satisfies the Kurdyka–Łojasiewicz inequality, we prove the convergence rates of the objective function values and the iterates. Moreover, numerical results on both convex and nonconvex problems are reported to demonstrate the effectiveness and superiority of the proposed method and stepsize strategy.

Linearized Proximal Algorithms with Adaptive Stepsizes for Convex Composite Optimization with Applications

On Convergence Rates of Linearized Proximal Algorithms for Convex Composite Optimization with Applications.

Linear Convergence of Inexact Descent Method and Inexact Proximal Gradient Algorithms for Lower-Order Regularization Problems

Adaptive proximal algorithms for convex optimization under local Lipschitz continuity of the gradient

An Inexact Preconditioned Zeroth-order Proximal Method for Composite Optimization

A nonmonotone accelerated proximal gradient method with variable stepsize strategy for nonsmooth and nonconvex minimization problems

An adaptive proximal point algorithm framework and application to large-scale optimization

Linear Convergence of the Proximal Gradient Method for Composite Optimization Under the Polyak-Łojasiewicz Inequality and Its Variant

Proximal Dogleg Opportunistic Majorization for Nonconvex and Nonsmooth Optimization

A globally convergent proximal Newton-type method in nonsmooth convex optimization

Distributed Algorithms for Composite Optimization: Unified Framework and Convergence Analysis

A Proximal-Gradient Method for Constrained Optimization

Smoothed Proximal Lagrangian Method for Nonlinear Constrained Programs

Parameter-free proximal bundle methods with adaptive stepsizes for hybrid convex composite optimization problems

A linearly convergent method for solving high-order proximal operator

A projected semismooth Newton method for a class of nonconvex composite programs with strong prox-regularity

An Adaptive Regularized Proximal Newton-Type Methods for Composite Optimization over the Stiefel Manifold

A Decentralized Proximal Gradient Tracking Algorithm for Composite Optimization on Riemannian Manifolds

A modified inertial proximal minimization algorithm for structured nonconvex and nonsmooth problem

Safeguarding adaptive methods: global convergence of Barzilai-Borwein and other stepsize choices

On the convergence of adaptive first order methods: proximal gradient and alternating minimization algorithms