Abstract:In this paper, we consider the problem that minimizing the sum of a nonsmooth function with a smooth one in the nonconvex setting, which arising in many contemporary applications such as machine learning, statistics, and signal/image processing. To solve this problem, we propose a new nonmonotone accelerated proximal gradient method with variable stepsize strategy. Note that incorporating inertial term into proximal gradient method is a simple and efficient acceleration technique, while the descent property of the proximal gradient algorithm will lost. In our algorithm, the iterates generated by inertial proximal gradient scheme are accepted when the objective function values decrease or increase appropriately; otherwise, the iteration point is generated by proximal gradient scheme, which makes the function values on a subset of iterates are decreasing. We also introduce a variable stepsize strategy, which does not need a line search or does not need to know the Lipschitz constant and makes the algorithm easy to implement. We show that the sequence of iterates generated by the algorithm converges to a critical point of the objective function. Further, under the assumption that the objective function satisfies the Kurdyka–Łojasiewicz inequality, we prove the convergence rates of the objective function values and the iterates. Moreover, numerical results on both convex and nonconvex problems are reported to demonstrate the effectiveness and superiority of the proposed method and stepsize strategy.

A Proximal Alternating Direction Method of Multipliers for a Minimization Problem with Nonconvex Constraints

A Symmetric Alternating Direction Method of Multipliers for Separable Nonconvex Minimization Problems

Proximal Alternating Direction Method of Multipliers with Convex Combination Proximal Centers

A Proximal Alternating Linearization Method for Nonconvex Optimization Problems

Alternating Direction Method of Multipliers with Variable Metric Indefinite Proximal Terms for Convex Optimization

A Proximal Alternating Direction Method of Multipliers for DC Programming with Structured Constraints

A Proximal Point Analysis of the Preconditioned Alternating Direction Method of Multipliers.

A Bregman-style Partially Symmetric Alternating Direction Method of Multipliers for Nonconvex Multi-block Optimization

Majorized Semi-proximal Alternating Coordinate Method for Nonsmooth Convex-Concave Minimax Optimization

Decentralized Proximal Method of Multipliers for Convex Optimization with Coupled Constraints

A family of multi-parameterized proximal point algorithms

Convergence of alternating direction method for minimizing sum of two nonconvex functions with linear constraints

Alternating Direction Method of Multipliers with Difference of Convex Functions.

Decentralized Proximal Method of Multipliers for Convex Optimization with Coupled Constraints

The Rate of Convergence of Proximal Method of Multipliers for Equality Constrained Optimization Problems.

Fast Proximal Linearized Alternating Direction Method of Multiplier with Parallel Splitting

Optimality Conditions and Numerical Algorithms for A Class of Linearly Constrained Minimax Optimization Problems

Semi-Proximal Point Method for Nonsmooth Convex-Concave Minimax Optimization

A Golden Ratio Proximal Alternating Direction Method of Multipliers for Separable Convex Optimization

A Stochastic Alternating Direction Method of Multipliers for Non-smooth and Non-convex Optimization

A nonmonotone accelerated proximal gradient method with variable stepsize strategy for nonsmooth and nonconvex minimization problems