Abstract:The alternating direction method of multipliers (ADMM) is a popular and efficient first-order method that has recently found numerous applications, and the proximal ADMM is an important variant of it. The main contributions of this paper are the proposition and the analysis of a class of inertial proximal ADMMs, which unify the basic ideas of the inertial proximal point method and the proximal ADMM, for linearly constrained separable convex optimization. This class of methods are of inertial nature because at each iteration the proximal ADMM is applied to a point extrapolated at the current iterate in the direction of last movement. The recently proposed inertial primal-dual algorithm [A. Chambolle and T. Pock, On the ergodic convergence rates of a first-order primaldual algorithm, preprint, 2014, Algorithm 3] and the inertial linearized ADMM [C. Chen, S. Ma, and J. Yang, arXiv:1407.8238, eq. (3.23)] are covered as special cases. The proposed algorithmic framework is very general in the sense that the weighting matrices in the proximal terms are allowed to be only positive semidefinite, but not necessarily positive definite as required by existing methods of the same kind. By setting the two proximal terms to zero, we obtain an inertial variant of the classical ADMM, which is to the best of our knowledge new. We carry out a unified analysis for the entire class of methods under very mild assumptions. In particular, convergence, as well as asymptotic o(1/root k) and nonasymptotic O(1/root k) rates of convergence, are established for the best primal function value and feasibility residues, where k denotes the iteration counter. The global iterate convergence of the generated sequence is established under an additional assumption. We also present extensive experimental results on total variation-based image reconstruction problems to illustrate the profits gained by introducing the inertial extrapolation steps.

Inertial self-adaptive algorithms for solving non-smooth convex optimization problems

Two-step inertial Bregman alternating structure-adapted proximal gradient descent algorithm for nonconvex and nonsmooth problems

An innovative inertial extra-proximal gradient algorithm for solving convex optimization problems with application to image and signal processing

Fast inertial dynamic algorithm with smoothing method for nonsmooth convex optimization

Inertial Proximal Admm for Linearly Constrained Separable Convex Optimization

Linearized Proximal Algorithms with Adaptive Stepsizes for Convex Composite Optimization with Applications

New self-adaptive methods with double inertial steps for solving splitting monotone variational inclusion problems with applications

A modified inertial proximal minimization algorithm for structured nonconvex and nonsmooth problem

Self adaptive alternated inertial algorithm for solving variational inequality and fixed point problems

Self-adaptive algorithms with inertial effects for solving the split problem of the demicontractive operators

A modified inertial three-term conjugate gradient method for nonsmooth convex optimization and its application

Multi-step inertial algorithms for equilibrium, fixed point, general systems of variational inequalities and split feasibility problems

A nonmonotone accelerated proximal gradient method with variable stepsize strategy for nonsmooth and nonconvex minimization problems

Fast convex optimization via closed-loop time scaling of gradient dynamics

Fast Multiobjective Gradient Methods with Nesterov Acceleration via Inertial Gradient-Like Systems

A General Inertial Proximal Point Algorithm for Mixed Variational Inequality Problem.

Smoothing accelerated algorithm for constrained nonsmooth convex optimization problems

Convergence Rates of Inertial Splitting Schemes for Nonconvex Composite Optimization

A general inertial proximal point method for mixed variational inequality problem

Convergence of inertial prox-penalization and inertial forward–backward algorithms for solving monotone bilevel equilibrium problems

Modified proximal gradient methods involving double inertial extrapolations for monotone inclusion