Abstract:The alternating direction method of multipliers (ADMM) is a popular and efficient first-order method that has recently found numerous applications, and the proximal ADMM is an important variant of it. The main contributions of this paper are the proposition and the analysis of a class of inertial proximal ADMMs, which unify the basic ideas of the inertial proximal point method and the proximal ADMM, for linearly constrained separable convex optimization. This class of methods are of inertial nature because at each iteration the proximal ADMM is applied to a point extrapolated at the current iterate in the direction of last movement. The recently proposed inertial primal-dual algorithm [A. Chambolle and T. Pock, On the ergodic convergence rates of a first-order primaldual algorithm, preprint, 2014, Algorithm 3] and the inertial linearized ADMM [C. Chen, S. Ma, and J. Yang, arXiv:1407.8238, eq. (3.23)] are covered as special cases. The proposed algorithmic framework is very general in the sense that the weighting matrices in the proximal terms are allowed to be only positive semidefinite, but not necessarily positive definite as required by existing methods of the same kind. By setting the two proximal terms to zero, we obtain an inertial variant of the classical ADMM, which is to the best of our knowledge new. We carry out a unified analysis for the entire class of methods under very mild assumptions. In particular, convergence, as well as asymptotic o(1/root k) and nonasymptotic O(1/root k) rates of convergence, are established for the best primal function value and feasibility residues, where k denotes the iteration counter. The global iterate convergence of the generated sequence is established under an additional assumption. We also present extensive experimental results on total variation-based image reconstruction problems to illustrate the profits gained by introducing the inertial extrapolation steps.

On a Unified and Simplified Proof for the Ergodic Convergence Rates of PPM, PDHG and ADMM

A Golden Ratio Proximal Alternating Direction Method of Multipliers for Separable Convex Optimization

Proximal Alternating Direction Method of Multipliers with Convex Combination Proximal Centers

Accelerating preconditioned ADMM via degenerate proximal point mappings

A Lyapunov Analysis of Accelerated PDHG Algorithms

Primal Dual Alternating Proximal Gradient Algorithms for Nonsmooth Nonconvex Minimax Problems with Coupled Linear Constraints

On convergence rates of proximal alternating direction method of multipliers

On the Infimal Sub-differential Size of Primal-Dual Hybrid Gradient Method and Beyond

New convergence results of the golden ratio primal-dual algorithm

On the convergence of the distributed proximal point algorithm

Golden Ratio Proximal Gradient ADMM for Distributed Composite Convex Optimization

Dynamical convergence analysis for nonconvex linearized proximal ADMM algorithms

Distributed Primal-Dual Proximal Algorithms for Convex Optimization Involving Three Composite Functions

Zeroth-Order primal-dual Alternating Projection Gradient Algorithms for Nonconvex Minimax Problems with Coupled linear Constraints

Relaxed Proximal Point Algorithm: Tight Complexity Bounds and Acceleration without Momentum

Understanding the PDHG Algorithm via High-Resolution Differential Equations

On the Convergence Analysis of the Alternating Direction Method of Multipliers with Three Blocks

Linear Convergence of the Alternating Direction Method of Multipliers for a Class of Convex Optimization Problems

Inertial Proximal Admm for Linearly Constrained Separable Convex Optimization

A Symplectic Discretization Based Proximal Point Algorithm for Convex Minimization

On Stochastic Primal-Dual Hybrid Gradient Approach for Compositely Regularized Minimization.