Abstract:The alternating direction method of multipliers (ADMM) is a popular and efficient first-order method that has recently found numerous applications, and the proximal ADMM is an important variant of it. The main contributions of this paper are the proposition and the analysis of a class of inertial proximal ADMMs, which unify the basic ideas of the inertial proximal point method and the proximal ADMM, for linearly constrained separable convex optimization. This class of methods are of inertial nature because at each iteration the proximal ADMM is applied to a point extrapolated at the current iterate in the direction of last movement. The recently proposed inertial primal-dual algorithm [A. Chambolle and T. Pock, On the ergodic convergence rates of a first-order primaldual algorithm, preprint, 2014, Algorithm 3] and the inertial linearized ADMM [C. Chen, S. Ma, and J. Yang, arXiv:1407.8238, eq. (3.23)] are covered as special cases. The proposed algorithmic framework is very general in the sense that the weighting matrices in the proximal terms are allowed to be only positive semidefinite, but not necessarily positive definite as required by existing methods of the same kind. By setting the two proximal terms to zero, we obtain an inertial variant of the classical ADMM, which is to the best of our knowledge new. We carry out a unified analysis for the entire class of methods under very mild assumptions. In particular, convergence, as well as asymptotic o(1/root k) and nonasymptotic O(1/root k) rates of convergence, are established for the best primal function value and feasibility residues, where k denotes the iteration counter. The global iterate convergence of the generated sequence is established under an additional assumption. We also present extensive experimental results on total variation-based image reconstruction problems to illustrate the profits gained by introducing the inertial extrapolation steps.

An Inertial Proximal Partially Symmetric ADMM-based Algorithm for Linearly Constrained Multi-Block Nonconvex Optimization Problems with Applications.

Inertial Proximal Admm for Linearly Constrained Separable Convex Optimization

A Bregman-style Partially Symmetric Alternating Direction Method of Multipliers for Nonconvex Multi-block Optimization

Sequential inertial linear ADMM algorithm for nonconvex and nonsmooth multiblock problems with nonseparable structure

A Generalization of Linearized Alternating Direction Method of Multipliers for Solving Two-Block Separable Convex Programming

The symmetric ADMM with indefinite proximal regularization and its application

Linearized ADMM for Nonconvex Nonsmooth Optimization with Convergence Analysis

Multi-block relaxed-dual linear inertial ADMM algorithm for nonconvex and nonsmooth problems with nonseparable structures

A class of accelerated GADMM-based method for multi-block nonconvex optimization problems

A Bregman-Style Improved ADMM and its Linearized Version in the Nonconvex Setting: Convergence and Rate Analyses

An inertial ADMM for a class of nonconvex composite optimization with nonlinear coupling constraints

A Unified Algorithmic Framework of Symmetric Gauss-Seidel Decomposition Based Proximal ADMMs for Convex Composite Programming

A Hybrid Type ADMM for Multi-Block Separable Convex Programming

ADMM for Nonconvex Optimization under Minimal Continuity Assumption

A Convergent $3$-Block Semi-Proximal ADMM for Convex Minimization Problems with One Strongly Convex Block

Multiblock ADMM for nonsmooth nonconvex optimization with nonlinear coupling constraints

An Extended ADMM for 3-Block Nonconvex Nonseparable Problems with Applications

Generalized Symmetric ADMM for Separable Convex Optimization

A LINEARLY CONVERGENT MAJORIZED ADMM WITH INDEFINITE PROXIMAL TERMS FOR CONVEX COMPOSITE PROGRAMMING AND ITS APPLICATIONS

An Adaptive Proximal ADMM for Nonconvex Linearly-Constrained Composite Programs

A Stochastic Alternating Direction Method of Multipliers for Non-smooth and Non-convex Optimization