Abstract:The alternating direction method of multipliers (ADMM) is a popular and efficient first-order method that has recently found numerous applications, and the proximal ADMM is an important variant of it. The main contributions of this paper are the proposition and the analysis of a class of inertial proximal ADMMs, which unify the basic ideas of the inertial proximal point method and the proximal ADMM, for linearly constrained separable convex optimization. This class of methods are of inertial nature because at each iteration the proximal ADMM is applied to a point extrapolated at the current iterate in the direction of last movement. The recently proposed inertial primal-dual algorithm [A. Chambolle and T. Pock, On the ergodic convergence rates of a first-order primaldual algorithm, preprint, 2014, Algorithm 3] and the inertial linearized ADMM [C. Chen, S. Ma, and J. Yang, arXiv:1407.8238, eq. (3.23)] are covered as special cases. The proposed algorithmic framework is very general in the sense that the weighting matrices in the proximal terms are allowed to be only positive semidefinite, but not necessarily positive definite as required by existing methods of the same kind. By setting the two proximal terms to zero, we obtain an inertial variant of the classical ADMM, which is to the best of our knowledge new. We carry out a unified analysis for the entire class of methods under very mild assumptions. In particular, convergence, as well as asymptotic o(1/root k) and nonasymptotic O(1/root k) rates of convergence, are established for the best primal function value and feasibility residues, where k denotes the iteration counter. The global iterate convergence of the generated sequence is established under an additional assumption. We also present extensive experimental results on total variation-based image reconstruction problems to illustrate the profits gained by introducing the inertial extrapolation steps.

Bregman Augmented Lagrangian and Its Acceleration

Damped Proximal Augmented Lagrangian Method for weakly-Convex Problems with Convex Constraints

Global Convergence Analysis of the Power Proximal Point and Augmented Lagrangian Method

On Convergence Rates of Linearized Proximal Algorithms for Convex Composite Optimization with Applications.

convergence rates of (dual-primal) balanced augmented Lagrangian methods for linearly constrained convex programming

Accelerated Bregman Proximal Gradient Methods for Relatively Smooth Convex Optimization

Accelerated Primal-Dual Proximal Gradient Splitting Methods for Convex-Concave Saddle-Point Problems

A Smoothing Augmented Lagrangian Method for Nonconvex, Nonsmooth Constrained Programs and Its Applications to Bilevel Problems

Acceleration and restart for the randomized Bregman-Kaczmarz method

A Bregman-Style Improved ADMM and its Linearized Version in the Nonconvex Setting: Convergence and Rate Analyses

An Augmented Lagrangian Method for Non-Lipschitz Nonconvex Programming

Inexact Augmented Lagrangian Methods for Conic Programs: Quadratic Growth and Linear Convergence

An inexact Bregman proximal point method and its acceleration version for unbalanced optimal transport

Bregman Proximal Gradient Algorithm with Extrapolation for a Class of Nonconvex Nonsmooth Minimization Problems

Accelerated first-order methods for convex optimization with locally Lipschitz continuous gradient

Proximal-Like Incremental Aggregated Gradient Method with Linear Convergence under Bregman Distance Growth Conditions

Inertial Proximal Admm for Linearly Constrained Separable Convex Optimization

Nonconvex and Nonsmooth Optimization with Generalized Orthogonality Constraints: an Approximate Augmented Lagrangian Method

Dynamical convergence analysis for nonconvex linearized proximal ADMM algorithms

Proximal alternating linearized method for nonconvex and nonsmooth problems

Fast Proximal Linearized Alternating Direction Method of Multiplier with Parallel Splitting