Abstract:The alternating direction method of multipliers (ADMM) is a popular and efficient first-order method that has recently found numerous applications, and the proximal ADMM is an important variant of it. The main contributions of this paper are the proposition and the analysis of a class of inertial proximal ADMMs, which unify the basic ideas of the inertial proximal point method and the proximal ADMM, for linearly constrained separable convex optimization. This class of methods are of inertial nature because at each iteration the proximal ADMM is applied to a point extrapolated at the current iterate in the direction of last movement. The recently proposed inertial primal-dual algorithm [A. Chambolle and T. Pock, On the ergodic convergence rates of a first-order primaldual algorithm, preprint, 2014, Algorithm 3] and the inertial linearized ADMM [C. Chen, S. Ma, and J. Yang, arXiv:1407.8238, eq. (3.23)] are covered as special cases. The proposed algorithmic framework is very general in the sense that the weighting matrices in the proximal terms are allowed to be only positive semidefinite, but not necessarily positive definite as required by existing methods of the same kind. By setting the two proximal terms to zero, we obtain an inertial variant of the classical ADMM, which is to the best of our knowledge new. We carry out a unified analysis for the entire class of methods under very mild assumptions. In particular, convergence, as well as asymptotic o(1/root k) and nonasymptotic O(1/root k) rates of convergence, are established for the best primal function value and feasibility residues, where k denotes the iteration counter. The global iterate convergence of the generated sequence is established under an additional assumption. We also present extensive experimental results on total variation-based image reconstruction problems to illustrate the profits gained by introducing the inertial extrapolation steps.

A parallel alternating direction method with application to compound l1-regularized imaging inverse problems.

Regularized Deconvolution for Structured Illumination Microscopy Via Accelerated Linearized ADMM

Parallel linearized ADMM with application to multichannel image restoration and reconstruction

Dual Alternating Direction Method of Multipliers for Inverse Imaging

An alternating direction method of multipliers for inverse lithography problem

A splitting primal-dual proximity algorithm for solving composite optimization problems

Alternating Direction Method of Multipliers for Linear Inverse Problems

An Efficient Algorithm for Spatial-Spectral Partial Volume Compartment Mapping with Applications to Multicomponent Diffusion and Relaxation MRI

A Unified Alternating Direction Method of Multipliers by Majorization Minimization

Proximal Alternating Direction Method of Multipliers with Convex Combination Proximal Centers

An Inexact Dual Alternating Direction Method of Multipliers for Image Decomposition and Restoration

Inertial Proximal Admm for Linearly Constrained Separable Convex Optimization

Alternating direction method of multipliers for linear hyperspectral unmixing

Relaxing Alternating Direction Method of Multipliers (ADMM) for Linear Inverse Problems

Convergence for nonconvex ADMM, with applications to CT imaging

ADMM-ADAM: A New Inverse Imaging Framework Blending the Advantages of Convex Optimization and Deep Learning

A Generalization of Linearized Alternating Direction Method of Multipliers for Solving Two-Block Separable Convex Programming

Parallel Direction Method of Multipliers

Linearized Alternating Direction Method with Parallel Splitting and Adaptive Penalty for Separable Convex Programs in Machine Learning

A HYBRID ALTERNATING MINIMIZATION ALGORITHM FOR STRUCTURED CONVEX OPTIMIZATION PROBLEMS WITH APPLICATION IN POISSONIAN IMAGE PROCESSING

A Stochastic Alternating Direction Method of Multipliers for Non-smooth and Non-convex Optimization