Abstract:The formulation min _x,y f(x)+g(y), subject to Ax+By=b, where f and g are extended-value convex functions, arises in many application areas such as signal processing, imaging and image processing, statistics, and machine learning either naturally or after variable splitting. In many common problems, one of the two objective functions is strictly convex and has Lipschitz continuous gradient. On this kind of problem, a very effective approach is the alternating direction method of multipliers (ADM or ADMM), which solves a sequence of f / g -decoupled subproblems. However, its effectiveness has not been matched by a provably fast rate of convergence; only sublinear rates such as O (1 / k ) and O(1/k^2) were recently established in the literature, though the O (1 / k ) rates do not require strong convexity. This paper shows that global linear convergence can be guaranteed under the assumptions of strong convexity and Lipschitz gradient on one of the two functions, along with certain rank assumptions on A and B . The result applies to various generalizations of ADM that allow the subproblems to be solved faster and less exactly in certain manners. The derived rate of convergence also provides some theoretical guidance for optimizing the ADM parameters. In addition, this paper makes meaningful extensions to the existing global convergence theory of ADM generalizations.

Linear Rate Convergence of the Alternating Direction Method of Multipliers for Convex Composite Programming.

Linear Convergence of the Alternating Direction Method of Multipliers for a Class of Convex Optimization Problems

A Generalized Alternating Direction Method of Multipliers with Semi-Proximal Terms for Convex Composite Conic Programming

Local Linear Convergence of the Alternating Direction Method of Multipliers for Quadratic Programs

Convergence Analysis on a Modified Generalized Alternating Direction Method of Multipliers

Convergence Analysis of Generalized ADMM with Majorization for Linearly Constrained Composite Convex Optimization

Linearized Block-Wise Alternating Direction Method of Multipliers for Multiple-Block Convex Programming

A LINEARLY CONVERGENT MAJORIZED ADMM WITH INDEFINITE PROXIMAL TERMS FOR CONVEX COMPOSITE PROGRAMMING AND ITS APPLICATIONS

Convergence Analysis of Alternating Direction Method of Multipliers for a Class of Separable Convex Programming

On the Q-linear Convergence of a Majorized Proximal ADMM for Convex Composite Programming and Its Applications to Regularized Logistic Regression

Linear Convergence Rate of Splitting Algorithms for Multi-Block Constrained Convex Minimizations

On the Convergence of a Generalized Alternating Direction Method of Multipliers with Semi-Proximal Terms

On the Global and Linear Convergence of the Generalized Alternating Direction Method of Multipliers

The Convergence Rate Analysis of the Symmetric ADMM for the Nonconvex Separable Optimization Problems

Linearized ADMM for Nonconvex Nonsmooth Optimization with Convergence Analysis

On the Convergence Analysis of the Alternating Direction Method of Multipliers with Three Blocks

A Bregman-Style Improved ADMM and its Linearized Version in the Nonconvex Setting: Convergence and Rate Analyses

Modified Alternating Direction Method of Multipliers for Convex Quadratic Semidefinite Programming

A Fundamental Proof of Convergence of Alternating Direction Method of Multipliers for Weakly Convex Optimization

On convergence rates of proximal alternating direction method of multipliers

A Linearized Alternating Direction Method of Multipliers with Substitution Procedure.