Abstract:The formulation min _x,y f(x)+g(y), subject to Ax+By=b, where f and g are extended-value convex functions, arises in many application areas such as signal processing, imaging and image processing, statistics, and machine learning either naturally or after variable splitting. In many common problems, one of the two objective functions is strictly convex and has Lipschitz continuous gradient. On this kind of problem, a very effective approach is the alternating direction method of multipliers (ADM or ADMM), which solves a sequence of f / g -decoupled subproblems. However, its effectiveness has not been matched by a provably fast rate of convergence; only sublinear rates such as O (1 / k ) and O(1/k^2) were recently established in the literature, though the O (1 / k ) rates do not require strong convexity. This paper shows that global linear convergence can be guaranteed under the assumptions of strong convexity and Lipschitz gradient on one of the two functions, along with certain rank assumptions on A and B . The result applies to various generalizations of ADM that allow the subproblems to be solved faster and less exactly in certain manners. The derived rate of convergence also provides some theoretical guidance for optimizing the ADM parameters. In addition, this paper makes meaningful extensions to the existing global convergence theory of ADM generalizations.

On the Convergence of a Generalized Alternating Direction Method of Multipliers with Semi-Proximal Terms

A Generalized Alternating Direction Method of Multipliers with Semi-Proximal Terms for Convex Composite Conic Programming

Convergence Analysis on a Modified Generalized Alternating Direction Method of Multipliers

Convergence Analysis of Alternating Direction Method of Multipliers for a Class of Separable Convex Programming

On the Convergence Analysis of the Alternating Direction Method of Multipliers with Three Blocks

Linear Rate Convergence of the Alternating Direction Method of Multipliers for Convex Composite Programming.

Alternating Direction Method of Multipliers with Variable Metric Indefinite Proximal Terms for Convex Optimization

Local Linear Convergence of the Alternating Direction Method of Multipliers for Nonconvex Separable Optimization Problems

A Homotopy Alternating Direction Method of Multipliers for Linearly Constrained Separable Convex Optimization

A Hybrid Bregman Alternating Direction Method of Multipliers for the Linearly Constrained Difference-of-convex Problems

Convergence Analysis of Generalized ADMM with Majorization for Linearly Constrained Composite Convex Optimization

A Note on the Alternating Direction Method of Multipliers

On the Global and Linear Convergence of the Generalized Alternating Direction Method of Multipliers

Convergence of alternating direction method for minimizing sum of two nonconvex functions with linear constraints

The proximal alternating direction method of multipliers in the nonconvex setting: convergence analysis and rates

Alternating Direction Method of Multipliers for a Class of Nonconvex Bilinear Optimization: Convergence Analysis and Applications.

A Generalization of Linearized Alternating Direction Method of Multipliers for Solving Two-Block Separable Convex Programming

A Linearized Alternating Direction Method of Multipliers with Substitution Procedure.

Linear Convergence of the Alternating Direction Method of Multipliers for a Class of Convex Optimization Problems

Linearized Block-Wise Alternating Direction Method of Multipliers for Multiple-Block Convex Programming

An Inertial Proximal Alternating Direction Method of Multipliers for Nonconvex Optimization.