Abstract:In this paper, we establish the convergence of the proximal alternating direction method of multipliers (ADMM) and block coordinate descent (BCD) method for nonseparable minimization models with quadratic coupling terms. The novel convergence results presented in this paper answer several open questions that have been the subject of considerable discussion. We firstly extend the 2-block proximal ADMM to linearly constrained convex optimization with a coupled quadratic objective function, an area where theoretical understanding is currently lacking, and prove that the sequence generated by the proximal ADMM converges in point-wise manner to a primal-dual solution pair. Moreover, we apply randomly permuted ADMM (RPADMM) to nonseparable multi-block convex optimization, and prove its expected convergence for a class of nonseparable quadratic programming problems. When the linear constraint vanishes, the 2-block proximal ADMM and RPADMM reduce to the 2-block cyclic proximal BCD method and randomly permuted BCD (RPBCD). Our study provides the first iterate convergence result for 2-block cyclic proximal BCD without assuming the boundedness of the iterates. We also theoretically establish the expected iterate convergence result concerning multi-block RPBCD for convex quadratic optimization. In addition, we demonstrate that RPBCD may have a worse convergence rate than cyclic proximal BCD for 2-block convex quadratic minimization problems. Although the results on RPADMM and RPBCD are restricted to quadratic minimization models, they provide some interesting insights: (1) random permutation makes ADMM and BCD more robust for multi-block convex minimization problems; (2) cyclic BCD may outperform RPBCD for “nice” problems, and RPBCD should be applied with caution when solving general convex optimization problems especially with a few blocks.

A Proximal Alternating Direction Method of Multipliers for DC Programming with Structured Constraints

Proximal Alternating Direction Method of Multipliers with Convex Combination Proximal Centers

Parallel Direction Method of Multipliers

An Alternating Direction Method of Multipliers with the BFGS Update for Structured Convex Quadratic Optimization

Fast Proximal Linearized Alternating Direction Method of Multiplier with Parallel Splitting

A Bregman-style Partially Symmetric Alternating Direction Method of Multipliers for Nonconvex Multi-block Optimization

Distributed Proximal Alternating Direction Method of Multipliers for Constrained Composite Optimization Over Directed Networks

A Golden Ratio Proximal Alternating Direction Method of Multipliers for Separable Convex Optimization

Efficient DC Algorithm for Constrained Sparse Optimization

Accelerating preconditioned ADMM via degenerate proximal point mappings

Global Complexity Bound of a Proximal ADMM for Linearly Constrained Nonseparable Nonconvex Composite Programming

An adaptive linearized alternating direction multiplier method with a relaxation step for convex programming

The symmetric ADMM with indefinite proximal regularization and its application

On the Convergence Analysis of the Alternating Direction Method of Multipliers with Three Blocks

Auto-conditioned primal-dual hybrid gradient method and alternating direction method of multipliers

Extended ADMM and BCD for Nonseparable Convex Minimization Models with Quadratic Coupling Terms: Convergence Analysis and Insights

Linear Convergence of the Alternating Direction Method of Multipliers for a Class of Convex Optimization Problems

Fully Distributed Alternating Direction Method of Multipliers in Digraphs via Finite-Time Termination Mechanisms

Alternating Direction Method of Multipliers with Variable Metric Indefinite Proximal Terms for Convex Optimization

A Boosted-DCA with Power-Sum-DC Decomposition for Linearly Constrained Polynomial Programs

A Convergent $3$-Block Semi-Proximal ADMM for Convex Minimization Problems with One Strongly Convex Block