Abstract:By coordinating terminal smart devices or microprocessors to engage in cooperative computation to achieve systemlevel targets, distributed optimization is incrementally favored by both engineering and computer science. The well-known alternating direction method of multipliers (ADMM) has turned out to be one of the most popular tools for distributed optimization due to many advantages, such as modular structure, superior convergence, easy implementation and high flexibility. In the past decade, ADMM has experienced widespread developments. The developments manifest in both handling more general problems and enabling more effective implementation. Specifically, the method has been generalized to broad classes of problems (i.e.,multi-block, coupled objective, nonconvex, etc.). Besides, it has been extensively reinforced for more effective implementation, such as improved convergence rate, easier subproblems, higher computation efficiency, flexible communication, compatible with inaccurate information, robust to communication delays, etc. These developments lead to a plentiful of ADMM variants to be celebrated by broad areas ranging from smart grids, smart buildings, wireless communications, machine learning and beyond. However, there lacks a survey to document those developments and discern the results. To achieve such a goal, this paper provides a comprehensive survey on ADMM variants. Particularly, we discern the five major classes of problems that have been mostly concerned and discuss the related ADMM variants in terms of main ideas, main assumptions, convergence behaviors and main features. In addition, we figure out several important future research directions to be addressed. This survey is expected to work as a tutorial for both developing distributed optimization in broad areas and identifying existing theoretical research gaps.

Linear Convergence of ADMM under Metric Subregularity for Distributed Optimization.

Sublinear and Linear Convergence of Modified ADMM for Distributed Nonconvex Optimization

On the Linear Convergence of the ADMM in Decentralized Consensus Optimization

Linear Convergence of the Alternating Direction Method of Multipliers for a Class of Convex Optimization Problems

Stochastic linearized generalized alternating direction method of multipliers: Expected convergence rates and large deviation properties

Linearized ADMM for Nonconvex Nonsmooth Optimization with Convergence Analysis

Explicit Convergence Rate of a Distributed Alternating Direction Method of Multipliers

Distributed ADMM with linear updates over directed networks

Linearly Convergent Decentralized Consensus Optimization with the Alternating Direction Method of Multipliers

Distributed ADMM for Time-Varying Communication Networks

A Bregman-Style Improved ADMM and its Linearized Version in the Nonconvex Setting: Convergence and Rate Analyses

A Fundamental Proof of Convergence of Alternating Direction Method of Multipliers for Weakly Convex Optimization

Proximal ADMM for Nonconvex and Nonsmooth Optimization

BFGS-ADMM for Large-Scale Distributed Optimization

A Survey of ADMM Variants for Distributed Optimization: Problems, Algorithms and Features

Distributed and Parallel ADMM for Structured Nonconvex Optimization Problem.

Convergence Analysis of Generalized ADMM with Majorization for Linearly Constrained Composite Convex Optimization

Understanding the ADMM Algorithm via High-Resolution Differential Equations

On Distributed Nonconvex Optimisation Via Modified ADMM

A Block-wise, Asynchronous and Distributed ADMM Algorithm for General Form Consensus Optimization.

An Accelerated Stochastic ADMM for Nonconvex and Nonsmooth Finite-Sum Optimization