Abstract:By coordinating terminal smart devices or microprocessors to engage in cooperative computation to achieve systemlevel targets, distributed optimization is incrementally favored by both engineering and computer science. The well-known alternating direction method of multipliers (ADMM) has turned out to be one of the most popular tools for distributed optimization due to many advantages, such as modular structure, superior convergence, easy implementation and high flexibility. In the past decade, ADMM has experienced widespread developments. The developments manifest in both handling more general problems and enabling more effective implementation. Specifically, the method has been generalized to broad classes of problems (i.e.,multi-block, coupled objective, nonconvex, etc.). Besides, it has been extensively reinforced for more effective implementation, such as improved convergence rate, easier subproblems, higher computation efficiency, flexible communication, compatible with inaccurate information, robust to communication delays, etc. These developments lead to a plentiful of ADMM variants to be celebrated by broad areas ranging from smart grids, smart buildings, wireless communications, machine learning and beyond. However, there lacks a survey to document those developments and discern the results. To achieve such a goal, this paper provides a comprehensive survey on ADMM variants. Particularly, we discern the five major classes of problems that have been mostly concerned and discuss the related ADMM variants in terms of main ideas, main assumptions, convergence behaviors and main features. In addition, we figure out several important future research directions to be addressed. This survey is expected to work as a tutorial for both developing distributed optimization in broad areas and identifying existing theoretical research gaps.

Adaptive Uncertainty-Weighted ADMM for Distributed Optimization

Accelerated Stochastic ADMM with Variance Reduction

Distributed ADMM for Time-Varying Communication Networks

BFGS-ADMM for Large-Scale Distributed Optimization

Toward Quality of Information Aware Distributed Machine Learning

DQC-ADMM: Decentralized Dynamic ADMM with Quantized and Censored Communications

Distributed and Parallel ADMM for Structured Nonconvex Optimization Problem.

On Distributed Nonconvex Optimisation Via Modified ADMM

A Block-wise, Asynchronous and Distributed ADMM Algorithm for General Form Consensus Optimization.

Graph-aware Weighted Hybrid ADMM for Fast Decentralized Optimization

ADMM-Tracking Gradient for Distributed Optimization over Asynchronous and Unreliable Networks

Augmented Distributed Gradient Methods for Multi-Agent Optimization under Uncoordinated Constant Stepsizes

A Unified Inexact Stochastic ADMM for Composite Nonconvex and Nonsmooth Optimization

Proximal ADMM for Nonconvex and Nonsmooth Optimization

A Survey of ADMM Variants for Distributed Optimization: Problems, Algorithms and Features

Improved Hierarchical ADMM for Nonconvex Cooperative Distributed Model Predictive Control

A General Distributed Dual Coordinate Optimization Framework for Regularized Loss Minimization

Adaptive Relaxed ADMM: Convergence Theory and Practical Implementation

Coded Stochastic ADMM for Decentralized Consensus Optimization With Edge Computing

A Stochastic Alternating Direction Method of Multipliers for Non-smooth and Non-convex Optimization

DN-ADMM: Distributed Newton ADMM for Multi-agent Optimization