Abstract:By coordinating terminal smart devices or microprocessors to engage in cooperative computation to achieve systemlevel targets, distributed optimization is incrementally favored by both engineering and computer science. The well-known alternating direction method of multipliers (ADMM) has turned out to be one of the most popular tools for distributed optimization due to many advantages, such as modular structure, superior convergence, easy implementation and high flexibility. In the past decade, ADMM has experienced widespread developments. The developments manifest in both handling more general problems and enabling more effective implementation. Specifically, the method has been generalized to broad classes of problems (i.e.,multi-block, coupled objective, nonconvex, etc.). Besides, it has been extensively reinforced for more effective implementation, such as improved convergence rate, easier subproblems, higher computation efficiency, flexible communication, compatible with inaccurate information, robust to communication delays, etc. These developments lead to a plentiful of ADMM variants to be celebrated by broad areas ranging from smart grids, smart buildings, wireless communications, machine learning and beyond. However, there lacks a survey to document those developments and discern the results. To achieve such a goal, this paper provides a comprehensive survey on ADMM variants. Particularly, we discern the five major classes of problems that have been mostly concerned and discuss the related ADMM variants in terms of main ideas, main assumptions, convergence behaviors and main features. In addition, we figure out several important future research directions to be addressed. This survey is expected to work as a tutorial for both developing distributed optimization in broad areas and identifying existing theoretical research gaps.

Distributed Stochastic ADMM for Matrix Factorization.

Distributed High-Dimension Matrix Operation Optimization on Spark

Accelerated Stochastic ADMM with Variance Reduction

Privacy Preserving Point-of-Interest Recommendation Using Decentralized Matrix Factorization.

Distributed ADMM for Time-Varying Communication Networks

Distributed matrix factorization based on fast optimization for implicit feedback recommendation

A Block-wise, Asynchronous and Distributed ADMM Algorithm for General Form Consensus Optimization.

Distributed and Parallel ADMM for Structured Nonconvex Optimization Problem.

Distributed Bayesian Matrix Decomposition for Big Data Mining and Clustering

Multi-linear Interactive Matrix Factorization.

Scalable Stochastic Alternating Direction Method of Multipliers.

Coded Stochastic ADMM for Decentralized Consensus Optimization With Edge Computing

Large-scale and Scalable Latent Factor Analysis via Distributed Alternative Stochastic Gradient Descent for Recommender Systems

Group-Based Alternating Direction Method of Multipliers for Distributed Linear Classification

Large-Scale Distributed Bayesian Matrix Factorization using Stochastic Gradient MCMC

Rima: An RDMA-Accelerated Model-Parallelized Solution to Large-Scale Matrix Factorization

A Survey of ADMM Variants for Distributed Optimization: Problems, Algorithms and Features

Alternating Mixing Stochastic Gradient Descent for Large-scale Matrix Factorization

Fully Distributed Alternating Direction Method of Multipliers in Digraphs via Finite-Time Termination Mechanisms

Semi-discrete Matrix Factorization

Magnitude Bounded Matrix Factorisation for Recommender Systems