Abstract:Distributed optimization is a fundamental framework for collaborative inference and decision making in decentralized multi-agent systems. The operation is modeled as the joint minimization of a shared objective which typically depends on observations gathered locally by each agent. Distributed optimization algorithms, such as the common D-ADMM, tackle this task by iteratively combining local computations and message exchanges. One of the main challenges associated with distributed optimization, and particularly with D-ADMM, is that it requires a large number of communications, i.e., messages exchanged between the agents, to reach consensus. This can make D-ADMM costly in power, latency, and channel resources. In this work we propose unfolded D-ADMM, which follows the emerging deep unfolding methodology to enable D-ADMM to operate reliably with a predefined and small number of messages exchanged by each agent. Unfolded D-ADMM fully preserves the operation of D-ADMM, while leveraging data to tune the hyperparameters of each iteration of the algorithm. These hyperparameters can either be agent-specific, aiming at achieving the best performance within a fixed number of iterations over a given network, or shared among the agents, allowing to learn to distributedly optimize over different networks. For both settings, our unfolded D-ADMM operates with limited communications, while preserving the interpretability and flexibility of the original D-ADMM algorithm. We specialize unfolded D-ADMM for two representative settings: a distributed estimation task, considering a sparse recovery setup, and a distributed learning scenario, where multiple agents collaborate in learning a machine learning model. Our numerical results demonstrate that the proposed approach dramatically reduces the number of communications utilized by D-ADMM, without compromising on its performance.

Distributed ADMM for Time-Varying Communication Networks

Distributed ADMM with Synergetic Communication and Computation.

Accelerated Stochastic ADMM with Variance Reduction

Distributed Mimo Network Optimization Based On Duality And Local Message Passing

Asynchronous Distributed Optimization via ADMM with Efficient Communication

ADMM-Tracking Gradient for Distributed Optimization over Asynchronous and Unreliable Networks

A Survey of ADMM Variants for Distributed Optimization: Problems, Algorithms and Features

Fully Distributed Alternating Direction Method of Multipliers in Digraphs via Finite-Time Termination Mechanisms

Distributed ADMM with linear updates over directed networks

Limited Communications Distributed Optimization via Deep Unfolded Distributed ADMM

Proximal ADMM for Nonconvex and Nonsmooth Optimization

Accelerated Distributed Stochastic Non-Convex Optimization over Time-Varying Directed Networks

On Distributed Nonconvex Optimisation Via Modified ADMM

Distributed Proximal Alternating Direction Method of Multipliers for Constrained Composite Optimization Over Directed Networks

Improved Hierarchical ADMM for Nonconvex Cooperative Distributed Model Predictive Control

ADMM-based Distributed Algorithm for Emergency Backup in Time-Variant Inter-Dc Networks

Distributed Event-Based Learning via ADMM

Decentralized ADMM with compressed and event-triggered communication

Sublinear and Linear Convergence of Modified ADMM for Distributed Nonconvex Optimization

Distributed Stochastic ADMM for Matrix Factorization.

Distributed Algorithm for a Finite Time Horizon Resource Allocation over a Directed Network