Abstract:The mirror descent algorithm is known to be effective in situations where it is beneficial to adapt the mirror map to the underlying geometry of the optimization model. However, the effect of mirror maps on the geometry of distributed optimization problems has not been previously addressed. In this paper we study an exact distributed mirror descent algorithm in continuous-time under additive noise. We establish a linear convergence rate of the proposed dynamics for the setting of convex optimization. Our analysis draws motivation from the Augmented Lagrangian and its relation to gradient tracking. To further explore the benefits of mirror maps in a distributed setting we present a preconditioned variant of our algorithm with an additional mirror map over the Lagrangian dual variables. This allows our method to adapt to both the geometry of the primal variables, as well as to the geometry of the consensus constraint. We also propose a Gauss-Seidel type discretization scheme for the proposed method and establish its linear convergence rate. For certain classes of problems we identify mirror maps that mitigate the effect of the graph's spectral properties on the convergence rate of the algorithm. Using numerical experiments we demonstrate the efficiency of the methodology on convex models, both with and without constraints. Our findings show that the proposed method outperforms other methods, especially in scenarios where the model's geometry is not captured by the standard Euclidean norm

Distributed Mirror Descent Algorithm over Unbalanced Digraphs Based on Gradient Weighting Technique

Stochastic Mirror Descent for Convex Optimization with Consensus Constraints

Improving Dynamic Regret in Distributed Online Mirror Descent Using Primal and Dual Information

A Mirror Descent-Based Algorithm for Corruption-Tolerant Distributed Gradient Descent

Distributed Online Optimization in Time-Varying Unbalanced Networks Without Explicit Subgradients

Convergence in High Probability of Distributed Stochastic Gradient Descent Algorithms

Privacy-Preserving Distributed Online Optimization Over Unbalanced Digraphs via Subgradient Rescaling

Data-Distributed Weighted Majority and Online Mirror Descent

A Push-Pull Gradient Method for Distributed Optimization in Networks.

Accelerated Primal-Dual Algorithms for Distributed Smooth Convex Optimization over Networks

Distributed Algorithms for Composite Optimization: Unified Framework and Convergence Analysis

Augmented Distributed Gradient Methods for Multi-Agent Optimization under Uncoordinated Constant Stepsizes

Stochastic Block Mirror Descent Methods for Nonsmooth and Stochastic Optimization

Taming Nonconvex Stochastic Mirror Descent with General Bregman Divergence

Distributed optimization for degenerate loss functions arising from over-parameterization

Distributed Multi-Step Subgradient Optimization for Multi-Agent System

Distributed Subgradient Method with Random Quantization and Flexible Weights: Convergence Analysis.

Distributed Randomized Gradient-Free Optimization Protocol of Multiagent Systems Over Weight-Unbalanced Digraphs

Distributed quasi-monotone subgradient algorithm for nonsmooth convex optimization over directed graphs

Convergence Analysis of Stochastic Saddle Point Mirror Descent Algorithm -- A Projected Dynamical View Point

Distributed Unbalanced Optimization Design over Nonidentical Constraints