Abstract:We study distributed multiagent optimization over (directed, time-varying) graphs. We consider the minimization of $F+G$ subject to convex constraints, where $F$ is the smooth strongly convex sum of the agent's losses and $G$ is a nonsmooth convex function. We build on the SONATA algorithm: the algorithm employs the use of surrogate objective functions in the agents' subproblems (going thus beyond linearization, such as proximal-gradient) coupled with a perturbed (push-sum) consensus mechanism that aims to track locally the gradient of $F$. SONATA achieves precision $\epsilon>0$ on the objective value in $\mathcal{O}(\kappa_g \log(1/\epsilon))$ gradient computations at each node and $\tilde{\mathcal{O}}\big(\kappa_g (1-\rho)^{-1/2} \log(1/\epsilon)\big)$ communication steps, where $\kappa_g$ is the condition number of $F$ and $\rho$ characterizes the connectivity of the network. This is the first linear rate result for distributed composite optimization; it also improves on existing (non-accelerated) schemes just minimizing $F$, whose rate depends on much larger quantities than $\kappa_g$ (e.g., the worst-case condition number among the agents). When considering in particular empirical risk minimization problems with statistically similar data across the agents, SONATA employing high-order surrogates achieves precision $\epsilon>0$ in $\mathcal{O}\big((\beta/\mu) \log(1/\epsilon)\big)$ iterations and $\tilde{\mathcal{O}}\big((\beta/\mu) (1-\rho)^{-1/2} \log(1/\epsilon)\big)$ communication steps, where $\beta$ measures the degree of similarity of the agents' losses and $\mu$ is the strong convexity constant of $F$. Therefore, when $\beta/\mu < \kappa_g$, the use of high-order surrogates yields provably faster rates than what achievable by first-order models; this is without exchanging any Hessian matrix over the network.

A Gradient Tracking Protocol for Optimization Over Nabla Fractional Multi-Agent Systems

Distributed Adaptive Optimization Algorithm for Fractional High-Order Multiagent Systems Based on Event-Triggered Strategy and Input Quantization

Augmented Distributed Gradient Methods for Multi-Agent Optimization under Uncoordinated Constant Stepsizes

Distributed Multi-Step Subgradient Optimization for Multi-Agent System

Distributed Consensus Tracking of Incommensurate Heterogeneous Fractional-Order Multi-Agent Systems Based on Vector Lyapunov Function Method

Leader‐following Consensus of Fractional‐order Multi‐agent Systems Via Adaptive Control

Consensus Tracking for Fractional-Order Multi-Agent Systems by Distributed Event-Triggered Control

A Flexible Gradient Tracking Algorithmic Framework for Decentralized Optimization

Distributed Optimization for Consensus Performance of Delayed Fractional-order Double-integrator Multi-agent Systems

The Novel Adaptive Fractional Order Gradient Decent Algorithms Design via Robust Control

Distributed Optimization Based on Gradient-tracking Revisited: Enhancing Convergence Rate via Surrogation

A Discrete-Time Fractional-Order Flocking Control Algorithm of Multi-Agent Systems

Optimal Gradient Tracking for Decentralized Optimization

On the Divergence of Decentralized Non-Convex Optimization

Adaptive Neural Consensus for Fractional-Order Multi-Agent Systems With Faults and Delays

Nabla fractional distributed optimization algorithms over undirected/directed graphs

An Accelerated Algorithm for Linear Quadratic Optimal Consensus of Heterogeneous Multiagent Systems.

Distributed Quantized Feedback Design Strategy for Adaptive Consensus Tracking of Uncertain Strict-Feedback Nonlinear Multiagent Systems With State Quantizers

Distributed proximal‐gradient algorithms for nonsmooth convex optimization of second‐order multiagent systems

On The Unified Design Of Accelerated Gradient Descent

Accelerated Alternating Direction Method of Multipliers Gradient Tracking for Distributed Optimization