Abstract:We study distributed composite optimization over networks: agents minimize a sum of smooth (strongly) convex functions, the agents' sum-utility, plus a nonsmooth (extended-valued) convex one. We propose a general unified algorithmic framework for such a class of problems and provide a unified convergence analysis leveraging the theory of operator splitting. Distinguishing features of our scheme are: (i) When the agents' functions are strongly convex, the algorithm converges at a linear rate, whose dependence on the agents' functions and network topology is decoupled, matching the typical rates of centralized optimization; the rate expression improves on existing results; (ii) When the objective function is convex (but not strongly convex), similar separation as in (i) is established for the coefficient of the proved sublinear rate; (iii) The algorithm can adjust the ratio between the number of communications and computations to achieve a rate (in terms of computations) independent on the network connectivity; and (iv) A by-product of our analysis is a tuning recommendation for several existing (non accelerated) distributed algorithms yielding the fastest provably (worst-case) convergence rate. This is the first time that a general distributed algorithmic framework applicable to composite optimization enjoys all such properties.

Feedback-Feedforward Control Approach To Distributed Optimization

A Gradient-based Kernel Optimization Approach for Parabolic Distributed Parameter Control Systems

Exponential convergence of distributed optimization for heterogeneous linear multi-agent systems

Distributed Algorithms for Composite Optimization: Unified Framework and Convergence Analysis

Distributed Optimization Algorithm with Superlinear Convergence Rate

A Unified Contraction Analysis of a Class of Distributed Algorithms for Composite Optimization

Inverse optimal stabilization of cooperative control in networked multi-agent systems

Optimal distributed cooperative control for multi-agent systems with constrains on convergence speed and control input

A Push-Pull Gradient Method for Distributed Optimization in Networks.

Exponentially Convergent Algorithm Design for Constrained Distributed Optimization via Non-smooth Approach

A Unified Algorithmic Framework for Distributed Composite Optimization.

Distributed Multiproximal Algorithm for Nonsmooth Convex Optimization With Coupled Inequality Constraints

On Linear Convergence of PI Consensus Algorithm under the Restricted Secant Inequality

Fast and robust consensus-based optimization via optimal feedback control

Linear Convergence of Pre-Conditioned PI Consensus Algorithm under Restricted Strong Convexity

A feedback control approach to convex optimization with inequality constraints

Distributed Optimization for Continuous Time Multiple Agents Systems with Unknown Parameters

On Decomposition and Convergence of Distributed Optimization Algorithms

Distributed Optimization with Projection-free Dynamics

Small-Gain Theorem Based Distributed Prescribed-Time Convex Optimization For Networked Euler-Lagrange Systems

Distributed proximal‐gradient algorithms for nonsmooth convex optimization of second‐order multiagent systems