Abstract:We consider a generic decentralized constrained optimization problem over static, directed communication networks, where each agent has exclusive access to only one convex, differentiable, local objective term and one convex constraint set. For this setup, we propose a novel decentralized algorithm, called DAGP (Double Averaging and Gradient Projection), based on local gradients, projection onto local constraints, and local averaging. We achieve global optimality through a novel distributed tracking technique we call distributed null projection. Further, we show that DAGP can be used to solve unconstrained problems with non-differentiable objective terms with a problem reduction scheme. Assuming only smoothness of the objective terms, we study the convergence of DAGP and establish sub-linear rates of convergence in terms of feasibility, consensus, and optimality, with no extra assumption (e.g. strong convexity). For the analysis, we forego the difficulties of selecting Lyapunov functions by proposing a new methodology of convergence analysis in optimization problems, which we refer to as aggregate lower-bounding. To demonstrate the generality of this method, we also provide an alternative convergence proof for the standard gradient descent algorithm with smooth functions. Finally, we present numerical results demonstrating the effectiveness of our proposed method in both constrained and unconstrained problems. In particular, we propose a distributed scheme by DAGP for the optimal transport problem with superior performance and speed.

Accelerated Projection Algorithm Based on Smoothing Approximation for Distributed Non-Smooth Optimization

Accelerated Primal-Dual Algorithms for Distributed Smooth Convex Optimization over Networks

Distributed Algorithms for Composite Optimization: Unified Framework and Convergence Analysis

The design of continuous-time accelerated algorithm for nonsmooth distributed optimization without sharing local decision variables

Double Averaging and Gradient Projection: Convergence Guarantees for Decentralized Constrained Optimization

Accelerated Distributed Aggregative Optimization

Network Optimization via Smooth Exact Penalty Functions Enabled by Distributed Gradient Computation

A Unified Algorithmic Framework for Distributed Composite Optimization.

Surplus-based accelerated algorithms for distributed optimization over directed networks

Distributed stochastic projection-free solver for constrained optimization

Distributed Accelerated Proximal Coordinate Gradient Methods

Distributed Proximal Alternating Direction Method of Multipliers for Constrained Composite Optimization Over Directed Networks

A Unified Contraction Analysis of a Class of Distributed Algorithms for Composite Optimization

A Mixing-Accelerated Primal-Dual Proximal Algorithm for Distributed Nonconvex Optimization

Asynchronous Decentralized Accelerated Stochastic Gradient Descent

Provably Accelerated Decentralized Gradient Method Over Unbalanced Directed Graphs

Distributed quasi-monotone subgradient algorithm for nonsmooth convex optimization over directed graphs

Exponentially Convergent Algorithm Design for Constrained Distributed Optimization via Non-smooth Approach

A Communication-Efficient Stochastic Gradient Descent Algorithm for Distributed Nonconvex Optimization

Stochastic smoothing accelerated gradient method for nonsmooth convex composite optimization

Smoothing accelerated algorithm for constrained nonsmooth convex optimization problems