Abstract:We consider a generic decentralized constrained optimization problem over static, directed communication networks, where each agent has exclusive access to only one convex, differentiable, local objective term and one convex constraint set. For this setup, we propose a novel decentralized algorithm, called DAGP (Double Averaging and Gradient Projection), based on local gradients, projection onto local constraints, and local averaging. We achieve global optimality through a novel distributed tracking technique we call distributed null projection. Further, we show that DAGP can be used to solve unconstrained problems with non-differentiable objective terms with a problem reduction scheme. Assuming only smoothness of the objective terms, we study the convergence of DAGP and establish sub-linear rates of convergence in terms of feasibility, consensus, and optimality, with no extra assumption (e.g. strong convexity). For the analysis, we forego the difficulties of selecting Lyapunov functions by proposing a new methodology of convergence analysis in optimization problems, which we refer to as aggregate lower-bounding. To demonstrate the generality of this method, we also provide an alternative convergence proof for the standard gradient descent algorithm with smooth functions. Finally, we present numerical results demonstrating the effectiveness of our proposed method in both constrained and unconstrained problems. In particular, we propose a distributed scheme by DAGP for the optimal transport problem with superior performance and speed.

Accelerated Projection Algorithm Based on Smoothing Approximation for Distributed Nonsmooth Optimization

Accelerated Projection Algorithm Based on Smoothing Approximation for Distributed Non-Smooth Optimization

Distributed Smoothing Projection Neurodynamic Approaches for Constrained Nonsmooth Optimization

An Accelerated Distributed Method with Inexact Model of Relative Smoothness and Strong Convexity

Accelerated Primal-Dual Algorithms for Distributed Smooth Convex Optimization over Networks

An Accelerated Gradient Tracking Algorithm with Projection Error for Distributed Optimization.

An Accelerated Algorithm for Distributed Optimization with Barzilai-Borwein Step Sizes

The design of continuous-time accelerated algorithm for nonsmooth distributed optimization without sharing local decision variables

A Distributed Stochastic Proximal-Gradient Algorithm for Composite Optimization

Network Optimization via Smooth Exact Penalty Functions Enabled by Distributed Gradient Computation

A Distributed Proximal Alternating Direction Multiplier Method for Multiblock Nonsmooth Composite Optimization

Accelerated Distributed Aggregative Optimization

Double Averaging and Gradient Projection: Convergence Guarantees for Decentralized Constrained Optimization

Distributed Proximal Alternating Direction Method of Multipliers for Constrained Composite Optimization Over Directed Networks

Surplus-based accelerated algorithms for distributed optimization over directed networks

Distributed projection-free algorithm for constrained aggregative optimization

A Proximal Algorithm for Distributed Optimization with Nonsmooth Inequality Constraints

Distributed Stochastic Projection-Free Algorithm for Constrained Optimization

Distributed quasi-monotone subgradient algorithm for nonsmooth convex optimization over directed graphs

Distributed Nonsmooth Optimization with Consensus and Inequality Constraints Via Distributed Smooth Proximal-Splitting Algorithm

Distributed stochastic projection-free solver for constrained optimization