Abstract:In this article, we concentrate on dealing with the distributed optimization problem over a directed network, where each unit possesses its own convex cost function and the principal target is to minimize a global cost function (formulated by the average of all local cost functions) while obeying the network connectivity structure. Most of the existing methods, such as push-sum strategy, have eliminated the unbalancedness induced by the directed network via utilizing column-stochastic weights, which may be infeasible if the distributed implementation requires each unit to gain access to (at least) its out-degree information. In contrast, to be suitable for the directed networks with row-stochastic weights, we propose a new directed distributed Nesterov-like gradient tracking algorithm, named as D-DNGT, that incorporates the gradient tracking into the distributed Nesterov method with momentum terms and employs nonuniform step-sizes. D-DNGT extends a number of outstanding consensus algorithms over strongly connected directed networks. The implementation of D-DNGT is straightforward if each unit locally chooses a suitable step-size and privately regulates the weights on information that acquires from in-neighbors. If the largest step-size and the maximum momentum coefficient are positive and small sufficiently, we can prove that D-DNGT converges linearly to the optimal solution provided that the cost functions are smooth and strongly convex. We provide numerical experiments to confirm the findings in this article and contrast D-DNGT with recently proposed distributed optimization approaches.

Distributed Adaptive Gradient Algorithm with Gradient Tracking for Stochastic Non-Convex Optimization

Distributed Adaptive Gradient Algorithm with Gradient Tracking for Stochastic Non-Convex Optimization

Stochastic Gradient Tracking Methods for Distributed Personalized Optimization over Networks

A Distributed Stochastic Optimization Algorithm with Gradient-Tracking and Distributed Heavy-Ball Acceleration

Distributed Stochastic Optimization with Gradient Tracking over Time-Varying Directed Networks

Distributed Stochastic Gradient Tracking Algorithm with Variance Reduction for Non-Convex Optimization

Edge-Based Stochastic Gradient Algorithm for Distributed Optimization

Distributed Stochastic Algorithm for Global Optimization in Networked System

Stochastic Strongly Convex Optimization Via Distributed Epoch Stochastic Gradient Algorithm

Distributed Proximal Gradient Algorithm for Nonconvex Optimization Over Time-Varying Networks

Augmented Distributed Gradient Methods for Multi-Agent Optimization under Uncoordinated Constant Stepsizes

Convergence in High Probability of Distributed Stochastic Gradient Descent Algorithms

A Nesterov-Like Gradient Tracking Algorithm for Distributed Optimization over Directed Networks

Distributed Gradient Tracking for Unbalanced Optimization with Different Constraint Sets

An Improved Distributed Nesterov Gradient Tracking Algorithm for Smooth Convex Optimization over Directed Networks

A Decentralized Nesterov Gradient Method for Stochastic Optimization over Unbalanced Directed Networks

A Stochastic Averaging Gradient Algorithm with Multi‐step Communication for Distributed Optimization

Distributed Stochastic Optimization With Unbounded Subgradients Over Randomly Time-Varying Networks

Distributed Subgradient-Free Stochastic Optimization Algorithm for Nonsmooth Convex Functions over Time-Varying Networks

Convergence of Distributed Gradient-Tracking-Based Optimization Algorithms with Random Graphs

Distributed Optimization With Personalization: A Flexible Algorithmic Framework