Abstract:This paper considers solving a class of optimization problems over a network of agents, in which the cost function is expressed as the sum of individual objectives of the agents. The underlying communication graph is assumed to be undirected and connected. A distributed algorithm in which agents employ time-varying and heterogeneous step-sizes is proposed by combining consensus of multi-agent systems with gradient tracking technique. The algorithm not only drives the agents' iterates to a global and consensual minimizer but also finds the optimal value of the cost function. When the individual objectives are convex and smooth, we prove that the algorithm converges at a rate of O(1/root t) if the homogeneous step-size does not exceed some upper bound, and it accelerates to O(1/t) if the homogeneous step-size is sufficiently small. When at least one of the individual objectives is strongly convex and all are smooth, we prove that the algorithm converges at a linear rate of O(lambda(t)) with 0 < lambda < 1 even though the step-sizes are time-varying and heterogeneous. Two numerical examples are provided to demonstrate the efficiency of the proposed algorithm and to validate the theoretical findings. (C) 2019 Published by Elsevier Ltd on behalf of The Franklin Institute.

Gossip-based Gradient-Free Method for Multi-Agent Optimization: Constant Step Size Analysis

Asynchronous Gossip-Based Gradient-Free Method for Multiagent Optimization

An Accelerated Gossip-Based Distributed Gradient Method

Augmented Distributed Gradient Methods for Multi-Agent Optimization under Uncoordinated Constant Stepsizes

Distributed Multi-Step Subgradient Optimization for Multi-Agent System

Gossip-based distributed stochastic mirror descent for constrained optimization

Distributed Subgradient-Based Multiagent Optimization with More General Step Sizes.

Random Gradient-Free Optimization for Multiagent Systems with Communication Noises under a Time-Varying Weight Balanced Digraph

Randomized Gradient-Free Method for Multiagent Optimization Over Time-Varying Networks.

Distributed Subgradient Algorithm for Multi-Agent Optimization with Dynamic Stepsize

Accelerated Primal-Dual Algorithms for Distributed Smooth Convex Optimization over Networks

A Push-Pull Gradient Method for Distributed Optimization in Networks.

Distributed Randomized Gradient-Free Optimization Protocol of Multiagent Systems Over Weight-Unbalanced Digraphs

Convergence of Distributed Gradient-Tracking-Based Optimization Algorithms with Random Graphs

Distributed Quantized Gradient-Free Algorithm for Multi-Agent Convex Optimization

Distributed Consensus-Based Multi-Agent Convex Optimization Via Gradient Tracking Technique

Gradient‐free Method for Distributed Multi‐agent Optimization Via Push‐sum Algorithms

Distributed Online Optimization with Gradient-free Design

Gossip-Based Asynchronous Algorithms for Distributed Composite Optimization

Gossip Gradient Descent.

Distributed Aggregative Optimization over Multi-Agent Networks