Fully asynchronous distributed optimization with linear convergence over directed networks

SHA Xingyu,ZHANG Jiaqi,YOU Keyou

2023-01-01

Abstract:We study distributed optimization problems over a directed network,where nodes aim to minimize the sum of local objective functions via directed communications with neighbors.Many algo-rithms are designed to solve it for synchronized or randomly activated implementation,which may create deadlocks in practice.In sharp contrast,we propose a fully asynchronous push-pull gradient(APPG)algorithm,where each node updates without waiting for any other node by using possibly de-layed information from neighbors.Then,we construct two novel augmented networks to analyze asyn-chrony and delays,and quantify its convergence rate from the worst-case point of view.Particularly,all nodes of APPG converge to the same optimal solution at a linear rate of O(λk)if local functions have Lipschitz-continuous gradients and their sum satisfies the Polyak-?ojasiewicz condition(convexity is not required),where λ ∈(0,1)is explicitly given and the virtual counter k increases by one when any node updates.Finally,the advantage of APPG over the synchronous counterpart and its linear speedup efficiency are numerically validated via a logistic regression problem.