Abstract:We consider a distributed personalized optimization problem over a network of nodes whose cost functions depend on a decision variable consisting of two parts: global (shared) part and local (node-specific) part. This problem structure arises in several important scenarios where the global part captures the common feature among nodes and the local part represents the personalized feature. To solve this problem, we develop a flexible algorithmic framework employing personalized distributed stochastic gradient tracking method, where each node updates variables locally and communicates the shared part with its neighbors for coordination in a stochastic way. Different from existing works, the proposed framework provides great flexibility in designing algorithms with uncoordinated stepsizes for updating local and global parts over changing topologies. Most importantly, it can eliminate the effect of heterogeneity among nodes thanks to the gradient tracking scheme. Leveraging a properly designed Lyapunov function, we prove the convergence of the proposed algorithm for both (strongly) convex and non-convex objective functions with cross-block Lipschitz gradients. The obtained rate results show clear dependence of the convergence performance on topology, heterogeneity of stepsize and properties of objective functions. Numerical examples are provided to demonstrate the effectiveness of the proposed algorithm accounting for node-specific heterogeneity.

Distributed Heterogeneous Multi-Agent Optimization with Stochastic Sub-Gradient

Distributed Optimization For Resource Allocation With Event-Triggered Communication Under Directed Topology

Distributed Stochastic Algorithm for Global Optimization in Networked System

Distributed Stochastic Optimization With Unbounded Subgradients Over Randomly Time-Varying Networks

Distributed Multi-Step Subgradient Optimization for Multi-Agent System

Augmented Distributed Gradient Methods for Multi-Agent Optimization under Uncoordinated Constant Stepsizes

Distributed Nonsmooth Convex Optimization over Markovian Switching Random Networks with Two Step-Sizes

Decentralized stochastic subgradient projection optimization algorithms over random networks

Distributed Stochastic Subgradient Projection Algorithms for Convex Optimization

A Push-Pull Gradient Method for Distributed Optimization in Networks.

Heterogeneous Distributed Subgradient

A Unified Contraction Analysis of a Class of Distributed Algorithms for Composite Optimization

Convergence in High Probability of Distributed Stochastic Gradient Descent Algorithms

Distributed Randomized Gradient-Free Optimization Protocol of Multiagent Systems Over Weight-Unbalanced Digraphs

Non-Smooth Setting of Stochastic Decentralized Convex Optimization Problem Over Time-Varying Graphs

Distributed Optimization With Personalization: A Flexible Algorithmic Framework

Distributed Adaptive Gradient Algorithm with Gradient Tracking for Stochastic Non-Convex Optimization

Distributed optimization algorithm for multi‐agent networks with lazy gradient information

A Communication-Efficient Stochastic Gradient Descent Algorithm for Distributed Nonconvex Optimization

Stochastic Gradient Tracking Methods for Distributed Personalized Optimization over Networks