Abstract:We consider a distributed personalized optimization problem over a network of nodes whose cost functions depend on a decision variable consisting of two parts: global (shared) part and local (node-specific) part. This problem structure arises in several important scenarios where the global part captures the common feature among nodes and the local part represents the personalized feature. To solve this problem, we develop a flexible algorithmic framework employing personalized distributed stochastic gradient tracking method, where each node updates variables locally and communicates the shared part with its neighbors for coordination in a stochastic way. Different from existing works, the proposed framework provides great flexibility in designing algorithms with uncoordinated stepsizes for updating local and global parts over changing topologies. Most importantly, it can eliminate the effect of heterogeneity among nodes thanks to the gradient tracking scheme. Leveraging a properly designed Lyapunov function, we prove the convergence of the proposed algorithm for both (strongly) convex and non-convex objective functions with cross-block Lipschitz gradients. The obtained rate results show clear dependence of the convergence performance on topology, heterogeneity of stepsize and properties of objective functions. Numerical examples are provided to demonstrate the effectiveness of the proposed algorithm accounting for node-specific heterogeneity.

Distributed finite-time optimization algorithms with a modified Newton–Raphson method

An Initialization-Free Continuous-Time Distributed Algorithm for Convex Optimization Problem

A distributed Newton–Raphson-based coordination algorithm for multi-agent optimization with discrete-time communication

Distributed Stochastic Algorithm for Global Optimization in Networked System

Distributed adaptive Newton methods with global superlinear convergence

Distributed Adaptive Newton Methods with Globally Superlinear Convergence

Distributed Optimization Algorithm with Superlinear Convergence Rate

Distributed Algorithms for Composite Optimization: Unified Framework and Convergence Analysis

Distributed Robust Optimization in Networked System.

Distributed Multi-Step Subgradient Optimization for Multi-Agent System

Distributed Continuous-Time Algorithm for Constrained Optimization of Networked Euler–Lagrange Systems

Distributed Optimization for Continuous Time Multiple Agents Systems with Unknown Parameters

Accelerating Distributed Optimization via Fixed-time Convergent Flows: Extensions to Non-convex Functions and Consistent Discretization

Distributed Asynchronous Discrete-Time Feedback Optimization

Distributed Optimization With Personalization: A Flexible Algorithmic Framework

Robust fixed-time distributed optimization with predefined convergence-time bound

Distributed Approximate Newton Algorithms and Weight Design for Constrained Optimization

Distributed initialization-free algorithms for multi-agent optimization problems with coupled inequality constraints

Distributed Inexact Newton Method with Adaptive Step Sizes

Achieving Globally Superlinear Convergence for Distributed Optimization with Adaptive Newton Method

Distributed Finite-Time Optimization for Second Order Continuous-Time Multiple Agents Systems with Time-Varying Cost Function.