Abstract:This paper investigates distributed convex optimization problems over an undirected and connected network, where each node’s variable lies in a private constrained convex set, and overall nodes aim at collectively minimizing the sum of all local objective functions. Motivated by a variety of applications in machine learning problems with large-scale training sets distributed to multiple autonomous nodes, each local objective function is further designed as the average of moderate number of local instantaneous functions. Each local objective function and constrained set cannot be shared with others. A primal-dual stochastic algorithm is presented to address the distributed convex optimization problems, where each node updates its state by resorting to unbiased stochastic averaging gradients and projects on its private constrained set. At each iteration, for each node the gradient of one local instantaneous function selected randomly is evaluated and the average of the most recent stochastic gradients is used to approximate the true local gradient. In the constrained case, we show that with strong-convexity of the local instantaneous function and Lipschitz continuity of its gradient, the algorithm converges to the global optimization solution almost surely. In the unconstrained case, an explicit linear convergence rate of the algorithm is provided. Numerical experiments are presented to demonstrate correctness of the theoretical results.

Primal-dual Algorithm for Distributed Optimization with Local Domains on Signed Networks

Primal-Dual Algorithm for Distributed Optimization with Coupled Constraints

Primal-Dual Algorithm for Distributed Constrained Optimization

Primal-dual Stochastic Distributed Algorithm for Constrained Convex Optimization

Distributed Robust Optimization in Networked System.

Accelerated Primal-Dual Algorithms for Distributed Smooth Convex Optimization over Networks

Distributed Primal-Dual Method for Convex Optimization With Coupled Constraints

Primal-Dual Subgradient Algorithm for Distributed Constraint Optimization over Unbalanced Digraphs.

Asymptotic Properties of Primal-Dual Algorithm for Distributed Stochastic Optimization Over Random Networks

Distributed Proximal Algorithms for Multiagent Optimization With Coupled Inequality Constraints

Distributed Online Optimization for Multi-Agent Networks With Coupled Inequality Constraints

Distributed Stochastic Algorithm for Global Optimization in Networked System

Decentralized Constrained Consensus Optimization with Primal Dual Splitting Projection.

Distributed Algorithms for Solving Locally Coupled Optimization Problems on Agent Networks

A Stochastic Primal-Dual Algorithm for Composite Constrained Optimization

Push-Pull Based Distributed Primal-Dual Algorithm for Coupled Constrained Convex Optimization in Multi-Agent Networks

Asymptotic Properties of Primal-Dual Algorithm for Distributed Stochastic Optimization over Random Networks with Imperfect Communications.

Primal-Dual Fixed Point Algorithms Based on Adapted Metric for Distributed Optimization with Bounded Communication Delays

Distributed Primal-dual Optimization for Heterogeneous Multi-agent Systems

Distributed Algorithms for Composite Optimization: Unified Framework and Convergence Analysis

Distributed Global Optimization for a Class of Nonconvex Optimization with Coupled Constraints