Abstract:Distributed optimization techniques offer high quality solutions to various engineering problems, such as resource allocation and distributed estimation and control. In this work, we first consider distributed convex constrained optimization problems where the objective function is encoded by multiple local and possibly nonsmooth objectives privately held by a group of agents, and propose a distributed subgradient method with double averaging (abbreviated as DSA2) that only requires peer-to-peer communication and local computation to solve the global problem. The algorithmic framework builds on dual methods and dynamic average consensus; the sequence of test points is formed by iteratively minimizing a local dual model of the overall objective where the coefficients, i.e., approximated subgradients of the objective, are supplied by the dynamic average consensus scheme. We theoretically show that DSA2 enjoys non-ergodic convergence properties, i.e., the local minimizing sequence itself is convergent, a distinct feature that cannot be found in existing results. Specifically, we establish a convergence rate of O(1t) in terms of objective function error. Then, extensions are made to tackle distributed optimization problems with coupled functional constraints by combining DSA2 and dual decomposition. This is made possible by Lagrangian relaxation that transforms the coupling in constraints of the primal problem into that in cost functions of the dual, thus allowing us to solve the dual problem via DSA2. Both the dual objective error and the quadratic penalty for the coupled constraint are proved to converge at a rate of O(1t), and the primal objective error asymptotically vanishes. Numerical experiments and comparisons are conducted to illustrate the advantage of the proposed algorithms and validate our theoretical findings.

Distributed Second-Order Method with Diffusion Strategy

Distributed Optimization Algorithm with Superlinear Convergence Rate

Distributed Robust Optimization in Networked System.

Distributed Learning with Convex SUM-of -Non-convex Objective

Distributed Stochastic Algorithm for Global Optimization in Networked System

Linearly Convergent Second-Order Distributed Optimization Algorithms

Accelerated Primal-Dual Algorithms for Distributed Smooth Convex Optimization over Networks

A Push-Pull Gradient Method for Distributed Optimization in Networks.

An Accelerated Distributed Method with Inexact Model of Relative Smoothness and Strong Convexity

A Second-Order Accelerated Neurodynamic Approach for Distributed Convex Optimization

Distributed Stochastic Optimization: Variance Reduction and Edge Based Method

A Distributed Stochastic Optimization Algorithm with Gradient-Tracking and Distributed Heavy-Ball Acceleration

Distributed Optimization With Personalization: A Flexible Algorithmic Framework

A Unitary Distributed Subgradient Method for Multi-Agent Optimization with Different Coupling Sources

Distributed Optimization for Second-Order Discrete-Time Multiagent Systems with Set Constraints.

Event-triggered Broadcasting for Distributed Smooth Optimization

Distributed Difference of Convex Optimization

Distributed proximal‐gradient algorithms for nonsmooth convex optimization of second‐order multiagent systems

An Accelerated Gossip-Based Distributed Gradient Method

Distributed Adaptive Newton Methods with Globally Superlinear Convergence

A Method for Transforming Non-Convex Optimization Problem to Distributed Form