Abstract:In this article, we study decentralized convex constrained optimization problems in networks. We focus on the dual averaging-based algorithmic framework that is well-documented to be superior in handling constraints and complex communication environments simultaneously. Two new decentralized dual averaging (DDA) algorithms are proposed. In the first one, a second-order dynamic average consensus protocol is tailored for DDA-type algorithms, which equips each agent with a provably more accurate estimate of the global dual variable than conventional schemes. We rigorously prove that the proposed algorithm attains $\mathcal {O}(1/t)$ convergence for general convex and smooth problems, for which existing DDA methods were only known to converge at $\mathcal {O}(1/\sqrt{t})$ prior to our work. In the second one, we use the extrapolation technique to accelerate the convergence of DDA. Compared to existing accelerated algorithms, where typically two different variables are exchanged among agents at each time, the proposed algorithm only seeks consensus on local gradients. Then, the extrapolation is performed based on two sequences of primal variables, which are determined by the accumulations of gradients at two consecutive time instants, respectively. The algorithm is proved to converge at $\mathcal {O}(1)\left(\frac{1}{t^{2}}+\frac{1}{t(1-\beta)^{2}}\right)$ , where $\beta$ denotes the second largest singular value of the mixing matrix. We remark that the condition for the algorithmic parameter to guarantee convergence does not rely on the spectrum of the mixing matrix, making itself easy to satisfy in practice. Finally, numerical results are presented to demonstrate the efficiency of the proposed methods.

Dynamical Primal-Dual Nesterov Accelerated Method and Its Application to Network Optimization

Accelerated Primal-Dual Algorithms for Distributed Smooth Convex Optimization over Networks

Accelerated Primal-Dual Projection Neurodynamic Approach With Time Scaling for Linear and Set Constrained Convex Optimization Problems

Primal-Dual Algorithm for Distributed Optimization with Coupled Constraints

Accelerated Distributed Nesterov Optimization Subject to Complex Constraints and Its Applications

A New Primal-Dual Algorithm for a Class of Nonlinear Compositional Convex Optimization Problems

The design of continuous-time accelerated algorithm for nonsmooth distributed optimization without sharing local decision variables

Distributed Stochastic Consensus Optimization With Momentum for Nonconvex Nonsmooth Problems

A fast primal-dual algorithm via dynamical system with variable mass for linearly constrained convex optimization

A Universal Accelerated Primal–Dual Method for Convex Optimization Problems

Accelerated Dual Averaging Methods for Decentralized Constrained Optimization

A Unified Distributed Method for Constrained Networked Optimization via Saddle-Point Dynamics

Accelerated Alternating Direction Method of Multipliers Gradient Tracking for Distributed Optimization

Optimal Primal-Dual Methods for a Class of Saddle Point Problems

Accelerated $AB$/Push-Pull Methods for Distributed Optimization over Time-Varying Directed Networks

Linear Convergence of Primal-Dual Gradient Methods and their Performance in Distributed Optimization

Distributed Online Optimization for Multi-Agent Networks With Coupled Inequality Constraints

Asymptotic Properties of Primal-Dual Algorithm for Distributed Stochastic Optimization Over Random Networks

On the Differentiability of the Primal-Dual Interior-Point Method

A Decentralized Primal-Dual Method with Quasi-Newton Tracking

A primal-dual interior-point relaxation method with global and rapidly local convergence for nonlinear programs