Abstract:We present a new class of decentralized first-order methods for nonsmooth and stochastic optimization problems defined over multiagent networks. Considering that communication is a major bottleneck in decentralized optimization, our main goal in this paper is to develop algorithmic frameworks which can significantly reduce the number of inter-node communications. Our major contribution is to present a new class of decentralized primal–dual type algorithms, namely the decentralized communication sliding (DCS) methods, which can skip the inter-node communications while agents solve the primal subproblems iteratively through linearizations of their local objective functions. By employing DCS, agents can find an ϵ\documentclass[12pt]{minimal}\usepackage{amsmath}\usepackage{wasysym}\usepackage{amsfonts}\usepackage{amssymb}\usepackage{amsbsy}\usepackage{mathrsfs}\usepackage{upgreek}\setlength{\oddsidemargin}{-69pt}\begin{document}$$\epsilon $$\end{document}-solution both in terms of functional optimality gap and feasibility residual in O(1/ϵ)\documentclass[12pt]{minimal}\usepackage{amsmath}\usepackage{wasysym}\usepackage{amsfonts}\usepackage{amssymb}\usepackage{amsbsy}\usepackage{mathrsfs}\usepackage{upgreek}\setlength{\oddsidemargin}{-69pt}\begin{document}$${{\mathcal {O}}}(1/\epsilon )$$\end{document} (resp., O(1/ϵ)\documentclass[12pt]{minimal}\usepackage{amsmath}\usepackage{wasysym}\usepackage{amsfonts}\usepackage{amssymb}\usepackage{amsbsy}\usepackage{mathrsfs}\usepackage{upgreek}\setlength{\oddsidemargin}{-69pt}\begin{document}$${{\mathcal {O}}}(1/\sqrt{\epsilon })$$\end{document}) communication rounds for general convex functions (resp., strongly convex functions), while maintaining the O(1/ϵ2)\documentclass[12pt]{minimal}\usepackage{amsmath}\usepackage{wasysym}\usepackage{amsfonts}\usepackage{amssymb}\usepackage{amsbsy}\usepackage{mathrsfs}\usepackage{upgreek}\setlength{\oddsidemargin}{-69pt}\begin{document}$${{\mathcal {O}}}(1/\epsilon ^2)$$\end{document} (resp., O(1/ϵ)\documentclass[12pt]{minimal}\usepackage{amsmath}\usepackage{wasysym}\usepackage{amsfonts}\usepackage{amssymb}\usepackage{amsbsy}\usepackage{mathrsfs}\usepackage{upgreek}\setlength{\oddsidemargin}{-69pt}\begin{document}$$\mathcal{O}(1/\epsilon )$$\end{document}) bound on the total number of intra-node subgradient evaluations. We also present a stochastic counterpart for these algorithms, denoted by SDCS, for solving stochastic optimization problems whose objective function cannot be evaluated exactly. In comparison with existing results for decentralized nonsmooth and stochastic optimization, we can reduce the total number of inter-node communication rounds by orders of magnitude while still maintaining the optimal complexity bounds on intra-node stochastic subgradient evaluations. The bounds on the (stochastic) subgradient evaluations are actually comparable to those required for centralized nonsmooth and stochastic optimization under certain conditions on the target accuracy.

An essentially decentralized interior point method for control

Decentralized Model Predictive Control of Nonlinear Continuous Systems with Contractive Constraints

Distributed Interior Point Methods for Optimization in Energy Networks

Decentralized non-convex optimization via bi-level SQP and ADMM

Proximal ADMM for Nonconvex and Nonsmooth Optimization

Decentralized Adaptive Neural Inverse Optimal Control of Nonlinear Interconnected Systems

A Decentralized Primal-Dual Method with Quasi-Newton Tracking

A fast, fully distributed nonlinear model predictive control algorithm with parametric sensitivity through Jacobi iteration

A Distributed Active Set Method for Model Predictive Control

A distributed primal-dual interior-point method for loosely coupled problems using ADMM

Distributed model predictive control of leader-follower systems using an interior point method with efficient computations

Fast and Stable Nonconvex Constrained Distributed Optimization: The ELLADA Algorithm

Sensitivity-Based Distributed Model Predictive Control for Nonlinear Systems under Inexact Optimization

Distributed Extremum Seeking Control of Networked Large-Scale Systems under Constraints.

A Unified Distributed Method for Constrained Networked Optimization via Saddle-Point Dynamics

Communication-efficient algorithms for decentralized and stochastic optimization

Decentralized Optimization with Distributed Features and Non-Smooth Objective Functions

Iterative distributed model predictive control for nonlinear systems with coupled non‐convex constraints and costs

Distributed Optimization Algorithm with Superlinear Convergence Rate

Regularized Interior Point Methods for Constrained Optimization and Control

A New Parallel Algorithm for Optimal-Control Problems of Interconnected Systems