Abstract:This paper proposes a decentralized algorithm for solving a consensus optimization problem defined in a static networked multi-agent system, where the local objective functions have the smooth+nonsmooth composite form. Examples of such problems include decentralized constrained quadratic programming and compressed sensing problems, as well as many regularization problems arising in inverse problems, signal processing, and machine learning, which have decentralized applications. This paper addresses the need for efficient decentralized algorithms that take advantages of proximal operations for the nonsmooth terms. We propose a proximal gradient exact first-order algorithm (PG-EXTRA) that utilizes the composite structure and has the best known convergence rate. It is a nontrivial extension to the recent algorithm EXTRA. At each iteration, each agent locally computes a gradient of the smooth part of its objective and a proximal map of the nonsmooth part, as well as exchanges information with its neighbors. The algorithm is “exact” in the sense that an exact consensus minimizer can be obtained with a fixed step size, whereas most previous methods must use diminishing step sizes. When the smooth part has Lipschitz gradients, PG-EXTRA has an ergodic convergence rate of O(1/k) in terms of the first-order optimality residual. When the smooth part vanishes, PG-EXTRA reduces to P-EXTRA, an algorithm without the gradients (so no “G” in the name), which has a slightly improved convergence rate at o(1/k) in a standard (non-ergodic) sense. Numerical experiments demonstrate effectiveness of PG-EXTRA and validate our convergence results.

Decentralized Proximal Splitting Algorithms for Composite Constrained Convex Optimization

Decentralized Dual Operator Splitting for Nonsmooth Composite Optimization

Decentralized Constrained Consensus Optimization with Primal Dual Splitting Projection.

A Primal-Dual Forward-Backward Splitting Algorithm for Distributed Convex Optimization

Distributed Algorithms for Composite Optimization: Unified Framework and Convergence Analysis

A Decentralized Stochastic Algorithm for Coupled Composite Optimization with Linear Convergence

A Unified Algorithmic Framework for Distributed Composite Optimization.

A Unified Contraction Analysis of a Class of Distributed Algorithms for Composite Optimization

A Forward-Backward Bregman Splitting Scheme for Regularized Distributed Optimization Problems

Distributed Primal-Dual Proximal Algorithms for Convex Optimization Involving Three Composite Functions

Distributed Nonsmooth Optimization with Consensus and Inequality Constraints Via Distributed Smooth Proximal-Splitting Algorithm

Decentralized Proximal Method of Multipliers for Convex Optimization with Coupled Constraints

Decentralized Proximal Method of Multipliers for Convex Optimization with Coupled Constraints

Linearized Proximal Algorithms with Adaptive Stepsizes for Convex Composite Optimization with Applications

A Computation-Efficient Decentralized Algorithm for Composite Constrained Optimization

Distributed Proximal Algorithms for Multiagent Optimization With Coupled Inequality Constraints

DISA: A Dual Inexact Splitting Algorithm for Distributed Convex Composite Optimization

A Dual Splitting Algorithm For Distributed Resource Allocation Problems

A One-Sample Decentralized Proximal Algorithm for Non-Convex Stochastic Composite Optimization

A Proximal Gradient Algorithm for Decentralized Composite Optimization

Distributed Multiproximal Algorithm for Nonsmooth Convex Optimization With Coupled Inequality Constraints