Abstract:We consider distributed optimization problems, in which a group of agents are to collaboratively seek the global optimum through peer-to-peer communication networks. The problem arises in various application areas, such as resource allocation, sensor fusion, and distributed learning. We present a general algorithmic framework based on the Bregman method and operator splitting, which allows us to easily recover most of the existing distributed algorithms. Under this framework, we propose a general efficient distributed algorithm-distributed forward-backward Bregman splitting (D-FBBS)-to simultaneously solve the above primal problem as well as its dual. The proposed algorithm allows agents to communicate asynchronously and, thus, lends itself to stochastic networks. This algorithm is shown to have close connections with some existing well-known algorithms when dealing with fixed networks. However, we will show that it is generally different from the existing ones due to its effectiveness in handling stochastic networks. With proper assumptions, we establish a nonergodic convergence rate of O(1/k) in terms of fixed-point residuals over fixed networks both for D-FBBS and its inexact version (ID-FBBS) that is more computationally efficient and an ergodic convergence rate of O(1/k ) for D-FBBS over stochastic networks. We also apply the proposed algorithm to sensor fusion problems to show its superior performance compared to existing methods.

A Forward-Backward Bregman Splitting Scheme for Regularized Distributed Optimization Problems

A Bregman Splitting Scheme for Distributed Optimization over Networks

A Primal-Dual Forward-Backward Splitting Algorithm for Distributed Convex Optimization

Distributed Robust Optimization in Networked System.

Forward backward splitting for regularized stochastic optimization problems

A Unified Algorithmic Framework for Distributed Composite Optimization.

Accelerated Primal-Dual Algorithms for Distributed Smooth Convex Optimization over Networks

Distributed Algorithms for Composite Optimization: Unified Framework and Convergence Analysis

A Unified Contraction Analysis of a Class of Distributed Algorithms for Composite Optimization

A Dual Splitting Approach for Distributed Resource Allocation with Regularization

A Dual Splitting Algorithm For Distributed Resource Allocation Problems

(Rectified Version) The Barzilai-Borwein Method for Distributed Optimization over Unbalanced Directed Networks

Decentralized Proximal Splitting Algorithms for Composite Constrained Convex Optimization

Toward Designing Convergent Deep Operator Splitting Methods for Task-specific Nonconvex Optimization.

Algorithm 1 : Non-stationary Forward – Douglas – Rachford splitting

Regularized dual gradient distributed method for constrained convex optimization over unbalanced directed graphs

Decentralized Dual Operator Splitting for Nonsmooth Composite Optimization

A Multi-step Inertial Forward-Backward Splitting Method for Non-convex Optimization.

Distributed Bilevel Optimization via Adaptive Penalization with Time-Scale Separation

Event-triggered Broadcasting for Distributed Smooth Optimization

Distributed Optimization Algorithm with Superlinear Convergence Rate