Abstract:In this work, we consider the distributed optimization problem in which each node has its own convex cost function and can communicate directly only with its neighbors, as determined by a directed communication topology (directed graph or digraph). First, we reformulate the optimization problem so that Alternating Direction Method of Multipliers (ADMM) can be utilized. Then, we propose an algorithm, herein called Distributed Alternating Direction Method of Multipliers using Finite-Time Exact Ratio Consensus (D-ADMM-FTERC), to solve the multi-node convex optimization problem, in which every node performs iterative computations and exchanges information with its neighbors. At every iteration of D-ADMM-FTERC, each node solves a local convex optimization problem for the one of the primal variables and utilizes a finite-time exact consensus protocol to obtain the optimal value of the other variable, since the cost function for the second primal variable is not decomposable. Since D-ADMM-FTERC requires to know the upper bound on the number of nodes in the network, we furthermore propose a new algorithm, called Fully D-ADMM Finite-Time Distributed Termination (FD-ADMM-FTDT) algorithm, which does not need any global information. If the individual cost functions are convex and not-necessarily differentiable, the proposed algorithms converge at a rate of O(1/k), where k is the iteration counter. Additionally, if the global objective function is strongly convex and smooth, the proposed algorithms have an "approximate" R-linear convergence rate. The efficacy of FD-ADMM-FTDT is demonstrated via a distributed L1 regularized logistic regression optimization example. Additionally, comparisons with other state-of-the-art algorithms are provided on large-scale networks showing the superior precision and time-efficient performance of FD-ADMM-FTDT.

Directed Transmission Method, A Fully Asynchronous approach to Solve Sparse Linear Systems in Parallel

Virtual Transmission Method, A New Distributed Algorithm to Solve Sparse Linear System

Virtual Transmission Method, A New Distributed Algorithm to Solve Sparse Linear Systems

Transmission line inspires a new distributed algorithm to solve linear system of circuit

Transmission line inspires a new distributed algorithm to solve the nonlinear dynamical system of physical circuit

Distributed Sparse Total Least-Squares over Networks

DSP: Dynamic Sequence Parallelism for Multi-Dimensional Transformers

Waveform Transmission Method, a New Waveform-relaxation Based Algorithm to Solve Ordinary Differential Equations in Parallel

Parallel Asynchronous Team Algorithms: Convergence and Performance Analysis

A Dual Splitting Algorithm For Distributed Resource Allocation Problems

Scalable Hierarchical Parallel Algorithm for the Solution of Super Large-Scale Sparse Linear Equations

Distributed synchronous and asynchronous algorithms for semi-definite programming with diagonal constraints

Robust Fully-Asynchronous Methods for Distributed Training over General Architecture

A distributed Douglas-Rachford splitting method for solving linear constrained multi-block weakly convex problems

Fully Distributed Alternating Direction Method of Multipliers in Digraphs via Finite-Time Termination Mechanisms

Task-D: A Task Based Programming Framework for Distributed System.

Nonlinear Asynchronous Block Iterative Method

Sparsity-Exploiting Distributed Projections onto a Simplex

Exploring the Design Space of Distributed Parallel Sparse Matrix-Multiple Vector Multiplication

A parallel ETD algorithm for large-scale rate theory simulation

A Distributed Asynchronous Method of Multipliers for Constrained Nonconvex Optimization