Abstract:Distributed distribution comparison aims to measure the distance between the distributions whose data are scattered across different agents in a distributed system and cannot even be shared directly among the agents. In this study, we propose a novel decentralized entropic optimal transport (DEOT) method, which provides a communication-efficient and privacy-preserving solution to this problem with theoretical guarantees. In particular, we design a mini-batch randomized block-coordinate descent (MRBCD) scheme to optimize the DEOT distance in its dual form. The dual variables are scattered across different agents and updated locally and iteratively with limited communications among partial agents. The kernel matrix involved in the gradients of the dual variables is estimated by a decentralized kernel approximation method, in which each agent only needs to approximate and store a sub-kernel matrix by one-shot communication and without sharing raw data. Besides computing entropic Wasserstein distance, we show that the proposed MRBCD scheme and kernel approximation method also apply to entropic Gromov-Wasserstein distance. We analyze our method's communication complexity and, under mild assumptions, provide a theoretical bound for the approximation error caused by the convergence error, the estimated kernel, and the mismatch between the storage and communication protocols. In addition, we discuss the trade-off between the precision of the EOT distance and the strength of privacy protection when implementing our method. Experiments on synthetic data and real-world distributed domain adaptation tasks demonstrate the effectiveness of our method.

Consensus-based Distributed Discrete Optimal Transport for Decentralized Resource Matching

Fair and Distributed Dynamic Optimal Transport for Resource Allocation over Networks

A Distributed Consensus Based Algorithm for Economic Dispatch over Time‐varying Digraphs

An Exponentially Convergent Distributed Algorithm for Resource Allocation Problem

Decentralized and Equitable Optimal Transport

Distributed Optimal Consensus over Resource Allocation Network and Its Application to Dynamical Economic Dispatch.

Distributed Algorithms for Composite Optimization: Unified Framework and Convergence Analysis

A Dual Splitting Algorithm For Distributed Resource Allocation Problems

Differentially Private Distributed Resource Allocation Via Deviation Tracking

Decentralized Entropic Optimal Transport for Distributed Distribution Comparison

Resilient and Distributed Discrete Optimal Transport with Deceptive Adversary: A Game-Theoretic Approach

Distributed Online Optimization for Multi-Agent Optimal Transport

Distributed Optimal Consensus Algorithms in Multi-Agent Systems

An Optimal Stochastic Algorithm for Decentralized Nonconvex Finite-sum Optimization

A Biased Min-Consensus-Based Approach for Optimal Power Transaction in Multi-Energy-Router Systems

Randomized Methods for Computing Optimal Transport Without Regularization and Their Convergence Analysis

Distributed Parametric Consensus Optimization with an Application to Model Predictive Consensus Problem

Distributed Optimal Consensus of Multi-Agent Systems: A Randomized Parallel Approach

Primal–Dual Algorithms for Distributed Economic Dispatch

Distributed resource allocation: an indirect dual ascent method with an exponential convergence rate

Consensus-based Distributed Optimization for Multi-agent Systems over Multiplex Networks