Abstract:We study the Unbalanced Optimal Transport (UOT) between two measures of possibly different masses with at most $n$ components, where the marginal constraints of standard Optimal Transport (OT) are relaxed via Kullback-Leibler divergence with regularization factor $\tau$. Although only Sinkhorn-based UOT solvers have been analyzed in the literature with the iteration complexity of ${O}\big(\tfrac{\tau \log(n)}{\varepsilon} \log\big(\tfrac{\log(n)}{\varepsilon}\big)\big)$ and per-iteration cost of $O(n^2)$ for achieving the desired error $\varepsilon$, their positively dense output transportation plans strongly hinder the practicality. On the other hand, while being vastly used as heuristics for computing UOT in modern deep learning applications and having shown success in sparse OT problem, gradient methods applied to UOT have not been formally studied. In this paper, we propose a novel algorithm based on Gradient Extrapolation Method (GEM-UOT) to find an $\varepsilon$-approximate solution to the UOT problem in $O\big( \kappa \log\big(\frac{\tau n}{\varepsilon}\big) \big)$ iterations with $\widetilde{O}(n^2)$ per-iteration cost, where $\kappa$ is the condition number depending on only the two input measures. Our proof technique is based on a novel dual formulation of the squared $\ell_2$-norm UOT objective, which fills the lack of sparse UOT literature and also leads to a new characterization of approximation error between UOT and OT. To this end, we further present a novel approach of OT retrieval from UOT, which is based on GEM-UOT with fine tuned $\tau$ and a post-process projection step. Extensive experiments on synthetic and real datasets validate our theories and demonstrate the favorable performance of our methods in practice.

Improved Complexity Analysis of the Sinkhorn and Greenkhorn Algorithms for Optimal Transport

A Sinkhorn-type Algorithm for Constrained Optimal Transport

Efficient Optimal Transport Algorithm by Accelerated Gradient descent

Sinkhorn Algorithm for Sequentially Composed Optimal Transports

Sharper Exponential Convergence Rates for Sinkhorn's Algorithm in Continuous Settings

Fast Computation of Optimal Transport via Entropy-Regularized Extragradient Methods

On the geometry and dynamical formulation of the Sinkhorn algorithm for optimal transport

Non-asymptotic convergence bounds for Sinkhorn iterates and their gradients: a coupling approach

Fast Sinkhorn I: an O(N) Algorithm for the Wasserstein-1 Metric

Fast Sinkhorn II: Collinear Triangular Matrix and Linear Time Accurate Computation of Optimal Transport

Algorithms for Euclidean-regularised Optimal Transport

Limit Theorems for Entropic Optimal Transport Maps and the Sinkhorn Divergence

Sinkhorn Distances: Lightspeed Computation of Optimal Transport

Algorithms for constrained optimal transport

Convergence of Sinkhorn's Algorithm for Entropic Martingale Optimal Transport Problem

Rethinking Initialization of the Sinkhorn Algorithm

Quantitative contraction rates for Sinkhorn algorithm: beyond bounded costs and compact marginals

Sinkhorn algorithms and linear programming solvers for optimal partial transport problems

A semiconcavity approach to stability of entropic plans and exponential convergence of Sinkhorn's algorithm

On Unbalanced Optimal Transport: Gradient Methods, Sparsity and Approximation Error

Annealed Sinkhorn for Optimal Transport: convergence, regularization path and debiasing