Abstract:The performance of unsupervised methods such as clustering depends on the choice of distance metric between features, or ground metric. Commonly, ground metrics are decided with heuristics or learned via supervised algorithms. However, since many datasets are unlabelled, unsupervised ground metric learning approaches have been introduced. One recent, promising option uses Wasserstein singular vectors (WSV), which emerge when computing optimal transport distances between features and samples simultaneously. While WSV is effective, it has complexity $\mathcal{O}(n^5)$, which is prohibitively expensive in some applications. In this work, we propose to augment the WSV method by embedding samples and features on trees, on which we compute the tree-Wasserstein distance (TWD). We demonstrate theoretically and empirically that the algorithm converges to a better approximation of the full WSV approach than the best known alternatives, and does so with $\mathcal{O}(n^3)$ complexity. In addition, we prove that the initial tree structure can be chosen flexibly, since tree geometry does not constrain the richness of the approximation up to the number of edge weights. This proof suggests a fast, recursive algorithm for computing the tree parameter basis set, which we find crucial to realising the efficiency gains at scale. Finally, we employ the tree-WSV algorithm to several single-cell RNA sequencing genomics datasets, demonstrating its scalability and utility for unsupervised cell-type clustering problems. These results poise unsupervised ground metric learning with TWD as a low-rank approximation of WSV with the potential for widespread low-compute application.

Supervised Word Mover's Distance

Wasserstein-Fisher-Rao Document Distance

Learning Unsupervised Word Mapping by Maximizing Mean Discrepancy

Alignment-Aware Word Distance.

Word Embedding based Edit Distance

Supervised Tree-Wasserstein Distance

A Simple Post-Processing Technique for Improving Readability Assessment of Texts using Word Mover's Distance

A Semantic Distance Metric Learning approach for Lexical Semantic Change Detection

A novel document similarity measure based on earth mover's distance

Gaussian Word Embedding with a Wasserstein Distance Loss

Earth Mover's Distance Minimization for Unsupervised Bilingual Lexicon Induction.

Document classification with distributions of word vectors

An Empirical Study of Self-Supervised Learning with Wasserstein Distance

Distance-weighted Support Vector Machine

The earth mover's distance as a semantic measure for document similarity.

WIMMC: Weighted Incremental Maximum Margin Criterion

Fast unsupervised ground metric learning with tree-Wasserstein distance

Local Relevance Weighted Maximum Margin Criterion for Text Classification.

Learning Word Embedding with Better Distance Weighting and Window Size Scheduling

Local Word Bag Model for Text Categorization