Abstract:We introduce a new family of distances, relative-translation invariant Wasserstein distances ($RW_p$), for measuring the similarity of two probability distributions under distribution shift. Generalizing it from the classical optimal transport model, we show that $RW_p$ distances are also real distance metrics defined on the quotient set $\mathcal{P}_p(\mathbb{R}^n)/\sim$ and invariant to distribution translations. When $p=2$, the $RW_2$ distance enjoys more exciting properties, including decomposability of the optimal transport model, translation-invariance of the $RW_2$ distance, and a Pythagorean relationship between $RW_2$ and the classical quadratic Wasserstein distance ($W_2$). Based on these properties, we show that a distribution shift, measured by $W_2$ distance, can be explained in the bias-variance perspective. In addition, we propose a variant of the Sinkhorn algorithm, named $RW_2$ Sinkhorn algorithm, for efficiently calculating $RW_2$ distance, coupling solutions, as well as $W_2$ distance. We also provide the analysis of numerical stability and time complexity for the proposed algorithm. Finally, we validate the $RW_2$ distance metric and the algorithm performance with three experiments. We conduct one numerical validation for the $RW_2$ Sinkhorn algorithm and show two real-world applications demonstrating the effectiveness of using $RW_2$ under distribution shift: digits recognition and similar thunderstorm detection. The experimental results report that our proposed algorithm significantly improves the computational efficiency of Sinkhorn in certain practical applications, and the $RW_2$ distance is robust to distribution translations compared with baselines.

RPD: A Distance Function Between Word Embeddings

Visual Exploration and Comparison of Word Embeddings.

WordRank: Learning Word Embeddings via Robust Ranking

An Exploration Of Semantic Relations In Neural Word Embeddings Using Extrinsic Knowledge

On the token distance modeling ability of higher RoPE attention dimension

Evaluation method of word embedding by roots and affixes

On the Dimensionality of Word Embedding

Evaluating the Utility of Document Embedding Vector Difference for Relation Learning

A New Robust Partial $p$-Wasserstein-Based Metric for Comparing Distributions

Wasserstein-Fisher-Rao Document Distance

Exploring Wasserstein Distance across Concept Embeddings for Ontology Matching

Word Embedding based Edit Distance

Learning Ordered Word Representations

The earth mover's distance as a semantic measure for document similarity.

Relative-Translation Invariant Wasserstein Distance

Measuring Unequal Knowledge Distance by Network Embedding and Multiple Relationships

A New Weighted Spearman's Footrule as A Measure of Distance between Rankings

GRI: Graph-based Relative Isomorphism of Word Embedding Spaces

Deconstructing and reconstructing word embedding algorithms

Towards Bridging the Distribution Gap: Instance to Prototype Earth Mover's Distance for Distribution Alignment.

Wasserstein task embedding for measuring task similarities