Abstract:Traditional multi-view learning methods often rely on two assumptions: (i) the samples in different views are well-aligned, and (ii) their representations obey the same distribution in a latent space. Unfortunately, these two assumptions may be questionable in practice, which limits the application of multi-view learning. In this work, we propose a differentiable hierarchical optimal transport (DHOT) method to mitigate the dependency of multi-view learning on these two assumptions. Given arbitrary two views of unaligned multi-view data, the DHOT method calculates the sliced Wasserstein distance between their latent distributions. Based on these sliced Wasserstein distances, the DHOT method further calculates the entropic optimal transport across different views and explicitly indicates the clustering structure of the views. Accordingly, the entropic optimal transport, together with the underlying sliced Wasserstein distances, leads to a hierarchical optimal transport distance defined for unaligned multi-view data, which works as the objective function of multi-view learning and leads to a bi-level optimization task. Moreover, our DHOT method treats the entropic optimal transport as a differentiable operator of model parameters. It considers the gradient of the entropic optimal transport in the backpropagation step and thus helps improve the descent direction for the model in the training phase. We demonstrate the superiority of our bi-level optimization strategy by comparing it to the traditional alternating optimization strategy. The DHOT method is applicable for both unsupervised and semi-supervised learning. Experimental results show that our DHOT method is at least comparable to state-of-the-art multi-view learning methods on both synthetic and real-world tasks, especially for challenging scenarios with unaligned multi-view data.

Hierarchical Optimal Transport for Multimodal Distribution Alignment

HOMDA: High-Order Moment-Based Domain Alignment for Unsupervised Domain Adaptation

Differentiable Hierarchical Optimal Transport for Robust Multi-View Learning

MM-Align: Learning Optimal Transport-based Alignment Dynamics for Fast and Accurate Inference on Missing Modality Sequences

Distributional Preference Alignment of LLMs via Optimal Transport

Modeling Hierarchical Structural Distance for Unsupervised Domain Adaptation

Gromov-Wasserstein optimal transport to align single-cell multi-omics data

Entropic Optimal Transport Eigenmaps for Nonlinear Alignment and Joint Embedding of High-Dimensional Datasets

Scalable unsupervised alignment of general metric and non-metric structures

Double-Bounded Optimal Transport for Advanced Clustering and Classification

Scalable Optimal Transport in High Dimensions for Graph Distances, Embedding Alignment, and More

A Class-aware Optimal Transport Approach with Higher-Order Moment Matching for Unsupervised Domain Adaptation

Graph Optimal Transport for Cross-Domain Alignment

Robust Graph Matching Using An Unbalanced Hierarchical Optimal Transport Framework

Integrating Efficient Optimal Transport and Functional Maps For Unsupervised Shape Correspondence Learning

A Unified Optimal Transport Framework for Cross-Modal Retrieval with Noisy Labels

Combining Optimal Transport and Embedding-Based Approaches for More Expressiveness in Unsupervised Graph Alignment

Inducing Semantic Hierarchy Structure in Empirical Risk Minimization with Optimal Transport Measures

Neural Entropic Gromov-Wasserstein Alignment

Gromov-Wasserstein Multi-modal Alignment and Clustering

Efficient and Exact Multimarginal Optimal Transport with Pairwise Costs