Abstract:Multi-view (or -modality) representation learning aims to understand the relationships between different view representations. Existing methods disentangle multi-view representations into consistent and view-specific representations by introducing strong inductive biases, which can limit their generalization ability. In this paper, we propose a novel multi-view representation disentangling method that aims to go beyond inductive biases, ensuring both interpretability and generalizability of the resulting representations. Our method is based on the observation that discovering multi-view consistency in advance can determine the disentangling information boundary, leading to a decoupled learning objective. We also found that the consistency can be easily extracted by maximizing the transformation invariance and clustering consistency between views. These observations drive us to propose a two-stage framework. In the first stage, we obtain multi-view consistency by training a consistent encoder to produce semantically-consistent representations across views as well as their corresponding pseudo-labels. In the second stage, we disentangle specificity from comprehensive representations by minimizing the upper bound of mutual information between consistent and comprehensive representations. Finally, we reconstruct the original data by concatenating pseudo-labels and view-specific representations. Our experiments on four multi-view datasets demonstrate that our proposed method outperforms 12 comparison methods in terms of clustering and classification performance. The visualization results also show that the extracted consistency and specificity are compact and interpretable. Our code can be found at \url{<a class="link-external link-https" href="https://github.com/Guanzhou-Ke/DMRIB" rel="external noopener nofollow">this https URL</a>}.

UNTIE: Clustering analysis with disentanglement in multi-view information fusion

Multi-View Clustering Via Simultaneously Learning Shared Subspace And Affinity Matrix

Multi-VAE: Learning Disentangled View-common and View-peculiar Visual Representations for Multi-view Clustering

Multi-view Subspace Clustering Via Partition Fusion.

Multi-View Clustering from the Perspective of Mutual Information

Learning disentangled user representation with multi-view information fusion on social networks

Interpretable Multi-View Clustering

Diverse embeddings learning for multi-view clustering

Multi-View Data Fusion Oriented Clustering via Nuclear Norm Minimization

Disentangling Multi-view Representations Beyond Inductive Bias

Multi-view Embedded Clustering with Unsupervised Trace Ratio LDA

Fast Disentangled Slim Tensor Learning for Multi-view Clustering

Multi-View Matrix Decomposition: A New Scheme for Exploring Discriminative Information.

A Novel Approach for Effective Multi-View Clustering with Information-Theoretic Perspective

Deep Embedded Complementary and Interactive Information for Multi-View Classification

Partition Level Multiview Subspace Clustering.

Simple Contrastive Multi-View Clustering with Data-Level Fusion

Deep Incomplete Multi-View Clustering Via Mining Cluster Complementarity.

Generalized Information-theoretic Multi-view Clustering

Multi-View Information-Theoretic Co-Clustering for Co-Occurrence Data

Multi-graph Fusion for Multi-view Spectral Clustering