Abstract:Multi-view (or -modality) representation learning aims to understand the relationships between different view representations. Existing methods disentangle multi-view representations into consistent and view-specific representations by introducing strong inductive biases, which can limit their generalization ability. In this paper, we propose a novel multi-view representation disentangling method that aims to go beyond inductive biases, ensuring both interpretability and generalizability of the resulting representations. Our method is based on the observation that discovering multi-view consistency in advance can determine the disentangling information boundary, leading to a decoupled learning objective. We also found that the consistency can be easily extracted by maximizing the transformation invariance and clustering consistency between views. These observations drive us to propose a two-stage framework. In the first stage, we obtain multi-view consistency by training a consistent encoder to produce semantically-consistent representations across views as well as their corresponding pseudo-labels. In the second stage, we disentangle specificity from comprehensive representations by minimizing the upper bound of mutual information between consistent and comprehensive representations. Finally, we reconstruct the original data by concatenating pseudo-labels and view-specific representations. Our experiments on four multi-view datasets demonstrate that our proposed method outperforms 12 comparison methods in terms of clustering and classification performance. The visualization results also show that the extracted consistency and specificity are compact and interpretable. Our code can be found at \url{<a class="link-external link-https" href="https://github.com/Guanzhou-Ke/DMRIB" rel="external noopener nofollow">this https URL</a>}.

Geometric Inductive Biases for Identifiable Unsupervised Learning of Disentangled Representations.

Deep Clustering and Representation Learning that Preserves Geometric Structures

Tripod: Three Complementary Inductive Biases for Disentangled Representation Learning

A Regularized Approach for Geodesic-Based Semisupervised Multimanifold Learning

Nonlinear independent component analysis for principled disentanglement in unsupervised deep learning

Unveiling and Mitigating Generalized Biases of DNNs through the Intrinsic Dimensions of Perceptual Manifolds

Geodesic Based Semi-supervised Multi-manifold Feature Extraction

Disentangling Factors of Variation in Deep Representations Using Adversarial Training.

Rethinking Controllable Variational Autoencoders

Disentanglement via Latent Quantization

Guided Variational Autoencoder for Disentanglement Learning

Geometric Inductive Biases of Deep Networks: The Role of Data and Architecture

Disentangling Multi-view Representations Beyond Inductive Bias

Improving disentanglement in variational auto-encoders via feature imbalance-informed dimension weighting

Extendable and invertible manifold learning with geometry regularized autoencoders

Learning Disentangled Discrete Representations

Disentanglement via Mechanism Sparsity Regularization: A New Principle for Nonlinear ICA

Identifiability Guarantees for Causal Disentanglement from Purely Observational Data

Disentanglement with Factor Quantized Variational Autoencoders

On Inductive Biases That Enable Generalization of Diffusion Transformers

Unbiased Semantic Representation Learning Based on Causal Disentanglement for Domain Generalization