Abstract:Multi-view representation learning aims to extract comprehensive information from multiple sources. It has achieved significant success in applications such as video understanding and 3D rendering. However, how to improve the robustness and generalization of multi-view representations from unsupervised and incomplete scenarios remains an open question in this field. In this study, we discovered a positive correlation between the semantic distance of multi-view representations and the tolerance for data corruption. Moreover, we found that the information ratio of consistency and complementarity significantly impacts the performance of discriminative and generative tasks related to multi-view representations. Based on these observations, we propose an end-to-end CLustering-guided cOntrastiVE fusioN (CLOVEN) method, which enhances the robustness and generalization of multi-view representations simultaneously. To balance consistency and complementarity, we design an asymmetric contrastive fusion module. The module first combines all view-specific representations into a comprehensive representation through a scaling fusion layer. Then, the information of the comprehensive representation and view-specific representations is aligned via contrastive learning loss function, resulting in a view-common representation that includes both consistent and complementary information. We prevent the module from learning suboptimal solutions by not allowing information alignment between view-specific representations. We design a clustering-guided module that encourages the aggregation of semantically similar views. This action reduces the semantic distance of the view-common representation. We quantitatively and qualitatively evaluate CLOVEN on five datasets, demonstrating its superiority over 13 other competitive multi-view learning methods in terms of clustering and classification performance. In the data-corrupted scenario, our proposed method resists noise interference better than competitors. Additionally, the visualization demonstrates that CLOVEN succeeds in preserving the intrinsic structure of view-specific representations and improves the compactness of view-common representations. Our code can be found at https://github.com/guanzhou-ke/cloven.

Dual Contrast-Driven Deep Multi-View Clustering

View-Driven Multi-View Clustering via Contrastive Double-Learning

Learning Consensus Representations in Multi-Latent Spaces for Multi-View Clustering

A Clustering-guided Contrastive Fusion for Multi-view Representation Learning

Multi-View Clustering with Semantic Fusion and Contrastive Learning

Dual-Contrastive Multi-view Clustering under the Guidance of Global Similarity and Pseudo-label

Anchor-Sharing and Clusterwise Contrastive Network for Multiview Representation Learning

Deep Contrastive Multi-View Subspace Clustering with Representation and Cluster Interactive Learning

Consistency Enhancement-Based Deep Multiview Clustering via Contrastive Learning

Deep Contrastive Multi-view Clustering under Semantic Feature Guidance

Self-Weighted Contrastive Fusion for Deep Multi-View Clustering

Self-Expressive Based Multi-View Clustering Via Contrastive Learning

CCR-Net: Consistent Contrastive Representation Network for Multi-View Clustering.

Dual Contrastive Learning for Multi-View Clustering

Composite attention mechanism network for deep contrastive multi-view clustering

Contrastive Consistency and Attentive Complementarity for Deep Multi-View Subspace Clustering

Incomplete Contrastive Multi-View Clustering with High-Confidence Guiding

Multi-View Subspace Clustering with Consensus Graph Contrastive Learning

Multi-level Feature Learning for Contrastive Multi-view Clustering

Dual-dimensional contrastive learning for incomplete multi-view clustering