Abstract:With the increasing availability of multi-view data, multi-view representation learning has emerged as a prominent research area. However, collecting strictly view-aligned data is usually expensive, and learning from both aligned and unaligned data can be more practicable. Therefore, Partially View-aligned Representation Learning (PVRL) has recently attracted increasing attention. After aligning multi-view representations based on their semantic similarity, the aligned representations can be utilized to facilitate downstream tasks, such as clustering. However, existing methods may be constrained by the following limitations: 1) They learn semantic relations across views using the known correspondences, which is incomplete and the existence of false negative pairs (FNP) can significantly impact the learning effectiveness; 2) Existing strategies for alleviating the impact of FNP are too intuitive and lack a theoretical explanation of their applicable conditions; 3) They attempt to find FNP based on distance in the common space and fail to explore semantic relations between multi-view data. In this paper, we propose a Nonparametric Clustering-guided Cross-view Contrastive Learning (NC3L) for PVRL, in order to address the above issues. Firstly, we propose to estimate the similarity matrix between multi-view data in the marginal cross-view contrastive loss to approximate the similarity matrix of supervised contrastive learning (CL). Secondly, we establish the theoretical foundation for our proposed method by analyzing the error bounds of the loss function and its derivatives between our method and supervised CL. Thirdly, we propose a Deep Variational Nonparametric Clustering (DeepVNC) by designing a deep reparameterized variational inference for Dirichlet process Gaussian mixture models to construct cluster-level similarity between multi-view data and discover FNP. Additionally, we propose a reparameterization trick to improve the robustness and the performance of our proposed CL method. Extensive experiments on four widely used benchmark datasets show the superiority of our proposed method compared with state-of-the-art methods.

Contrastive Multi-View Learning for 3D Shape Clustering

Multi-View Clustering Via Simultaneously Learning Shared Subspace And Affinity Matrix

View-Driven Multi-View Clustering via Contrastive Double-Learning

Dual Contrast-Driven Deep Multi-View Clustering

Structure-guided feature and cluster contrastive learning for multi-view clustering

Joint Learning of Latent Similarity and Local Embedding for Multi-View Clustering

Subspace-Contrastive Multi-View Clustering

Deep Multiview Clustering by Contrasting Cluster Assignments

Iterative Deep Structural Graph Contrast Clustering for Multiview Raw Data

Global and local combined contrastive learning for multi-view clustering

Multiview Long-Short Spatial Contrastive Learning For 3D Medical Image Analysis

Mejigclu: more effective jigsaw clustering for unsupervised visual representation learning

Contrastive and attentive graph learning for multi-view clustering

Selective Contrastive Learning for Unpaired Multi-View Clustering

Multi-View Subspace Clustering with Consensus Graph Contrastive Learning

Nonparametric Clustering-Guided Cross-View Contrastive Learning for Partially View-Aligned Representation Learning

Incomplete multi-view clustering via attention-based contrast learning

Dual-dimensional contrastive learning for incomplete multi-view clustering

MLVCNN: Multi-Loop-View Convolutional Neural Network for 3D Shape Retrieval

MVContrast: Unsupervised Pretraining for Multi-view 3D Object Recognition

Composite attention mechanism network for deep contrastive multi-view clustering