Abstract:Recently, contrastive learning based on augmentation invariance and instance discrimination has made great achievements, owing to its excellent ability to learn beneficial representations without any manual annotations. However, the natural similarity among instances conflicts with instance discrimination which treats each instance as a unique individual. In order to explore the natural relationship among instances and integrate it into contrastive learning, we propose a novel approach in this paper, Relationship Alignment (RA for abbreviation), which forces different augmented views of current batch instances to main a consistent relationship with other instances. In order to perform RA effectively in existing contrastive learning framework, we design an alternating optimization algorithm where the relationship exploration step and alignment step are optimized respectively. In addition, we add an equilibrium constraint for RA to avoid the degenerate solution, and introduce the expansion handler to make it approximately satisfied in practice. In order to better capture the complex relationship among instances, we additionally propose Multi-Dimensional Relationship Alignment (MDRA for abbreviation), which aims to explore the relationship from multiple dimensions. In practice, we decompose the final high-dimensional feature space into a cartesian product of several low-dimensional subspaces and perform RA in each subspace respectively. We validate the effectiveness of our approach on multiple self-supervised learning benchmarks and get consistent improvements compared with current popular contrastive learning methods. On the most commonly used ImageNet linear evaluation protocol, our RA obtains significant improvements over other methods, our MDRA gets further improvements based on RA to achieve the best performance. The source code of our approach will be released soon.

Align Representations with Base: A New Approach to Self-Supervised Learning

Towards Self-Similarity Consistency and Feature Discrimination for Unsupervised Domain Adaptation.

CSR-Net++: Rethinking Context Structure Representation Learning for Feature Matching

Unsupervised Visual Representation Learning via Multi-Dimensional Relationship Alignment

Hyperspherical Consistency Regularization.

Align Yourself: Self-supervised Pre-training for Fine-grained Recognition via Saliency Alignment.

Seed the Views: Hierarchical Semantic Alignment for Contrastive Representation Learning

Enhancing Representations through Heterogeneous Self-Supervised Learning

A Unified Framework for Contrastive Learning from a Perspective of Affinity Matrix

An unsupervised multi-view contrastive learning framework with attention-based reranking strategy for entity alignment

Adaptive Similarity Bootstrapping for Self-Distillation based Representation Learning

Towards a Unified Theoretical Understanding of Non-contrastive Learning via Rank Differential Mechanism

Constrained Multiview Representation for Self-supervised Contrastive Learning

Rethinking Positive Pairs in Contrastive Learning

Switchable Representation Learning Framework with Self-Compatibility.

Towards Universal Backward-Compatible Representation Learning

Understanding Contrastive Representation Learning through Alignment and Uniformity on the Hypersphere

Boosting Zero-shot Learning via Contrastive Optimization of Attribute Representations

P4Contrast: Contrastive Learning with Pairs of Point-Pixel Pairs for RGB-D Scene Understanding

A Novel Contrastive Learning Model for Aerial Images

ContrastAlign: Toward Robust BEV Feature Alignment via Contrastive Learning for Multi-Modal 3D Object Detection