Abstract:Recently, contrastive learning based on augmentation invariance and instance discrimination has made great achievements, owing to its excellent ability to learn beneficial representations without any manual annotations. However, the natural similarity among instances conflicts with instance discrimination which treats each instance as a unique individual. In order to explore the natural relationship among instances and integrate it into contrastive learning, we propose a novel approach in this paper, Relationship Alignment (RA for abbreviation), which forces different augmented views of current batch instances to main a consistent relationship with other instances. In order to perform RA effectively in existing contrastive learning framework, we design an alternating optimization algorithm where the relationship exploration step and alignment step are optimized respectively. In addition, we add an equilibrium constraint for RA to avoid the degenerate solution, and introduce the expansion handler to make it approximately satisfied in practice. In order to better capture the complex relationship among instances, we additionally propose Multi-Dimensional Relationship Alignment (MDRA for abbreviation), which aims to explore the relationship from multiple dimensions. In practice, we decompose the final high-dimensional feature space into a cartesian product of several low-dimensional subspaces and perform RA in each subspace respectively. We validate the effectiveness of our approach on multiple self-supervised learning benchmarks and get consistent improvements compared with current popular contrastive learning methods. On the most commonly used ImageNet linear evaluation protocol, our RA obtains significant improvements over other methods, our MDRA gets further improvements based on RA to achieve the best performance. The source code of our approach will be released soon.

Align Yourself: Self-supervised Pre-training for Fine-grained Recognition via Saliency Alignment.

Exploring Localization for Self-supervised Fine-grained Contrastive Learning

Saliency Guided Contrastive Learning on Scene Images

Point Contrastive Prediction with Semantic Clustering for Self-Supervised Learning on Point Cloud Videos

Boost Supervised Pretraining for Visual Transfer Learning: Implications of Self-Supervised Contrastive Representation Learning.

Learning multi-view visual correspondences with self-supervision

Distilling Localization for Self-Supervised Representation Learning

LoDisc: Learning Global-Local Discriminative Features for Self-Supervised Fine-Grained Visual Recognition

Toward High Quality Facial Representation Learning

Dense Contrastive Learning for Self-Supervised Visual Pre-Training

HAPiCLR: heuristic attention pixel-level contrastive loss representation learning for self-supervised pretraining

Cross-view motion consistent self-supervised video inter-intra contrastive for action representation understanding

GroupContrast: Semantic-aware Self-supervised Representation Learning for 3D Understanding

P4Contrast: Contrastive Learning with Pairs of Point-Pixel Pairs for RGB-D Scene Understanding

Contrastive Transformation for Self-supervised Correspondence Learning

LPCL: Localized Prominence Contrastive Learning for Self-Supervised Dense Visual Pre-Training

Object Representation Enhancement for Self‐supervised Colocalization

Unsupervised Visual Representation Learning via Multi-Dimensional Relationship Alignment

CAVL: Learning Contrastive and Adaptive Representations of Vision and Language

Masking Improves Contrastive Self-Supervised Learning for ConvNets, and Saliency Tells You Where

Investigating Contrastive Pair Learning's Frontiers in Supervised, Semisupervised, and Self-Supervised Learning