Abstract:Contrastive learning (CL) methods effectively learn data representations in a self-supervision manner, where the encoder contrasts each positive sample over multiple negative samples via a one-vs-many softmax cross-entropy loss. By leveraging large amounts of unlabeled image data, recent CL methods have achieved promising results when pretrained on large-scale datasets, such as ImageNet. However, most of them consider the augmented views from the same instance are positive pairs, while views from other instances are negative ones. Such binary partition insufficiently considers the relation between samples and tends to yield worse performance when generalized on images in the wild. In this paper, to further improve the performance of CL and enhance its robustness on various datasets, {we propose a doubly CL strategy that separately compares positive and negative samples within their own groups, and then proceeds with a contrast between positive and negative groups}. We realize this strategy with contrastive attraction and contrastive repulsion (CACR), which makes the query not only exert a greater force to attract more distant positive samples but also do so to repel closer negative samples. Theoretical analysis reveals that CACR generalizes CL's behavior by positive attraction and negative repulsion, and it further considers the intra-contrastive relation within the positive and negative pairs to narrow the gap between the sampled and true distribution, which is important when datasets are less curated. With our extensive experiments, CACR not only demonstrates good performance on CL benchmarks, but also shows better robustness when generalized on imbalanced image datasets. Code and pre-trained checkpoints are available at <a class="link-external link-https" href="https://github.com/JegZheng/CACR-SSL" rel="external noopener nofollow">this https URL</a>.

PCL: Proxy-based Contrastive Learning for Domain Generalization

Debiased Graph Contrastive Learning.

Probabilistic Contrastive Learning for Domain Adaptation

Causality-based Dual-Contrastive Learning Framework for Domain Generalization

Contrastive-enhanced Domain Generalization with Federated Learning

Debiased Contrastive Curriculum Learning for Progressive Generalizable Person Re-identification

Contrast and Clustering: Learning Neighborhood Pair Representation for Source-free Domain Adaptation

Generalized Semantic Segmentation by Self-Supervised Source Domain Projection and Multi-Level Contrastive Learning

Domain Contrast for Domain Adaptive Object Detection

Negative as Positive: Enhancing Out-of-distribution Generalization for Graph Contrastive Learning

Calibration-based Dual Prototypical Contrastive Learning Approach for Domain Generalization Semantic Segmentation

Unbiased Semantic Representation Learning Based on Causal Disentanglement for Domain Generalization

Domain Generalization for Domain-Linked Classes

Constrained Maximum Cross-Domain Likelihood for Domain Generalization

Domain Generalization for Mammographic Image Analysis with Contrastive Learning

Bilateral Proxy Federated Domain Generalization for Privacy-preserving Medical Image Diagnosis

Multi-Scale and Multi-Layer Contrastive Learning for Domain Generalization

Multi-Domain Learning From Insufficient Annotations

Domain Generalization Via Inter-Domain Alignment and Intra-Domain Expansion

A Robust Contrastive Alignment Method For Multi-Domain Text Classification

Contrastive Attraction and Contrastive Repulsion for Representation Learning