Abstract:The goal of contrastive learning based pre-training is to leverage large quantities of unlabeled data to produce a model that can be readily adapted downstream. Current approaches revolve around solving an image discrimination task: given an anchor image, an augmented counterpart of that image, and some other images, the model must produce representations such that the distance between the anchor and its counterpart is small, and the distances between the anchor and the other images are large. There are two significant problems with this approach: (i) by contrasting representations at the image-level, it is hard to generate detailed object-sensitive features that are beneficial to downstream object-level tasks such as instance segmentation; (ii) the augmentation strategy of producing an augmented counterpart is fixed, making learning less effective at the later stages of pre-training. In this work, we introduce Curricular Contrastive Object-level Pre-training (CCOP) to tackle these problems: (i) we use selective search to find rough object regions and use them to build an inter-image object-level contrastive loss and an intra-image object-level discrimination loss into our pre-training objective; (ii) we present a curriculum learning mechanism that adaptively augments the generated regions, which allows the model to consistently acquire a useful learning signal, even in the later stages of pre-training. Our experiments show that our approach improves on the MoCo v2 baseline by a large margin on multiple object-level tasks when pre-training on multi-object scene image datasets. Code is available at <a class="link-external link-https" href="https://github.com/ChenhongyiYang/CCOP" rel="external noopener nofollow">this https URL</a>.

CP2: Copy-Paste Contrastive Pretraining for Semantic Segmentation

Cross-Image Pixel Contrasting for Semantic Segmentation

Dense Contrastive Learning for Self-Supervised Visual Pre-Training

DenseCL: A Simple Framework for Self-Supervised Dense Visual Pre-Training

Exploring Cross-Image Pixel Contrast for Semantic Segmentation

Propagate Yourself: Exploring Pixel-Level Consistency for Unsupervised Visual Representation Learning

Pairwise-Pixel Self-Supervised and Superpixel-Guided Prototype Contrastive Loss for Weakly Supervised Semantic Segmentation

Pixel Contrastive-Consistent Semi-Supervised Semantic Segmentation

Region-aware Contrastive Learning for Semantic Segmentation

SegCLIP: Patch Aggregation with Learnable Centers for Open-Vocabulary Semantic Segmentation

Contrastive Object-level Pre-training with Spatial Noise Curriculum Learning

CLIP2Scene: Towards Label-efficient 3D Scene Understanding by CLIP

SCNet: Enhancing Few-Shot Semantic Segmentation by Self-Contrastive Background Prototypes

Adversarial Dense Contrastive Learning for Semi-Supervised Semantic Segmentation

Dual Prototypical Contrastive Learning for Few-shot Semantic Segmentation

ICPC: Instance-Conditioned Prompting with Contrastive Learning for Semantic Segmentation

CLIPer: Hierarchically Improving Spatial Representation of CLIP for Open-Vocabulary Semantic Segmentation

HAPiCLR: heuristic attention pixel-level contrastive loss representation learning for self-supervised pretraining

Multi-dataset Pretraining: A Unified Model for Semantic Segmentation

PRCL: Probabilistic Representation Contrastive Learning for Semi-Supervised Semantic Segmentation