Abstract:Self-supervised pre-training has attracted increasing attention given its promising performance in train-ing backbone networks without using labels. By far, most methods focus on image classification with datasets containing iconic objects and simple background, e.g. ImageNet. However, these methods show sub-optimal performance for dense prediction tasks (e.g. object detection and scene parsing) when di-rectly pre-training on datasets (e.g. PASCAL VOC and COCO) with multiple objects and cluttered back-grounds. Researchers explored self-supervised dense pre-training methods by adapting recent image pre -training methods. Nevertheless, they require a large number of negative samples and a long training time to reach reasonable performance. In this paper, we propose LPCL, a novel self-supervised representation learning method for dense predictions to settle these issues. To guide the instance information in multi -instance datasets, we define an online object patch selection module to select the local patches with the high possibility of containing instance area in the augmented views efficiently during learning. After ob-taining the patches, we present a novel multi-level contrastive learning method considering the instance representation of global-level, local-level and position-level without using negative samples. We conduct extensive experiments with LPCL directly pre-trained on PASCAL VOC and COCO. For PASCAL VOC im-age classification task, our model achieves state-of-the-art 86 . 2% accuracy pre-trained on COCO( +9 . 7% top-1 accuracy compared with baseline BYOL). On object detection, instance segmentation and semantic segmentation task, our proposed model also achieved competitive results compared with other state-of-the-art methods.(c) 2022 Elsevier Ltd. All rights reserved.

Dense Contrastive Learning for Self-Supervised Visual Pre-Training

DenseCL: A Simple Framework for Self-Supervised Dense Visual Pre-Training

LPCL: Localized Prominence Contrastive Learning for Self-Supervised Dense Visual Pre-Training

Point Contrastive Prediction with Semantic Clustering for Self-Supervised Learning on Point Cloud Videos

Dense Semantic Contrast for Self-Supervised Visual Representation Learning

Multi-Level Contrastive Learning for Dense Prediction Task

Space-correlated Contrastive Representation Learning with Multiple Instances.

Boost Supervised Pretraining for Visual Transfer Learning: Implications of Self-Supervised Contrastive Representation Learning.

Align Yourself: Self-supervised Pre-training for Fine-grained Recognition via Saliency Alignment.

HAPiCLR: heuristic attention pixel-level contrastive loss representation learning for self-supervised pretraining

Saliency Guided Contrastive Learning on Scene Images

Heterogeneous Contrastive Learning: Encoding Spatial Information for Compact Visual Representations

DMF-CL: Dense Multi-scale Feature Contrastive Learning for Semantic Segmentation of Remote-Sensing Images

Dense Contrastive Visual-Linguistic Pretraining

CP2: Copy-Paste Contrastive Pretraining for Semantic Segmentation

Resource and data efficient self supervised learning

Contrastive Object-level Pre-training with Spatial Noise Curriculum Learning

Self-supervised Cross-stage Regional Contrastive Learning for Object Detection.

Siamese Prototypical Contrastive Learning

Semi-supervised Contrastive Learning with Similarity Co-calibration.

Self-Supervised Visual Representations Learning by Contrastive Mask Prediction