Abstract:The goal of contrastive learning based pre-training is to leverage large quantities of unlabeled data to produce a model that can be readily adapted downstream. Current approaches revolve around solving an image discrimination task: given an anchor image, an augmented counterpart of that image, and some other images, the model must produce representations such that the distance between the anchor and its counterpart is small, and the distances between the anchor and the other images are large. There are two significant problems with this approach: (i) by contrasting representations at the image-level, it is hard to generate detailed object-sensitive features that are beneficial to downstream object-level tasks such as instance segmentation; (ii) the augmentation strategy of producing an augmented counterpart is fixed, making learning less effective at the later stages of pre-training. In this work, we introduce Curricular Contrastive Object-level Pre-training (CCOP) to tackle these problems: (i) we use selective search to find rough object regions and use them to build an inter-image object-level contrastive loss and an intra-image object-level discrimination loss into our pre-training objective; (ii) we present a curriculum learning mechanism that adaptively augments the generated regions, which allows the model to consistently acquire a useful learning signal, even in the later stages of pre-training. Our experiments show that our approach improves on the MoCo v2 baseline by a large margin on multiple object-level tasks when pre-training on multi-object scene image datasets. Code is available at <a class="link-external link-https" href="https://github.com/ChenhongyiYang/CCOP" rel="external noopener nofollow">this https URL</a>.

Contrastive Pre-training for Imbalanced Corporate Credit Ratings

Adversarial Semi-supervised Learning for Corporate Credit Ratings

Contrastive-weighted Self-Supervised Model for Long-Tailed Data Classification with Vision Transformer Augmented

Contrastive Learning for Conversion Rate Prediction

Pre-trained Online Contrastive Learning for Insurance Fraud Detection

Boost Supervised Pretraining for Visual Transfer Learning: Implications of Self-Supervised Contrastive Representation Learning.

Toward High Quality Facial Representation Learning

Corporate Credit Rating: A Survey

Multi-task Envisioning Transformer-based Autoencoder for Corporate Credit Rating Migration Early Prediction

Contrastive Object-level Pre-training with Spatial Noise Curriculum Learning

CL4CTR: A Contrastive Learning Framework for CTR Prediction.

Self-Supervised Pre-Training Based on Contrastive Complementary Masking for Semi-Supervised Cardiac Image Segmentation

Self-Damaging Contrastive Learning

An adaptive fault diagnosis framework under class-imbalanced conditions based on contrastive augmented deep reinforcement learning

Cost-sensitive Classifiers in Credit Rating: A Comparative Study on P2P Lending

A novel SSA-CatBoost machine learning model for credit rating

DenseCL: A Simple Framework for Self-Supervised Dense Visual Pre-Training

CAPT: Contrastive Pre-Training for LearningDenoised Sequence Representations

Scene Text Recognition with Self-supervised Contrastive Predictive Coding

BaCon: Boosting Imbalanced Semi-supervised Learning via Balanced Feature-Level Contrastive Learning

A ResNet-LSTM Based Credit Scoring Approach for Imbalanced Data