Abstract:Long-tailed distributions frequently emerge in real-world data, where a large number of minority categories contain a limited number of samples. Such imbalance issue considerably impairs the performance of standard supervised learning algorithms, which are mainly designed for balanced training sets. Recent investigations have revealed that supervised contrastive learning exhibits promising potential in alleviating the data imbalance. However, the performance of supervised contrastive learning is plagued by an inherent challenge: it necessitates sufficiently large batches of training data to construct contrastive pairs that cover all categories, yet this requirement is difficult to meet in the context of class-imbalanced data. To overcome this obstacle, we propose a novel probabilistic contrastive (ProCo) learning algorithm that estimates the data distribution of the samples from each class in the feature space, and samples contrastive pairs accordingly. In fact, estimating the distributions of all classes using features in a small batch, particularly for imbalanced data, is not feasible. Our key idea is to introduce a reasonable and simple assumption that the normalized features in contrastive learning follow a mixture of von Mises-Fisher (vMF) distributions on unit space, which brings two-fold benefits. First, the distribution parameters can be estimated using only the first sample moment, which can be efficiently computed in an online manner across different batches. Second, based on the estimated distribution, the vMF distribution allows us to sample an infinite number of contrastive pairs and derive a closed form of the expected contrastive loss for efficient optimization. Our code is available at <a class="link-external link-https" href="https://github.com/LeapLabTHU/ProCo" rel="external noopener nofollow">this https URL</a>.

An imbalanced contrastive classification method via similarity comparison within sample-neighbors with adaptive generation coefficient

Similar classes latent distribution modelling-based oversampling method for imbalanced image classification

Similarity-Dissimilarity Loss with Supervised Contrastive Learning for Multi-label Classification

A hybrid sampling method for highly imbalanced and overlapped data classification with complex distribution

Probabilistic Contrastive Learning for Long-Tailed Visual Recognition

Contrastive Learning Based Hybrid Networks for Long-Tailed Image Classification

A Dissimilarity-Based Imbalance Data Classification Algorithm

Solving Data Imbalance in Text Classification with Constructing Contrastive Samples

$\mathbb{X}$-Sample Contrastive Loss: Improving Contrastive Learning with Sample Similarity Graphs

Class-Aware Contrastive Optimization for Imbalanced Text Classification

Bt-Vmf Contrastive and Collaborative Learning for Long-Tailed Visual Recognition

Balanced Contrastive Learning for Long-Tailed Visual Recognition

MMD-encouraging convolutional autoencoder: a novel classification algorithm for imbalanced data

Adaptive Multi-head Contrastive Learning

VCOS: A Novel Synergistic Oversampling Algorithm in Binary Imbalance Classification

BaCon: Boosting Imbalanced Semi-supervised Learning via Balanced Feature-Level Contrastive Learning

A conditional variational autoencoder based self-transferred algorithm for imbalanced classification

A Positive Sample Enhancement Algorithm with Fuzzy Nearest Neighbor Hybridization for Imbalance Data

Center-wise Local Image Mixture for Contrastive Representation Learning.

Understanding Contrastive Learning via Distributionally Robust Optimization