Abstract:Deep learning has achieved remarkable progress for visual recognition on large-scale balanced datasets but still performs poorly on real-world long-tailed data. Previous methods often adopt class re-balanced training strategies to effectively alleviate the imbalance issue, but might be a risk of over-fitting tail classes. The recent decoupling method overcomes over-fitting issues by using a multi-stage training scheme, yet, it is still incapable of capturing tail class information in the feature learning stage. In this paper, we show that soft label can serve as a powerful solution to incorporate label correlation into a multi-stage training scheme for long-tailed recognition. The intrinsic relation between classes embodied by soft labels turns out to be helpful for long-tailed recognition by transferring knowledge from head to tail classes. Specifically, we propose a conceptually simple yet particularly effective multi-stage training scheme, termed as Self Supervised to Distillation (SSD). This scheme is composed of two parts. First, we introduce a self-distillation framework for long-tailed recognition, which can mine the label relation automatically. Second, we present a new distillation label generation module guided by self-supervision. The distilled labels integrate information from both label and data domains that can model long-tailed distribution effectively. We conduct extensive experiments and our method achieves the state-of-the-art results on three long-tailed recognition benchmarks: ImageNet-LT, CIFAR100-LT and iNaturalist 2018. Our SSD outperforms the strong LWS baseline by from 2.7% to 4.5% on various datasets.

Residual diverse ensemble for long-tailed multi-label text classification

A Dual-Branch Learning Model with Gradient-Balanced Loss for Long-Tailed Multi-Label Text Classification

Improving Tail Label Prediction for Extreme Multi-label Learning

Towards Robust Prediction on Tail Labels

Does Tail Label Help for Large-Scale Multi-Label Learning

Learning for Tail Label Data: A Label-Specific Feature Approach.

Label-Specific Feature Augmentation for Long-Tailed Multi-Label Text Classification

On the Value of Head Labels in Multi-Label Text Classification

Dynamic Ensemble Learning for Multi-label Classification

Learning Label-Adaptive Representation for Large-Scale Multi-Label Text Classification

Adaptively Weighted Copy-Decoupling Resampling Strategy for Long-Tailed Multi-label Classification

Decoupling Representation and Classifier for Long-Tailed Recognition

Robust Long-Tailed Learning under Label Noise

A dual-branch model with inter- and intra-branch contrastive loss for long-tailed recognition

Three Heads Are Better Than One: Complementary Experts for Long-Tailed Semi-supervised Learning

Distribution-Balanced Loss for Multi-label Classification in Long-Tailed Datasets

A Debiased Nearest Neighbors Framework for Multi-Label Text Classification

Text-Guided Diverse Image Synthesis for Long-Tailed Remote Sensing Object Classification

Self Supervision to Distillation for Long-Tailed Visual Recognition

Addressing Long-Tail Noisy Label Learning Problems: a Two-Stage Solution with Label Refurbishment Considering Label Rarity

Adaptive Embedding and Distribution Re-margin for Long-Tail Recognition