Abstract:Multimedia data has the characteristics of large scale and skewed distribution with a long-tailed shape, which is a challenging imbalance problem faced by deep learning. In long-tailed image instance segmentation, the existing methods deal with this imbalance problem from a single perspective, ignoring the presence of multiple imbalance factors, which results in the limitation of performance. Considering that imbalances exist not only between positive and negative classes, but also between foreground and background subclasses, as well as between hard and easy examples, we argue that the losses of samples should be hierarchically equalized at multi-levels (HEL). In line with this idea, we first propose a focus based hierarchical-equalization loss (FHEL), which employs a class gradient ratio based reweighting mechanism to achieve the balance between classes, and uses a subclass-balance term and a sample-balance term to separately deal with the inter-subclass and inter-sample imbalances. FHEL can improve the performance of long-tailed instance segmentation in an end-to-end manner, avoiding the overfitting risk and manual hard division in the traditional methods. On the basis of FHEL, we further explore the relationship between inter-subclass imbalance and inter-sample imbalance, and propose a constrained-focus based hierarchical-equalization loss (CFHEL) that copes with the imbalances at multi-levels simultaneously with fewer hyperparameters. CFHEL is effective and easy to tune hyperparameters. We conduct extensive experiments on LVIS v1.0 and COCO-LT datasets with different benchmarks. Both FHEL and CFHEL are superior to the existing methods. On LVIS v1.0, with ResNet50 Mask R-CNN, ResNet101Mask R-CNN, ResNeXt101 Mask R-CNN and ResNet101 Cascade Mask R-CNN, CFHEL outperforms its baselines respectively with 19.8%, 18.5%, 21.6% and 21.2% AP% gains, and with 6.7%, 6.6% and 6.5% AP gains, achieving the new state-of-the-arts. On COCO-LT, our CFHEL outperforms the baseline with 13.2% tail AP gains and 3.3% whole AP gains, also achieving the new best performances.

Equalization Loss V2: A New Gradient Balance Approach for Long-tailed Object Detection

The Equalization Losses: Gradient-Driven Training for Long-tailed Object Recognition

Equalized Focal Loss for Dense Long-Tailed Object Detection

Hierarchical Equalization Loss for Long-Tailed Instance Segmentation

Long-Tailed Object Detection Pre-training: Dynamic Rebalancing Contrastive Learning with Dual Reconstruction

Balanced Classification: A Unified Framework for Long-Tailed Object Detection

Adaptive Class Suppression Loss for Long-Tail Object Detection

Long-tailed image recognition through balancing discriminant quality

Balanced complement loss for long-tailed image classification

Long-tail Detection with Effective Class-Margins

Divide and Retain: A Dual-Phase Modeling for Long-Tailed Visual Recognition

GREB: Gradient Re-Balanced Loss for Long-Tailed Multi-Lable Classification

YOLO-Parallel: Positive Gradient Modeling for Long-Tail Remote Sensing Object Detection

An Equalized Margin Loss for Face Recognition.

Boosting Long-tailed Object Detection via Step-wise Learning on Smooth-tail Data

Generalized Focal Loss V2: Learning Reliable Localization Quality Estimation for Dense Object Detection

Balanced Contrastive Learning for Long-Tailed Visual Recognition

Rectify the Regression Bias in Long-Tailed Object Detection

Decoupled Contrastive Learning for Long-Tailed Recognition

Long-tailed Visual Recognition via Gaussian Clouded Logit Adjustment

Balanced Gradient Penalty Improves Deep Long-Tailed Learning