Abstract:We propose an embarrassingly simple method -- instance-aware repeat factor sampling (IRFS) to address the problem of imbalanced data in long-tailed object detection. Imbalanced datasets in real-world object detection often suffer from a large disparity in the number of instances for each class. To improve the generalization performance of object detection models on rare classes, various data sampling techniques have been proposed. Repeat factor sampling (RFS) has shown promise due to its simplicity and effectiveness. Despite its efficiency, RFS completely neglects the instance counts and solely relies on the image count during re-sampling process. However, instance count may immensely vary for different classes with similar image counts. Such variation highlights the importance of both image and instance for addressing the long-tail distributions. Thus, we propose IRFS which unifies instance and image counts for the re-sampling process to be aware of different perspectives of the imbalance in long-tailed datasets. Our method shows promising results on the challenging LVIS v1.0 benchmark dataset over various architectures and backbones, demonstrating their effectiveness in improving the performance of object detection models on rare classes with a relative $+50\%$ average precision (AP) improvement over counterpart RFS. IRFS can serve as a strong baseline and be easily incorporated into existing long-tailed frameworks.

What problem does this paper attempt to address?

The paper primarily addresses the issue of object detection in long-tailed distribution datasets by proposing a new resampling method—Instance-Aware Repeat Factor Sampling (IRFS)—to overcome the shortcomings of existing methods in handling rare categories. The paper points out that in real-world datasets, the number of instances per category varies significantly, and this imbalance negatively impacts the generalization performance of object detection models, especially for rare categories. To improve the detection performance of rare categories, researchers have proposed various data sampling techniques, among which Repeat Factor Sampling (RFS) has gained attention due to its simplicity and effectiveness. However, RFS only resamples based on the number of images, ignoring the significant differences in the number of instances of different categories within the same number of images. To address this issue, the authors propose the IRFS method, which combines the number of images and the number of instances to determine the repeat factor for each category. Specifically, IRFS utilizes the information on the number of instances in an image, along with the number of images, for the resampling process, making the sampling strategy more comprehensively consider the imbalance in the dataset. The experimental section demonstrates the effectiveness of IRFS on the LVIS v1.0 benchmark dataset, which contains a large number of long-tailed distribution categories. The results show that compared to RFS, IRFS achieves a significant improvement in the average precision (AP) of rare categories, while the performance on common and frequent categories does not decline and even improves. Additionally, the authors explore the potential of combining IRFS with other loss functions (such as ECM loss), further enhancing the overall performance of the model. In summary, IRFS effectively addresses the poor detection performance of rare categories in long-tailed distribution datasets by comprehensively considering the number of images and instances, providing a simple and effective solution for handling such issues.

Instance-Aware Repeat Factor Sampling for Long-Tailed Object Detection

Boosting Dense Long-Tailed Object Detection from Data-Centric View

DIM: Long-tailed Object Detection and Instance Segmentation via Dynamic Instance Memory

Fractal Calibration for long-tailed object detection

Reference Twice: A Simple and Unified Baseline for Few-Shot Instance Segmentation

Forest R-CNN: Large-Vocabulary Long-Tailed Object Detection and Instance Segmentation

Attention Erasing and Instance Sampling for Weakly Supervised Object Detection

Feature Re-Balancing for Long-Tailed Visual Recognition.

Adaptive Class Suppression Loss for Long-Tail Object Detection

Long-Tailed Object Detection for Multimodal Remote Sensing Images

Rectify the Regression Bias in Long-Tailed Object Detection

Multi-Scale Positive Sample Refinement for Few-Shot Object Detection

ForestDet: Large-Vocabulary Long-Tailed Object Detection and Instance Segmentation

Few-Shot Object Detection in Remote-Sensing Images via Label-Consistent Classifier and Gradual Regression

InfRS: Incremental Few-Shot Object Detection in Remote Sensing Images

Long-tail Detection with Effective Class-Margins

Improved Region Proposal Network for Enhanced Few-Shot Object Detection

Equalized Focal Loss for Dense Long-Tailed Object Detection

Balanced Classification: A Unified Framework for Long-Tailed Object Detection

Inverse Image Frequency for Long-tailed Image Recognition