Abstract:Real-world datasets follow an imbalanced distribution, which poses significant challenges in rare-category object detection. Recent studies tackle this problem by developing re-weighting and re-sampling methods, that utilise the class frequencies of the dataset. However, these techniques focus solely on the frequency statistics and ignore the distribution of the classes in image space, missing important information. In contrast to them, we propose FRActal CALibration (FRACAL): a novel post-calibration method for long-tailed object detection. FRACAL devises a logit adjustment method that utilises the fractal dimension to estimate how uniformly classes are distributed in image space. During inference, it uses the fractal dimension to inversely downweight the probabilities of uniformly spaced class predictions achieving balance in two axes: between frequent and rare categories, and between uniformly spaced and sparsely spaced classes. FRACAL is a post-processing method and it does not require any training, also it can be combined with many off-the-shelf models such as one-stage sigmoid detectors and two-stage instance segmentation models. FRACAL boosts the rare class performance by up to 8.6% and surpasses all previous methods on LVIS dataset, while showing good generalisation to other datasets such as COCO, V3Det and OpenImages. The code will be released.

What problem does this paper attempt to address?

The paper attempts to address the issue of rare category detection in long-tail distribution datasets. Specifically, real-world datasets often have imbalanced distributions, which pose significant challenges for detecting rare categories. Existing methods mainly handle this imbalance through reweighting or resampling techniques, but these methods often focus only on the frequency statistics of categories, ignoring the distribution of categories in the image space. To overcome this limitation, the authors propose a new method called Fractal Calibration (FRACAL). FRACAL uses fractal dimensions to estimate the uniformity of category distribution in the image space and balances the relationship between frequent and rare categories, as well as uniformly distributed and sparsely distributed categories, by adjusting the prediction probabilities during the inference phase. FRACAL is a post-processing method that does not require additional training and can be combined with various existing detection models, such as single-stage detectors and two-stage instance segmentation models. The main contributions of FRACAL include: 1. Demonstrating for the first time the importance of category-location dependency in long-tail object detection post-calibration. 2. Capturing category-location dependency through a spatially aware long-tail object detection calibration method based on fractal dimensions. 3. Exhibiting excellent performance on multiple detectors and backbone networks, particularly on highly imbalanced datasets (e.g., LVIS) and less imbalanced datasets (e.g., COCO, V3Det, and OpenImages), outperforming existing methods with up to an 8.6% improvement in rare category mean average precision (APm_r). Through these improvements, FRACAL can significantly enhance the detection performance of rare categories while maintaining good performance for frequent categories.

Fractal Calibration for long-tailed object detection

On Model Calibration for Long-Tailed Object Detection and Instance Segmentation

Class Distribution Aware Focal Loss for Long-Tailed Object Detection

The Devil is in Classification: A Simple Framework for Long-tail Object Detection and Instance Segmentation

Classification Calibration for Long-tail Instance Segmentation

Industrial Few-Shot Fractal Object Detection

Adaptive Class Suppression Loss for Long-Tail Object Detection

Balanced Classification: A Unified Framework for Long-Tailed Object Detection

Margin and Average Precision Loss Calibration for Long-Tail Object Detection

Equalized Focal Loss for Dense Long-Tailed Object Detection

Improving Fractal Pre-training

Fractals as Pre-Training Datasets for Anomaly Detection and Localization

Long-Tailed Object Detection Pre-training: Dynamic Rebalancing Contrastive Learning with Dual Reconstruction

Boosting Dense Long-Tailed Object Detection from Data-Centric View

Long-tailed Visual Recognition with Deep Models: A Methodological Survey and Evaluation

Divide and Retain: A Dual-Phase Modeling for Long-Tailed Visual Recognition

FractalAD: A simple industrial anomaly detection method using fractal anomaly generation and backbone knowledge distillation

Long-tail Detection with Effective Class-Margins

Long-tailed Instance Segmentation using Gumbel Optimized Loss

Label-Aware Distribution Calibration for Long-tailed Classification

Margin Calibration for Long-Tailed Visual Recognition