Abstract:Real-world data often follow a long-tailed distribution as the frequency of each class is typically different. For example, a dataset can have a large number of under-represented classes and a few classes with more than sufficient data. However, a model to represent the dataset is usually expected to have reasonably homogeneous performances across classes. Introducing class-balanced loss and advanced methods on data re-sampling and augmentation are among the best practices to alleviate the data imbalance problem. However, the other part of the problem about the under-represented classes will have to rely on additional knowledge to recover the missing information.
In this work, we present a novel approach to address the long-tailed problem by augmenting the under-represented classes in the feature space with the features learned from the classes with ample samples. In particular, we decompose the features of each class into a class-generic component and a class-specific component using class activation maps. Novel samples of under-represented classes are then generated on the fly during training stages by fusing the class-specific features from the under-represented classes with the class-generic features from confusing classes. Our results on different datasets such as iNaturalist, ImageNet-LT, Places-LT and a long-tailed version of CIFAR have shown the state of the art performances.
What problem does this paper attempt to address?
The problem that this paper attempts to solve is how to improve the model's recognition performance for the minority classes (i.e., the classes with a small number of samples) in classification tasks under long - tailed data distributions. Specifically, datasets in the real world often exhibit the characteristics of long - tailed distributions, that is, the number of samples in some classes is very small, while the number of samples in other classes is relatively large. This unbalanced data distribution will cause the model to be biased towards the majority classes during the training process, thus affecting the recognition effect on the minority classes. To alleviate this problem, the paper proposes a new method to improve the overall performance of the model by enhancing the representation of the minority classes in the feature space.
### Main contributions of the paper
1. **Feature space enhancement**: The paper proposes a novel method to generate new samples in the feature space by fusing the specific features of the minority classes and the general features of the majority classes, thereby enhancing the representation of the minority classes.
2. **Application of Class Activation Map (CAM)**: Class activation maps are used to decompose the features of each class into class - specific features and class - general features, so as to perform feature enhancement more effectively.
3. **Two - stage training scheme**: In the first stage, the basic model is trained using samples of all classes, and in the second stage, enhanced samples are generated online to fine - tune the model, with special attention to the performance improvement of the minority classes.
### Specific methods
1. **Initial feature learning**:
- In the first stage, the feature sub - network and the basic classifier are trained using samples of all classes.
- A network architecture containing a single fully - connected layer is selected as the final classifier in order to calculate the class activation map.
2. **Feature space enhancement**:
- Using the pre - trained feature sub - network and classifier, new samples are generated in the feature space by mixing the class - specific features of a given minority class and the class - general features of the majority class.
- The class that is most "confused" with the minority class is selected as the source of the class - general features to better restore the distribution of the minority class.
3. **Fine - tuning**:
- In the second stage, the model trained in the first stage is fine - tuned by generating enhanced samples online to further improve the recognition performance of the minority class.
- Each batch contains samples randomly drawn from the minority class and the majority class to maintain the balance of the data distribution.
### Experimental results
The paper conducted experiments on multiple datasets, including the artificially generated long - tailed CIFAR dataset, ImageNet - LT, Places - LT, and the real - world long - tailed iNaturalist dataset. The experimental results show that the proposed method achieves better performance than other mainstream methods on these datasets, especially with a significant improvement in the recognition performance of the minority classes.
### Conclusion
The method proposed in the paper effectively solves the imbalance problem in classification tasks under long - tailed data distributions by enhancing the representation of the minority classes in the feature space, and improves the overall performance of the model, especially its performance on the minority classes.