Rethinking Few-shot Class-incremental Learning: Learning from Yourself

Yu-Ming Tang,Yi-Xing Peng,Jingke Meng,Wei-Shi Zheng

2024-07-10

Abstract:Few-shot class-incremental learning (FSCIL) aims to learn sequential classes with limited samples in a few-shot fashion. Inherited from the classical class-incremental learning setting, the popular benchmark of FSCIL uses averaged accuracy (aAcc) and last-task averaged accuracy (lAcc) as the evaluation metrics. However, we reveal that such evaluation metrics may not provide adequate emphasis on the novel class performance, and the continual learning ability of FSCIL methods could be ignored under this benchmark. In this work, as a complement to existing metrics, we offer a new metric called generalized average accuracy (gAcc) which is designed to provide an extra equitable evaluation by incorporating different perspectives of the performance under the guidance of a parameter $\alpha$. We also present an overall metric in the form of the area under the curve (AUC) along the $\alpha$. Under the guidance of gAcc, we release the potential of intermediate features of the vision transformers to boost the novel-class performance. Taking information from intermediate layers which are less class-specific and more generalizable, we manage to rectify the final features, leading to a more generalizable transformer-based FSCIL framework. Without complex network designs or cumbersome training procedures, our method outperforms existing FSCIL methods at aAcc and gAcc on three datasets. See codes at <a class="link-external link-https" href="https://github.com/iSEE-Laboratory/Revisting_FSCIL" rel="external noopener nofollow">this https URL</a>

Computer Vision and Pattern Recognition

What problem does this paper attempt to address?

The problem that this paper attempts to solve is that in few - shot class - incremental learning (FSCIL), the existing evaluation metrics may not fully emphasize the performance of new classes and may overlook the continuous learning ability of FSCIL methods. Specifically:

1. **Limitations of existing evaluation metrics**: Traditional FSCIL benchmarks use average accuracy (aAcc) and the average accuracy of the last task (lAcc) as evaluation metrics. However, these metrics may focus too much on the performance of base classes and ignore the changes in the performance of new classes. Since FSCIL benchmarks usually set most classes as base tasks, the existing aAcc or lAcc metrics are mainly affected by the performance of base classes, which leads to the fact that methods that perform poorly on new classes may also obtain high evaluation scores because of their excellent performance on base classes.

2. **Proposed new metrics**: To overcome the above limitations, the author proposes a new evaluation metric - Generalized Average Accuracy (gAcc). gAcc provides a more balanced and detailed evaluation perspective by introducing the parameter α to balance the performance weights of base classes and new classes. In addition, a comprehensive evaluation metric - Area Under the Curve (AUC) is also proposed to comprehensively evaluate the performance of the model under different α values.

3. **Utilizing the potential of vision transformers**: The author also finds that the intermediate - layer features of Vision Transformer (ViT) have good generalization ability for dealing with new classes. Based on this observation, the author proposes a lightweight Feature Rectification (FR) module. This module modifies the final features by combining intermediate - layer features, thereby improving the performance of new classes while maintaining the performance of base classes.

In summary, the main objective of this paper is to improve the evaluation metrics of FSCIL so that they can more fairly evaluate the performance of models on new classes, and propose a method of using the intermediate - layer features of vision transformers to improve the performance of new classes.