NCL++: Nested Collaborative Learning for Long-Tailed Visual Recognition

Zichang Tan,Jun Li,Jinhao Du,Jun Wan,Zhen Lei,Guodong Guo

2024-07-18

Abstract:Long-tailed visual recognition has received increasing attention in recent years. Due to the extremely imbalanced data distribution in long-tailed learning, the learning process shows great uncertainties. For example, the predictions of different experts on the same image vary remarkably despite the same training settings. To alleviate the uncertainty, we propose a Nested Collaborative Learning (NCL++) which tackles the long-tailed learning problem by a collaborative learning. To be specific, the collaborative learning consists of two folds, namely inter-expert collaborative learning (InterCL) and intra-expert collaborative learning (IntraCL). In-terCL learns multiple experts collaboratively and concurrently, aiming to transfer the knowledge among different experts. IntraCL is similar to InterCL, but it aims to conduct the collaborative learning on multiple augmented copies of the same image within the single expert. To achieve the collaborative learning in long-tailed learning, the balanced online distillation is proposed to force the consistent predictions among different experts and augmented copies, which reduces the learning uncertainties. Moreover, in order to improve the meticulous distinguishing ability on the confusing categories, we further propose a Hard Category Mining (HCM), which selects the negative categories with high predicted scores as the hard categories. Then, the collaborative learning is formulated in a nested way, in which the learning is conducted on not just all categories from a full perspective but some hard categories from a partial perspective. Extensive experiments manifest the superiority of our method with outperforming the state-of-the-art whether with using a single model or an ensemble. The code will be publicly released.

Computer Vision and Pattern Recognition

What problem does this paper attempt to address?

The problem that this paper attempts to solve is the uncertainty in the learning process due to the extremely unbalanced data distribution in long - tailed visual recognition. Specifically, the characteristic of data distribution in long - tailed learning is that a few head classes occupy the majority of data, while many tail classes have only a small number of samples. This data distribution causes the model to be easily dominated by a few head classes, and the prediction accuracy for many other tail classes is usually low. This long - tailed characteristic not only challenges deep visual recognition, but also greatly hinders the practical application of deep models. To alleviate this problem, the paper proposes a Nested Collaborative Learning (NCL++) method, which reduces the uncertainty in the learning process through collaborative learning among multiple experts and improves the model's recognition ability on tail classes.