Abstract:Existing methods based on transfer learning leverage auxiliary information to help tail generalization and improve the performance of the tail classes. However, they cannot fully exploit the relationships between auxiliary information and tail classes and bring irrelevant knowledge to the tail classes. To solve this problem, we propose a hierarchical CNN with knowledge complementation, which regards hierarchical relationships as auxiliary information and transfers relevant knowledge to tail classes. First, we integrate semantics and clustering relationships as hierarchical knowledge into the CNN to guide feature learning. Then, we design a complementary strategy to jointly exploit the two types of knowledge, where semantic knowledge acts as a prior dependence and clustering knowledge reduces the negative information caused by excessive semantic dependence (i.e., semantic gaps). In this way, the CNN facilitates the utilization of the two complementary hierarchical relationships and transfers useful knowledge to tail data to improve long-tailed classification accuracy. Experimental results on public benchmarks show that the proposed model outperforms existing methods. In particular, our model improves accuracy by 3.46% compared with the second-best method on the long-tailed tieredImageNet dataset.

What problem does this paper attempt to address?

The paper primarily proposes a new solution to the long-tail classification problem. Specifically, it focuses on handling the extremely imbalanced class sample distribution in deep learning models, particularly Convolutional Neural Networks (CNNs), where the number of samples in a few head classes far exceeds that in a large number of tail classes. This imbalance can lead the model to accurately predict head classes while ignoring tail classes, thereby affecting overall classification performance. To address the above issue, the authors propose a Hierarchical Convolutional Neural Network (HCKC) that leverages the concept of knowledge completion. The main contributions of this approach are: 1. **Exploration of Clustering and Semantic Relationships**: By establishing hierarchical superclasses (including both semantic-based and clustering-based superclasses), the method aids in feature learning for tail classes. These hierarchical structures not only provide a physical explanation of the relationships between classes but also serve as auxiliary information to improve the performance of tail classes. 2. **Addressing the Semantic Gap Problem**: It is demonstrated that clustering relationships can help alleviate the semantic gap problem present in hierarchical relationships, guiding the long-tail classification task from both data and label perspectives. 3. **Proposing a Complementary Strategy**: By combining semantic and clustering knowledge, the method helps tail classes learn more diverse knowledge, thereby enhancing feature diversity. This includes first transferring semantic knowledge to tail data to learn relevant features, and then using clustering knowledge to mitigate the negative impact caused by the semantic gap. In summary, the paper aims to improve the performance of tail classes in long-tail classification problems by introducing a hierarchical convolutional neural network and a knowledge completion strategy. Especially in deep learning models, it utilizes semantic and clustering information to enhance the generalization ability of tail classes. Experimental results show that the proposed model outperforms existing methods on multiple public benchmark datasets.

Hierarchical Convolutional Neural Network with Knowledge Complementation for Long-Tailed Classification

Multi-task Convolutional Neural Network with Coarse-to-fine Knowledge Transfer for Long-Tailed Classification

Hierarchical long-tailed classification based on multi-granularity knowledge transfer driven by multi-scale feature fusion

Hierarchical Auxiliary Learning

Coarse-to-fine Knowledge Transfer Based Long-Tailed Classification Via Bilateral-Sampling Network

Hierarchical Classification of Data with Long-Tailed Distributions Via Global and Local Granulation

Hierarchical classification based on coarse- to fine-grained knowledge transfer

Semantic Guided Level-Category Hybrid Prediction Network for Hierarchical Image Classification.

Hierarchical Bilinear Convolutional Neural Network for Image Classification.

Dynamic collaborative learning with heterogeneous knowledge transfer for long-tailed visual recognition

Cumulative Dual-Branch Network Framework for Long-Tailed Multi-Class Classification

Hierarchical Transfer Convolutional Neural Networks for Image Classification

Relieving the Incompatibility of Network Representation and Classification for Long-Tailed Data Distribution

Deep Multi-task Learning for Hierarchical Classification

Learning with Hierarchical Complement Objective

Long-Tailed Recognition by Hierarchical Rebalancing Dual-Classifier

Two-level Hierarchical Feature Learning for Image Classification

Balanced complement loss for long-tailed image classification

Hybrid ResNet Based on Joint Basic and Attention Modules for Long-Tailed Classification

Hierarchical Convolutional Neural Network Via Hierarchical Cluster Validity Based Visual Tree Learning

Exploring the auxiliary learning for long-tailed visual recognition