Abstract:Although neural networks have been used extensively in pattern recognition scenarios, the pre-acquisition of datasets is still challenging. In most pattern recognition areas, preparing a training dataset that covers all data domains is difficult. Incremental learning was proposed to update neural networks in an online manner, but the catastrophic forgetting issue still needs to be studied. Class-incremental learning is one of the most challenging incremental learning contexts; it trains a unified model to classify all incrementally arriving classes learned thus far equally. Prior studies on class-incremental learning favor model stability over plasticity to realize old knowledge reservation and prevent catastrophic forgetting. Consequently, the model's plasticity is omitted, leading to difficult generalization on new data. We propose a novel distillation-based method named Hyper-feature Aggregation and Relaxed Distillation (HARD) to realize balanced optimization of old and new knowledge. The aggregation of features is proposed to capture the global semantics while maintaining the diversity of the feature distribution after promoting representations of exemplars to higher dimensions. The proposed algorithm also introduces a relaxed restriction in the hyper-feature space to conditions the hyper-feature space through a normalized comparison of the relation matrices. Following generalization on more classes, the model is encouraged to rebuild the feature distribution when meeting new classes and to fine-tune the feature space to realize more distinct interclass boundaries. Extensive experiments were conducted on two benchmark datasets, and consistent improvements under diverse experimental settings demonstrated the effectiveness of the proposed approach.

Hybrid Rotation Self-Supervision and Feature Space Normalization for Class Incremental Learning

Class Incremental Learning with Self-Supervised Pre-Training and Prototype Learning

Hyper-feature aggregation and relaxed distillation for class incremental learning

Incremental Classification of Remote Sensing Images with Feature Pyramid and Class Hierarchy

Self-Organizing Pathway Expansion for Non-Exemplar Class-Incremental Learning

Efficient Non-Exemplar Class-Incremental Learning with Retrospective Feature Synthesis

Combining Past, Present and Future: A Self-Supervised Approach for Class Incremental Learning.

Class-Incremental Recognition of Objects in Remote Sensing Images with Dynamic Hybrid Exemplar Selection

Self-Sustaining Representation Expansion for Non-Exemplar Class-Incremental Learning

Class Incremental Learning Based on Local Structure Constraints in Feature Space

Rotation Augmented Distillation for Exemplar-Free Class Incremental Learning with Detailed Analysis

Representation Robustness and Feature Expansion for Exemplar-Free Class-Incremental Learning

More Task-Balanced Class-Incremental Learning

Large Scale Incremental Learning

Preserve Knowledge with Auxiliary Feature Extractor for Class Incremental Learning

Non-exemplar Class-incremental Learning by Random Auxiliary Classes Augmentation and Mixed Features

A class-incremental learning approach for learning feature-compatible embeddings

ICFD: an Incremental Learning Method Based on Data Feature Distribution.

Multi-feature Space Similarity Supplement for Few-Shot Class Incremental Learning

Self-Paced Imbalance Rectification for Class Incremental Learning