Abstract:This study focuses on incremental learning for image classification, exploring how to reduce catastrophic forgetting of all learned knowledge when access to old data is restricted due to memory or privacy constraints. The challenge of incremental learning lies in achieving an optimal balance between plasticity, the ability to learn new knowledge, and stability, the ability to retain old knowledge. Based on whether the task identifier (task-ID) of an image can be obtained during the test stage, incremental learning for image classifcation is divided into two main paradigms, which are task incremental learning (TIL) and class incremental learning (CIL). The TIL paradigm has access to the task-ID, allowing it to use multiple task-specific classification heads selected based on the task-ID. Consequently, in CIL, where the task-ID is unavailable, TIL methods must predict the task-ID to extend their application to the CIL paradigm. Our previous method for TIL adds task-specific batch normalization and classification heads incrementally. This work extends the method by predicting task-ID through an "unknown" class added to each classification head. The head with the lowest "unknown" probability is selected, enabling task-ID prediction and making the method applicable to CIL. The task-specific batch normalization (BN) modules effectively adjust the distribution of output feature maps across different tasks, enhancing the model's <a class="link-external link-http" href="http://plasticity.Moreover" rel="external noopener nofollow">this http URL</a>, since BN has much fewer parameters compared to convolutional kernels, by only modifying the BN layers as new tasks arrive, the model can effectively manage parameter growth while ensuring stability across tasks. The innovation of this study lies in the first-time introduction of task-specific BN into CIL and verifying the feasibility of extending TIL methods to CIL through task-ID prediction with state-of-the-art performance on multiple datasets.

Revisiting Class-Incremental Learning with Pre-Trained Models: Generalizability and Adaptivity are All You Need

Revisiting Class-Incremental Learning with Pre-Trained Models: Generalizability and Adaptivity are All You Need

Rethinking Class-incremental Learning in the Era of Large Pre-trained Models via Test-Time Adaptation

Expandable Subspace Ensemble for Pre-Trained Model-Based Class-Incremental Learning

Class-Incremental Learning with Strong Pre-trained Models

Class-Incremental Learning: A Survey

Balancing the Causal Effects in Class-Incremental Learning

Integrating Dual Prototypes for Task-Wise Adaption in Pre-Trained Model-Based Class-Incremental Learning

Multi-granularity knowledge distillation and prototype consistency regularization for class-incremental learning

Reformulating Classification As Image-Class Matching for Class Incremental Learning

Complementary Learning Subnetworks for Parameter-Efficient Class-Incremental Learning

GFR: Generic Feature Representations for Class Incremental Learning

Non-Exemplar Class-Incremental Learning Via Adaptive Old Class Reconstruction

PASS++: A Dual Bias Reduction Framework for Non-Exemplar Class-Incremental Learning

Class-Incremental Exemplar Compression for Class-Incremental Learning

TaE: Task-aware Expandable Representation for Long Tail Class Incremental Learning

Towards Diverse Evaluation of Class Incremental Learning: A Representation Learning Perspective

Towards Non-Exemplar Semi-Supervised Class-Incremental Learning

DiffClass: Diffusion-Based Class Incremental Learning

Class Incremental Learning with Task-Specific Batch Normalization and Out-of-Distribution Detection