Abstract:In the scenario of class-incremental learning (CIL), deep neural networks have to adapt their model parameters to non-stationary data distributions, e.g., the emergence of new classes over time. However, CIL models are challenged by the well-known catastrophic forgetting phenomenon. Typical methods such as rehearsal-based ones rely on storing exemplars of old classes to mitigate catastrophic forgetting, which limits real-world applications considering memory resources and privacy issues. In this paper, we propose a novel rehearsal-free CIL approach that learns continually via the synergy between two Complementary Learning Subnetworks. Our approach involves jointly optimizing a plastic CNN feature extractor and an analytical feed-forward classifier. The inaccessibility of historical data is tackled by holistically controlling the parameters of a well-trained model, ensuring that the decision boundary learned fits new classes while retaining recognition of previously learned classes. Specifically, the trainable CNN feature extractor provides task-dependent knowledge separately without interference; and the final classifier integrates task-specific knowledge incrementally for decision-making without forgetting. In each CIL session, it accommodates new tasks by attaching a tiny set of declarative parameters to its backbone, in which only one matrix per task or one vector per class is kept for knowledge retention. Extensive experiments on a variety of task sequences show that our method achieves competitive results against state-of-the-art methods, especially in accuracy gain, memory cost, training efficiency, and task-order robustness. Furthermore, to make the non-growing backbone (i.e., a model with limited network capacity) suffice to train on more incoming tasks, a graceful forgetting implementation on previously learned trivial tasks is empirically investigated.

Efficient Statistical Sampling Adaptation for Exemplar-Free Class Incremental Learning

Efficient Non-Exemplar Class-Incremental Learning with Retrospective Feature Synthesis

FSCIL-EACA: Few-Shot Class-Incremental Learning Network Based on Embedding Augmentation and Classifier Adaptation for Image Classification

Memory Efficient Class-Incremental Learning for Image Classification

DiffClass: Diffusion-Based Class Incremental Learning

Class Incremental Learning with Important and Diverse Memory.

Non-Exemplar Class-Incremental Learning Via Adaptive Old Class Reconstruction

Preserve Knowledge with Auxiliary Feature Extractor for Class Incremental Learning

Class-Incremental Exemplar Compression for Class-Incremental Learning

Few-Shot Incremental Learning with Continually Evolved Classifiers

Task-Adaptive Saliency Guidance for Exemplar-free Class Incremental Learning

PASS++: A Dual Bias Reduction Framework for Non-Exemplar Class-Incremental Learning

Complementary Learning Subnetworks for Parameter-Efficient Class-Incremental Learning

FeCAM: Exploiting the Heterogeneity of Class Distributions in Exemplar-Free Continual Learning

Exemplar-Supported Representation for Effective Class-Incremental Learning

Elastic Feature Consolidation for Cold Start Exemplar-Free Incremental Learning

Improved Continually Evolved Classifiers for Few-Shot Class-Incremental Learning

Exemplar-Free Class Incremental Learning via Incremental Representation

Class Incremental Learning with Self-Supervised Pre-Training and Prototype Learning

FRMM: Feature Reprojection for Exemplar-Free Class-Incremental Learning

Etag: Class-Incremental Learning Via Embedding Distillation and Task-Oriented Generation