Abstract:Despite the outstanding performance in many individual tasks, deep neural networks suffer from catastrophic forgetting when learning from continuous data streams in real-world scenarios. Current Non-Exemplar Class-Incremental Learning (NECIL) methods mitigate forgetting by storing a single prototype per class, which serves to inject previous information when sequentially learning new classes. However, these stored prototypes or their augmented variants often fail to simultaneously capture spatial distribution diversity and precision needed for representing old classes. Moreover, as the model acquires new knowledge, these prototypes gradually become outdated, making them less effective. To overcome these limitations, we propose a more efficient NECIL method that replaces prototypes with synthesized retrospective features for old classes. Specifically, we model each old class's feature space using a multivariate Gaussian distribution and generate deep representations by sampling from high-likelihood regions. Additionally, we introduce a similarity-based feature compensation mechanism that integrates generated old class features with similar new class features to synthesize robust retrospective representations. These retrospective features are then incorporated into our incremental learning framework to preserve the decision boundaries of previous classes while learning new ones. Extensive experiments on CIFAR-100, TinyImageNet, and ImageNet-Subset demonstrate that our method significantly improves the efficiency of non-exemplar class-incremental learning and achieves state-of-the-art performance.

Incremental Concept Formation over Visual Images Without Catastrophic Forgetting

Cobweb: An Incremental and Hierarchical Model of Human-Like Category Learning

Incremental Concept Learning via Online Generative Memory Recall

Incremental and Data-Efficient Concept Formation to Support Masked Word Prediction

SynapNet: A Complementary Learning System Inspired Algorithm With Real-Time Application in Multimodal Perception

Catastrophic forgetting: still a problem for DNNs

Incremental Learning of Object Detectors without Catastrophic Forgetting

Progressive Learning without Forgetting

Measuring Catastrophic Forgetting in Neural Networks

Incremental Learning Without Looking Back: a Neural Connection Relocation Approach

Explaining How Deep Neural Networks Forget by Deep Visualization

A Methodology-Oriented Study of Catastrophic Forgetting in Incremental Deep Neural Networks

Learn to Grow: A Continual Structure Learning Framework for Overcoming Catastrophic Forgetting

Incremental Learning Using Conditional Adversarial Networks

A Bio-Inspired Incremental Learning Architecture for Applied Perceptual Problems

A Robust and Anti-Forgettiable Model for Class-Incremental Learning

Efficient Non-Exemplar Class-Incremental Learning with Retrospective Feature Synthesis

Incremental class learning using variational autoencoders with similarity learning

Wide Neural Networks Forget Less Catastrophically

COBRA: A Continual Learning Approach to Vision-Brain Understanding

Lifelong Neural Predictive Coding: Learning Cumulatively Online without Forgetting