Abstract:Continual Learning is an unresolved challenge, whose relevance increases when considering modern applications. Unlike the human brain, trained deep neural networks suffer from a phenomenon called Catastrophic Forgetting, where they progressively lose previously acquired knowledge upon learning new tasks. To mitigate this problem, numerous methods have been developed, many relying on replaying past exemplars during new task training. However, as the memory allocated for replay decreases, the effectiveness of these approaches diminishes. On the other hand, maintaining a large memory for the purpose of replay is inefficient and often impractical. Here we introduce TEAL, a novel approach to populate the memory with exemplars, that can be integrated with various experience-replay methods and significantly enhance their performance on small memory buffers. We show that TEAL improves the average accuracy of the SOTA method XDER as well as ER and ER-ACE on several image recognition benchmarks, with a small memory buffer of 1-3 exemplars per class in the final task. This confirms the hypothesis that when memory is scarce, it is best to prioritize the most typical data.

What problem does this paper attempt to address?

The problem that this paper attempts to solve is how to effectively select representative samples to alleviate catastrophic forgetting in class - incremental learning (CIL) when the memory buffer is very small. Specifically, the paper proposes a new selection strategy - TEAL (Typicality Election Approach to continual Learning) for populating a small memory buffer in the experience replay (ER) method, thereby significantly improving the performance of the model when dealing with new tasks, especially when memory resources are limited. ### Background and Motivation - **Problem Background**: Continual learning is an unsolved challenge and is particularly important in modern applications. Traditional deep neural networks gradually forget previously learned knowledge when learning new tasks, a phenomenon known as catastrophic forgetting. - **Limitations of Existing Methods**: To alleviate this problem, many existing methods rely on replaying past examples when training new tasks. However, as the memory allocated for replay decreases, the effectiveness of these methods also decreases. On the other hand, maintaining a large memory for replay is inefficient and impractical. ### Core Contributions of the Paper - **TEAL Method**: TEAL is a new method for selecting representative samples when the memory buffer is very small. This method improves the performance of the model with a small memory buffer by selecting typical and diverse samples. - **Experimental Verification**: The paper verifies the effectiveness of TEAL in a small memory buffer (1 - 3 examples per class) through multiple image recognition benchmark tests. The results show that TEAL can significantly improve the average accuracy of existing methods such as XDER, ER, and ER - ACE. ### Method Overview - **Typicality Definition**: TEAL selects samples based on the typicality of points. Typicality is defined as: \[ \text{Typicality}(x)=\left(\frac{1}{K} \sum_{x_{i} \in K\text{-NN}(x)}\|x - x_{i}\|^{2}\right)^{-1} \] where \(K\) is a fixed number of nearest neighbors. - **Selection Process**: TEAL selects samples through the following steps: 1. **Clustering**: Divide the labeled points into several clusters to ensure that typical samples are selected from different regions. 2. **Typicality**: Select the most typical samples from each cluster and rank them according to the typicality score. ### Experimental Results - **Performance Improvement**: On multiple datasets such as Split CIFAR - 100, Split tinyImageNet, and Split CUB - 200, TEAL significantly improves the performance of existing methods, especially in a small memory buffer. - **Comparison with Existing Methods**: TEAL outperforms the random selection and Herding methods in most cases, especially when the memory buffer is small. ### Conclusion The paper proposes a new selection strategy, TEAL, specifically for the class - incremental learning problem with a small memory buffer. The experimental results show that TEAL can significantly improve the performance of the model, especially when memory resources are limited. Future work will further explore the applicability of TEAL under different conditions and potential improvement directions.

TEAL: New Selection Strategy for Small Buffers in Experience Replay Class Incremental Learning

Dual Memory Model for Experience-Once Task-Incremental Lifelong Learning.

The Effectiveness of Memory Replay in Large Scale Continual Learning

Replay-enhanced Continual Reinforcement Learning

Improving Replay Sample Selection and Storage for Less Forgetting in Continual Learning

M EMORY R EPLAY WITH D ATA C OMPRESSION FOR C ONTINUAL L EARNING

Continual Learning with Strong Experience Replay

Hybrid Memory Replay: Blending Real and Distilled Data for Class Incremental Learning

Saliency Guided Experience Packing for Replay in Continual Learning

Adaptive Memory Replay for Continual Learning

Continual Learning: Less Forgetting, More OOD Generalization via Adaptive Contrastive Replay

Watch Your Step: Optimal Retrieval for Continual Learning at Scale

May the Forgetting Be with You: Alternate Replay for Learning with Noisy Labels

AdaER: An Adaptive Experience Replay Approach for Continual Lifelong Learning

Continual Recognition with Adaptive Memory Update.

Map-based Experience Replay: A Memory-Efficient Solution to Catastrophic Forgetting in Reinforcement Learning

Relational Experience Replay: Continual Learning by Adaptively Tuning Task-wise Relationship

Principal Gradient Direction and Confidence Reservoir Sampling for Continual Learning

Saliency-Guided Hidden Associative Replay for Continual Learning

An Efficient Replay for Class-Incremental Learning with Pre-trained Models

CORE: Mitigating Catastrophic Forgetting in Continual Learning through Cognitive Replay