Abstract:Continual learning enables learning systems to adapt to evolving data distributions by sequentially acquiring knowledge from a series of tasks. Unsupervised lifelong learning refers to the ability to learn over time while memorizing previous patterns without supervision. However, the prior methods in this field heavily rely on supervised or reinforcement learning, which necessitates annotated data, thereby limiting their scalability in real-world applications where data is often biased and lacks annotations. To overcome these challenges, this work introduces a novel approach called L ifelong L earning gets better with MixUp and U nsupervised C ontinual R epresentation (LL-UCR). LL-UCR aims to learn feature representations from unlabeled tasks, eliminating the need for annotated data. Within the LL-UCR framework, two innovative techniques are introduced: LL-MixUp, which mitigates catastrophic forgetting by interpolating samples between current and previous tasks, and Dark Experience Replay (DER) Buzzega et al. ( Adv Neural Inf Process Syst , 33 , 15920–15930 2020) adapted for UCR, aligning network logits across tasks. To overcome buffer size limitations in replay-based methods, the Retrospective Adversarial Replay (RAR) framework is incorporated, facilitating diverse replay sample generation. Through systematic analysis, we demonstrate that unsupervised visual representations exhibit remarkable resilience to catastrophic forgetting, consistently outperforming supervised methods in terms of performance and generalization on out-of-distribution tasks. Furthermore, our qualitative analysis reveals that LL-UCR fosters a smoother loss landscape and acquires meaningful feature representations. Extensive experimental evaluations conducted on diverse datasets validate the superior performance of LL-UCR compared to state-of-the-art supervised continual learning methods and the unsupervised LUMP Madaan et al. ( International conference on learning representations , 2020) method, effectively mitigating catastrophic forgetting.

Lifelong learning gets better with MixUp and unsupervised continual representation

R epresentational c ontinuity for u nsupervised c ontinual l earning

Rethinking the Representational Continuity: Towards Unsupervised Continual Learning

Representational Continuity for Unsupervised Continual Learning

Online continual learning through unsupervised mutual information maximization

Meta-Learning Representations for Continual Learning

Addressing Loss of Plasticity and Catastrophic Forgetting in Continual Learning

InfoUCL: Learning Informative Representations for Unsupervised Continual Learning

Unsupervised Lifelong Learning with Curricula

Must Unsupervised Continual Learning Relies on Previous Information?

Adaptive online continual multi-view learning

Effective Data Selection and Replay for Unsupervised Continual Learning

Adaptive Progressive Continual Learning.

Self-Supervised Learning Aided Class-Incremental Lifelong Learning

Continual Learning: Less Forgetting, More OOD Generalization via Adaptive Contrastive Replay

Probing Representation Forgetting in Supervised and Unsupervised Continual Learning

Integrating Present and Past in Unsupervised Continual Learning

Adaptive Update Direction Rectification for Unsupervised Continual Learning

Continuous Unsupervised Domain Adaptation Using Stabilized Representations and Experience Replay

Automating Continual Learning

Understanding Catastrophic Forgetting and Remembering in Continual Learning with Optimal Relevance Mapping