Abstract:Continual learning enables learning systems to adapt to evolving data distributions by sequentially acquiring knowledge from a series of tasks. Unsupervised lifelong learning refers to the ability to learn over time while memorizing previous patterns without supervision. However, the prior methods in this field heavily rely on supervised or reinforcement learning, which necessitates annotated data, thereby limiting their scalability in real-world applications where data is often biased and lacks annotations. To overcome these challenges, this work introduces a novel approach called L ifelong L earning gets better with MixUp and U nsupervised C ontinual R epresentation (LL-UCR). LL-UCR aims to learn feature representations from unlabeled tasks, eliminating the need for annotated data. Within the LL-UCR framework, two innovative techniques are introduced: LL-MixUp, which mitigates catastrophic forgetting by interpolating samples between current and previous tasks, and Dark Experience Replay (DER) Buzzega et al. ( Adv Neural Inf Process Syst , 33 , 15920–15930 2020) adapted for UCR, aligning network logits across tasks. To overcome buffer size limitations in replay-based methods, the Retrospective Adversarial Replay (RAR) framework is incorporated, facilitating diverse replay sample generation. Through systematic analysis, we demonstrate that unsupervised visual representations exhibit remarkable resilience to catastrophic forgetting, consistently outperforming supervised methods in terms of performance and generalization on out-of-distribution tasks. Furthermore, our qualitative analysis reveals that LL-UCR fosters a smoother loss landscape and acquires meaningful feature representations. Extensive experimental evaluations conducted on diverse datasets validate the superior performance of LL-UCR compared to state-of-the-art supervised continual learning methods and the unsupervised LUMP Madaan et al. ( International conference on learning representations , 2020) method, effectively mitigating catastrophic forgetting.

R epresentational c ontinuity for u nsupervised c ontinual l earning

Representational Continuity for Unsupervised Continual Learning

Rethinking the Representational Continuity: Towards Unsupervised Continual Learning

Lifelong learning gets better with MixUp and unsupervised continual representation

Progressive Learning without Forgetting

InfoUCL: Learning Informative Representations for Unsupervised Continual Learning

Must Unsupervised Continual Learning Relies on Previous Information?

Effective Data Selection and Replay for Unsupervised Continual Learning

Integrating Present and Past in Unsupervised Continual Learning

Adaptive online continual multi-view learning

Online continual learning through unsupervised mutual information maximization

Adaptive Progressive Continual Learning.

Probing Representation Forgetting in Supervised and Unsupervised Continual Learning

Patch-Based Contrastive Learning and Memory Consolidation for Online Unsupervised Continual Learning

Feature Forgetting in Continual Representation Learning

UnCLe: Unsupervised Continual Learning of Depth Completion

U-TELL: Unsupervised Task Expert Lifelong Learning

A Comprehensive Empirical Evaluation on Online Continual Learning

CUCL: Codebook for Unsupervised Continual Learning.

Learning Invariant Features for Online Continual Learning

Representation Space Maintenance: Against Forgetting in Continual Learning