Abstract:Continual learning enables learning systems to adapt to evolving data distributions by sequentially acquiring knowledge from a series of tasks. Unsupervised lifelong learning refers to the ability to learn over time while memorizing previous patterns without supervision. However, the prior methods in this field heavily rely on supervised or reinforcement learning, which necessitates annotated data, thereby limiting their scalability in real-world applications where data is often biased and lacks annotations. To overcome these challenges, this work introduces a novel approach called L ifelong L earning gets better with MixUp and U nsupervised C ontinual R epresentation (LL-UCR). LL-UCR aims to learn feature representations from unlabeled tasks, eliminating the need for annotated data. Within the LL-UCR framework, two innovative techniques are introduced: LL-MixUp, which mitigates catastrophic forgetting by interpolating samples between current and previous tasks, and Dark Experience Replay (DER) Buzzega et al. ( Adv Neural Inf Process Syst , 33 , 15920–15930 2020) adapted for UCR, aligning network logits across tasks. To overcome buffer size limitations in replay-based methods, the Retrospective Adversarial Replay (RAR) framework is incorporated, facilitating diverse replay sample generation. Through systematic analysis, we demonstrate that unsupervised visual representations exhibit remarkable resilience to catastrophic forgetting, consistently outperforming supervised methods in terms of performance and generalization on out-of-distribution tasks. Furthermore, our qualitative analysis reveals that LL-UCR fosters a smoother loss landscape and acquires meaningful feature representations. Extensive experimental evaluations conducted on diverse datasets validate the superior performance of LL-UCR compared to state-of-the-art supervised continual learning methods and the unsupervised LUMP Madaan et al. ( International conference on learning representations , 2020) method, effectively mitigating catastrophic forgetting.

Meta-Learning Representations for Continual Learning

Online Fast Adaptation and Knowledge Accumulation: a New Approach to Continual Learning

CLFR-M: Continual Learning Framework for Robots Via Human Feedback and Dynamic Memory

Adaptive online continual multi-view learning

Learning to Learn without Forgetting using Attention

Deep Online Learning via Meta-Learning: Continual Adaptation for Model-Based RL

Automating Continual Learning

Online Continual Learning Via the Meta-learning Update with Multi-scale Knowledge Distillation and Data Augmentation

Lifelong learning gets better with MixUp and unsupervised continual representation

Understanding Catastrophic Forgetting and Remembering in Continual Learning with Optimal Relevance Mapping

Meta Continual Learning via Dynamic Programming

Rethinking the Representational Continuity: Towards Unsupervised Continual Learning

Task Agnostic Continual Learning via Meta Learning

Improving Plasticity in Online Continual Learning via Collaborative Learning

Brain-Inspired Continual Learning-Robust Feature Distillation and Re-Consolidation for Class Incremental Learning

Adaptive Progressive Continual Learning.

Probing Representation Forgetting in Supervised and Unsupervised Continual Learning

Learning Fast, Learning Slow: A General Continual Learning Method based on Complementary Learning System

R epresentational c ontinuity for u nsupervised c ontinual l earning

CLEO: Continual Learning of Evolving Ontologies

Interactive Continual Learning: Fast and Slow Thinking