Abstract:The goal of Few-Shot Continual Learning (FSCL) is to incrementally learn novel tasks with limited labeled samples and preserve previous capabilities simultaneously, while current FSCL methods are all for the class-incremental purpose. Moreover, the evaluation of FSCL solutions is only the cumulative performance of all encountered tasks, but there is no work on exploring the domain generalization ability. Domain generalization is a challenging yet practical task that aims to generalize beyond training domains. In this paper, we set up a Generalized FSCL (GFSCL) protocol involving both class- and domain-incremental situations together with the domain generalization assessment. Firstly, two benchmark datasets and protocols are newly arranged, and detailed baselines are provided for this unexplored configuration. We find that common continual learning methods have poor generalization ability on unseen domains and cannot better cope with the catastrophic forgetting issue in cross-incremental tasks. In this way, we further propose a rehearsal-free framework based on Vision Transformer (ViT) named Contrastive Mixture of Adapters (CMoA). Due to different optimization targets of class increment and domain increment, the CMoA contains two parts: (1) For the class-incremental issue, the Mixture of Adapters (MoA) module is incorporated into ViT, then cosine similarity regularization and the dynamic weighting are designed to make each adapter learn specific knowledge and concentrate on particular classes. (2) For the domain-related issues and domain-invariant representation learning, we alleviate the inner-class variation by prototype-calibrated contrastive learning. The codes and protocols are available at <a class="link-external link-https" href="https://github.com/yawencui/CMoA" rel="external noopener nofollow">this https URL</a>.

Mutual Mentor: Online Contrastive Distillation Network for General Continual Learning

UniGrad-FS: Unified Gradient Projection with Flatter Sharpness for Continual Learning

Online Knowledge Distillation Via Mutual Contrastive Learning for Visual Recognition

Complementary Calibration: Boosting General Continual Learning With Collaborative Distillation and Self-Supervision

Imbalance Mitigation for Continual Learning via Knowledge Decoupling and Dual Enhanced Contrastive Learning

Adaptive online continual multi-view learning

Beyond Knowledge Distillation: Collaborative Learning for Bidirectional Model Assistance

Heterogeneous Continual Learning

DistilCSE: Effective Knowledge Distillation For Contrastive Sentence Embeddings

Unlock the Power: Competitive Distillation for Multi-Modal Large Language Models

Generalized Few-Shot Continual Learning with Contrastive Mixture of Adapters

Provable Contrastive Continual Learning

Unlocking Continual Learning Abilities in Language Models

LMACL: Improving Graph Collaborative Filtering with Learnable Model Augmentation Contrastive Learning

Boosting Continual Learning of Vision-Language Models via Mixture-of-Experts Adapters

Multi-Stage Knowledge Integration of Vision-Language Models for Continual Learning

Continual Learners are Incremental Model Generalizers

Improving Plasticity in Online Continual Learning via Collaborative Learning

Disentangled Contrastive Collaborative Filtering

Online Continual Learning through Mutual Information Maximization.