Abstract:Existing continual image classification methods demonstrate that samples from all sequences of continual classification tasks contain common (task-invariant) features and class-specific (task-variant) features that can be decoupled for classification tasks. However, the existing feature decomposition strategies only focus on individual tasks while neglecting the essential cues that the relationship between different tasks can provide, thereby hindering the improvement of continual image classification results. To address this issue, we propose an Adversarial Contrastive Continual Learning (ACCL) method that decouples task-invariant and task-variant features by constructing all-round, multi-level contrasts on sample pairs within individual tasks or from different tasks. Specifically, three constraints on the distribution of task-invariant and task-variant features are included, i.e. , task-invariant features across different tasks should remain consistent, task-variant features should exhibit differences, and task-invariant and task-variant features should differ from each other. At the same time, we also design an effective contrastive replay strategy to make full use of the replay samples to participate in the construction of sample pairs, further alleviating the forgetting problem, and modeling cross-task relationships. Through extensive experiments on continual image classification tasks on CIFAR100, MiniImageNet and TinyImageNet, we show the superiority of our proposed strategy, improving the accuracy and with better visualized outcomes. The code and results can be found from the link of https://github.com/rmcong/ACCL_TMM24 .

Exemplar-Free Continual Transformer with Convolutions

Exemplar-free Continual Learning of Vision Transformers via Gated Class-Attention and Cascaded Feature Drift Compensation

Dynamic Transformer Architecture for Continual Learning of Multimodal Tasks

SemiCVT: Semi-Supervised Convolutional Vision Transformer for Semantic Segmentation

Task-Attentive Transformer Architecture for Continual Learning of Vision-and-Language Tasks Using Knowledge Distillation

Online Continual Learning with Contrastive Vision Transformer

Continual HyperTransformer: A Meta-Learner for Continual Few-Shot Learning

Video Class-Incremental Learning with Clip Based Transformer

Remembering Transformer for Continual Learning

Transformers for Supervised Online Continual Learning

Towards Exemplar-Free Continual Learning in Vision Transformers: an Account of Attention, Functional and Weight Regularization

Soft-TransFormers for Continual Learning

Continual Learning via Learning a Continual Memory in Vision Transformer

FCL-ViT: Task-Aware Attention Tuning for Continual Learning

Modeling Inner- and Cross-Task Contrastive Relations for Continual Image Classification

In-Context Convergence of Transformers

CEAT: Continual Expansion and Absorption Transformer for Non-Exemplar Class-Incremental Learning

Vision Transformer Adapters for Generalizable Multitask Learning

Simpler is Better: off-the-shelf Continual Learning Through Pretrained Backbones

Conv2Former: A Simple Transformer-Style ConvNet for Visual Recognition

Vision Conformer: Incorporating Convolutions into Vision Transformer Layers