Abstract:Modern natural language processing (NLP) state-of-the-art (SoTA) deep learning (DL) models have hundreds of millions of parameters, making them extremely complex. Large datasets are required for training these models, and while pretraining has reduced this requirement, human-labelled datasets are still necessary for fine-tuning. Few-shot learning (FSL) techniques, such as meta-learning, try to train models from smaller datasets to mitigate this cost. However, the tasks used to evaluate these meta-learners frequently diverge from the problems in the real world that they are meant to resolve. This work aims to apply meta-learning to a problem that is more pertinent to the real world: class incremental learning (IL). In this scenario, after completing its training, the model learns to classify newly introduced classes. One unique quality of meta-learners is that they can generalise from a small sample size to classes that have never been seen before, which makes them especially useful for class incremental learning (IL). The method describes how to emulate class IL using proxy new classes. This method allows a meta-learner to complete the task without the need for retraining. To generate predictions, the transformer-based aggregation function in a meta-learner that modifies data from examples across all classes has been proposed. The principal contributions of the model include concurrently considering the entire support and query sets, and prioritising attention to crucial samples, such as the question, to increase the significance of its impact during inference. The outcomes demonstrate that the model surpasses prevailing benchmarks in the industry. Notably, most meta-learners demonstrate significant generalisation in the context of class IL even without specific training for this task. This paper establishes a high-performing baseline for subsequent transformer-based aggregation techniques, thereby emphasising the practical significance of meta-learners in class IL.

Incremental Meta-Learning via Indirect Discriminant Alignment

MetaFSCIL: A Meta-Learning Approach for Few-Shot Class Incremental Learning

Incremental Meta-Learning Via Episodic Replay Distillation for Few-Shot Image Recognition

Distilled Meta-learning for Multi-Class Incremental Learning

Meta-Learning-Based Incremental Few-Shot Object Detection

Boosting Meta-Training with Base Class Information for Few-Shot Learning

Meta-SGD: Learning to Learn Quickly for Few-Shot Learning

Meta-Learning across Meta-Tasks for Few-Shot Learning

Meta-learning for real-world class incremental learning: a transformer-based approach

Meta-learning methodology based on meta-unsupervised algorithm for meta-model selection to solve few-shot base-tasks

A metric-based meta-learning approach combined attention mechanism and ensemble learning for few-shot learning

Learning to Learn with Indispensable Connections

Scalable Multi-Modal Continual Meta-Learning

Few-shot Learning with Meta Metric Learners

Task Attended Meta-Learning for Few-Shot Learning

MetaDelta: A Meta-Learning System for Few-shot Image Classification

Variable-Shot Adaptation for Online Meta-Learning

Improve Meta-learning for Few-Shot Text Classification with All You Can Acquire from the Tasks

Few-Shot Directed Meta-Learning for Image Classification

A Hybrid Approach with Optimization-Based and Metric-Based Meta-Learner for Few-Shot Learning

Unsupervised meta-learning for few-shot learning