Abstract:Meta-learning has been proposed as a framework to address the challenging few-shot learning setting. The key idea is to leverage a large number of similar few-shot tasks in order to learn how to adapt a base-learner to a new task for which only a few labeled samples are available. As deep neural networks (DNNs) tend to overfit using a few samples only, typical meta-learning models use shallow neural networks, thus limiting its effectiveness. In order to achieve top performance, some recent works tried to use the DNNs pre-trained on large-scale datasets but mostly in straight-forward manners, e.g., (1) taking their weights as a warm start of meta-training, and (2) freezing their convolutional layers as the feature extractor of base-learners. In this paper, we propose a novel approach called meta-transfer learning (MTL) , which learns to transfer the weights of a deep NN for few-shot learning tasks. Specifically, meta refers to training multiple tasks, and transfer is achieved by learning scaling and shifting functions of DNN weights (and biases) for each task. To further boost the learning efficiency of MTL, we introduce the hard task (HT) meta-batch scheme as an effective learning curriculum of few-shot classification tasks. We conduct experiments for five-class few-shot classification tasks on three challenging benchmarks, mini ImageNet, tiered ImageNet, and Fewshot-CIFAR100 (FC100), in both supervised and semi-supervised settings. Extensive comparisons to related works validate that our MTL approach trained with the proposed HT meta-batch scheme achieves top performance. An ablation study also shows that both components contribute to fast convergence and high accuracy.

Clustered Task-Aware Meta-Learning by Learning From Learning Paths

Task Attended Meta-Learning for Few-Shot Learning

Task Similarity Aware Meta Learning - Theory-Inspired Improvement on MAML.

Rethinking Meta-Learning from a Learning Lens

Meta-Learning to Cluster

Multimodal meta-learning through meta-learned task representations

Task-Distributionally Robust Data-Free Meta-Learning.

Hierarchical Meta-Learning with Hyper-Tasks for Few-Shot Learning

Scalable Multi-Modal Continual Meta-Learning

EEML: Ensemble Embedded Meta-Learning

ST-MAML: A Stochastic-Task based Method for Task-Heterogeneous Meta-Learning

Hacking Task Confounder in Meta-Learning

A Task-Aware Attention-Based Method for Improved Meta-Learning.

Automated Relational Meta-learning

MAML2: meta reinforcement learning via meta-learning for task categories

ConML: A Universal Meta-Learning Framework with Task-Level Contrastive Learning

Task Groupings Regularization: Data-Free Meta-Learning with Heterogeneous Pre-trained Models

Meta-Learning with Heterogeneous Tasks

Meta-Learning with MAML on Trees

Meta-Transfer Learning Through Hard Tasks

Meta Reinforcement Learning with Task Embedding and Shared Policy.