Abstract:Context. Neural networks require a large amount of annotated data to learn. Meta-learning algorithms propose a way to decrease number of training samples to only a few. One of the most prominent optimization-based meta-learning algorithms is MAML. However, its adaptation to new tasks is quite slow. The object of study is the process of meta-learning and adaptation phase as defined by the MAML algorithm.Objective. The goal of this work is creation of an approach, which should make it possible to: 1) increase the execution speed of MAML adaptation phase; 2) improve MAML accuracy in certain cases. The testing results will be shown on a publicly available few-shot learning dataset CIFAR-FS.Method. In this work an improvement to MAML meta-learning algorithm is proposed. Meta-learning procedure is defined in terms of tasks. In case of image classification problem, each task is to try to learn to classify images of new classes given only a few training examples. MAML defines 2 stages for the learning procedure: 1) adaptation to the new task; 2) meta-weights update. The whole training procedure requires Hessian computation, which makes the method computationally expensive. After being trained, the network will typically be used for adaptation to new tasks and the subsequent prediction on them. Thus, improving adaptation time is an important problem, which we focus on in this work. We introduce lambda pattern by which we restrict which weight we update in the network during the adaptation phase. This approach allows us to skip certain gradient computations. The pattern is selected given an allowed quality degradation threshold parameter. Among the pattern that fit the criteria, the fastest pattern is then selected. However, as it is discussed later, quality improvement is also possible is certain cases by a careful pattern selection.Results. The MAML algorithm with lambda pattern adaptation has been implemented, trained and tested on the open CIFAR-FS dataset. This makes our results easily reproducible.Conclusions. The experiments conducted have shown that via lambda adaptation pattern selection, it is possible to significantly improve the MAML method in the following areas: adaptation time has been decreased by a factor of 3 with minimal accuracy loss. Interestingly, accuracy for one-step adaptation has been substantially improved by using lambda patterns as well. Prospects for further research are to investigate a way of a more robust automatic pattern selection scheme.

EW-CACTUs-MAML: A Robust Metalearning System for Rapid Classification on a Large Number of Tasks

A Survey of Meta-learning for Classification Tasks

BI-MAML: Balanced Incremental Approach for Meta Learning

MAC: a meta-learning approach for feature learning and recombination

MAML2: meta reinforcement learning via meta-learning for task categories

Meta-LMTC - Meta-Learning for Large-Scale Multi-Label Text Classification.

Revisiting Unsupervised Meta-Learning via the Characteristics of Few-Shot Tasks

Scalable Multi-Modal Continual Meta-Learning

Clustered Task-Aware Meta-Learning by Learning From Learning Paths

MAML-en-LLM: Model Agnostic Meta-Training of LLMs for Improved In-Context Learning

Task Similarity Aware Meta Learning - Theory-Inspired Improvement on MAML.

ST-MAML: A Stochastic-Task based Method for Task-Heterogeneous Meta-Learning

Alpha MAML: Adaptive Model-Agnostic Meta-Learning

A Joint Optimization Framework of the Embedding Model and Classifier for Meta-Learning

When MAML Can Adapt Fast and How to Assist When It Cannot

A Nested Bi-level Optimization Framework for Robust Few Shot Learning

FASTER OPTIMIZATION-BASED META-LEARNING ADAPTATION PHASE

EEML: Ensemble Embedded Meta-Learning

Context Adaptive Metric Model for Meta-learning

How to Train Your MAML to Excel in Few-Shot Classification