Abstract:Context. Neural networks require a large amount of annotated data to learn. Meta-learning algorithms propose a way to decrease number of training samples to only a few. One of the most prominent optimization-based meta-learning algorithms is MAML. However, its adaptation to new tasks is quite slow. The object of study is the process of meta-learning and adaptation phase as defined by the MAML algorithm.Objective. The goal of this work is creation of an approach, which should make it possible to: 1) increase the execution speed of MAML adaptation phase; 2) improve MAML accuracy in certain cases. The testing results will be shown on a publicly available few-shot learning dataset CIFAR-FS.Method. In this work an improvement to MAML meta-learning algorithm is proposed. Meta-learning procedure is defined in terms of tasks. In case of image classification problem, each task is to try to learn to classify images of new classes given only a few training examples. MAML defines 2 stages for the learning procedure: 1) adaptation to the new task; 2) meta-weights update. The whole training procedure requires Hessian computation, which makes the method computationally expensive. After being trained, the network will typically be used for adaptation to new tasks and the subsequent prediction on them. Thus, improving adaptation time is an important problem, which we focus on in this work. We introduce lambda pattern by which we restrict which weight we update in the network during the adaptation phase. This approach allows us to skip certain gradient computations. The pattern is selected given an allowed quality degradation threshold parameter. Among the pattern that fit the criteria, the fastest pattern is then selected. However, as it is discussed later, quality improvement is also possible is certain cases by a careful pattern selection.Results. The MAML algorithm with lambda pattern adaptation has been implemented, trained and tested on the open CIFAR-FS dataset. This makes our results easily reproducible.Conclusions. The experiments conducted have shown that via lambda adaptation pattern selection, it is possible to significantly improve the MAML method in the following areas: adaptation time has been decreased by a factor of 3 with minimal accuracy loss. Interestingly, accuracy for one-step adaptation has been substantially improved by using lambda patterns as well. Prospects for further research are to investigate a way of a more robust automatic pattern selection scheme.

BI-MAML: Balanced Incremental Approach for Meta Learning

When MAML Can Adapt Fast and How to Assist When It Cannot

MAML-en-LLM: Model Agnostic Meta-Training of LLMs for Improved In-Context Learning

Alpha MAML: Adaptive Model-Agnostic Meta-Learning

To Learn Effective Features: Understanding the Task-Specific Adaptation of MAML

ST-MAML: A Stochastic-Task based Method for Task-Heterogeneous Meta-Learning

MAML2: meta reinforcement learning via meta-learning for task categories

Scalable Multi-Modal Continual Meta-Learning

Task Similarity Aware Meta Learning - Theory-Inspired Improvement on MAML.

XB-MAML: Learning Expandable Basis Parameters for Effective Meta-Learning with Wide Task Coverage

FASTER OPTIMIZATION-BASED META-LEARNING ADAPTATION PHASE

Making Scalable Meta Learning Practical

Distilled Meta-learning for Multi-Class Incremental Learning

Fast Adaptation with Kernel and Gradient based Meta Leaning

Adaptive Compositional Continual Meta-Learning.

EW-CACTUs-MAML: A Robust Metalearning System for Rapid Classification on a Large Number of Tasks

How Does the Task Landscape Affect MAML Performance?

MAML is a Noisy Contrastive Learner in Classification

Model-agnostic Multi-Stage Loss Optimization Meta Learning

Memory-Based Optimization Methods for Model-Agnostic Meta-Learning and Personalized Federated Learning

MAC: a meta-learning approach for feature learning and recombination