Abstract:Meta-learning has been widely applied to solving few-shot reinforcement learning problems, where we hope to obtain an agent that can learn quickly in a new task. However, these algorithms often ignore some isolated tasks in pursuit of the average performance, which may result in negative adaptation in these isolated tasks, and they usually need sufficient learning in a stationary task distribution. In this paper, our algorithm presents a hierarchical framework of double meta-learning, and the whole framework includes classification, meta-learning, and re-adaptation. Firstly, in the classification process, we classify tasks into several task subsets, considered as some categories of tasks, by learned parameters of each task, which can separate out some isolated tasks thereafter. Secondly, in the meta-learning process, we learn category parameters in all subsets via meta-learning. Simultaneously, based on the gradient of each category parameter in each subset, we use meta-learning again to learn a new meta-parameter related to the whole task set, which can be used as an initial parameter for the new task. Finally, in the re-adaption process, we adapt the parameter of the new task with two steps, by the meta-parameter and the appropriate category parameter successively. Experimentally, we demonstrate our algorithm prevents the agent from negative adaptation without losing the average performance for the whole task set. Additionally, our algorithm presents a more rapid adaptation process within re-adaptation. Moreover, we show the good performance of our algorithm with fewer samples as the agent is exposed to an online meta-learning setting.

MAML-en-LLM: Model Agnostic Meta-Training of LLMs for Improved In-Context Learning

When MAML Can Adapt Fast and How to Assist When It Cannot

Cross-lingual Adaption Model-Agnostic Meta-Learning for Natural Language Understanding

BI-MAML: Balanced Incremental Approach for Meta Learning

Meta-Learning the Difference: Preparing Large Language Models for Efficient Adaptation

Model-Agnostic Meta-Learning for Natural Language Understanding Tasks in Finance

To Learn Effective Features: Understanding the Task-Specific Adaptation of MAML

Alpha MAML: Adaptive Model-Agnostic Meta-Learning

Context-Aware Meta-Learning

How Does the Task Landscape Affect MAML Performance?

ST-MAML: A Stochastic-Task based Method for Task-Heterogeneous Meta-Learning

When does MAML Work the Best? An Empirical Study on Model-Agnostic Meta-Learning in NLP Applications

Task Similarity Aware Meta Learning - Theory-Inspired Improvement on MAML.

Meta-in-context learning in large language models

Demonstrative Instruction Following in Multimodal LLMs Via Integrating Low-Rank Adaptation with Ensemble Learning

MAML2: meta reinforcement learning via meta-learning for task categories

Meta-Learning Adaptable Foundation Models

MetaTool: Facilitating Large Language Models to Master Tools with Meta-task Augmentation

How to train your MAML

LLMs-as-Instructors: Learning from Errors Toward Automating Model Improvement

XB-MAML: Learning Expandable Basis Parameters for Effective Meta-Learning with Wide Task Coverage