Abstract:Robust Model-Agnostic Meta-Learning (MAML) is usually adopted to train a meta-model which may fast adapt to novel classes with only a few exemplars and meanwhile remain robust to adversarial attacks. The conventional solution for robust MAML is to introduce robustness-promoting regularization during meta-training stage. With such a regularization, previous robust MAML methods simply follow the typical MAML practice that the number of training shots should match with the number of test shots to achieve an optimal adaptation performance. However, although the robustness can be largely improved, previous methods sacrifice clean accuracy a lot. In this paper, we observe that introducing robustness-promoting regularization into MAML reduces the intrinsic dimension of clean sample features, which results in a lower capacity of clean representations. This may explain why the clean accuracy of previous robust MAML methods drops severely. Based on this observation, we propose a simple strategy, i.e., increasing the number of training shots, to mitigate the loss of intrinsic dimension caused by robustness-promoting regularization. Though simple, our method remarkably improves the clean accuracy of MAML without much loss of robustness, producing a robust yet accurate model. Extensive experiments demonstrate that our method outperforms prior arts in achieving a better trade-off between accuracy and robustness. Besides, we observe that our method is less sensitive to the number of fine-tuning steps during meta-training, which allows for a reduced number of fine-tuning steps to improve training efficiency.

Model-agnostic Multi-Stage Loss Optimization Meta Learning

How to train your MAML

Gradient-Based Meta-Learning Using Uncertainty to Weigh Loss for Few-Shot Learning

Rethinking the Number of Shots in Robust Model-Agnostic Meta-Learning

Meta-Learning with Task-Adaptive Loss Function for Few-Shot Learning

How to Train Your MAML to Excel in Few-Shot Classification

Fast Adaptation with Kernel and Gradient based Meta Leaning

Alpha MAML: Adaptive Model-Agnostic Meta-Learning

When MAML Can Adapt Fast and How to Assist When It Cannot

On the Global Optimality of Model-Agnostic Meta-Learning

A Nested Bi-level Optimization Framework for Robust Few Shot Learning

Memory-Based Optimization Methods for Model-Agnostic Meta-Learning and Personalized Federated Learning

ST-MAML: A Stochastic-Task based Method for Task-Heterogeneous Meta-Learning

Convolutional Shrinkage Neural Networks Based Model-Agnostic Meta-Learning for Few-Shot Learning

Boosting Meta-Training with Base Class Information for Few-Shot Learning

Improving Meta-Learning Model Via Meta-Contrastive Loss

Repurposing Pretrained Models for Robust Out-of-domain Few-Shot Learning

Task-Agnostic Meta-Learning for Few-shot Learning

Dual role of alpha-acetolactate decarboxylase in Lactococcus lactis subsp. lactis

Revisiting Unsupervised Meta-Learning via the Characteristics of Few-Shot Tasks

Task Similarity Aware Meta Learning - Theory-Inspired Improvement on MAML.