Abstract:The gradient-based meta learning and its approximation algorithms have been widely used in the few-shot scenarios. In practice, it is common for the trained meta-model to employ uniform settings for gradient descent steps across different tasks. However, the meta-model may be biased toward some tasks. The convergence issue occurs that some tasks may see convergence in a few steps while others fail to approach the optimum in the whole inner loop. The bias problem may cause the trained meta-model works well in some tasks but has unexpected bad performance in other tasks, which hurts the generality of the meta-model. To address this issue, in this paper, we formally establish the approximation between the metric-based strategy and gradient descent in meta-test. By directly calculating similarity to classify data, the trained meta-model avoids the convergence issue. We point out that the metric-based methods can closely approximate the gradient descent in meta-test if the representation capability of the derived features and the convergence of the inner loop during meta -training are guaranteed. Based on such observation, we propose a new meta-learning model GMT2 (Gradient-based Meta-Train with Metric-based meta-Test) by combining gradient descent in meta -training with metric-based methods in meta-test. GMT2 employs a new first-order approximation scheme using the adversarial update strategy which not only enhances the feature representation of inner layers, but also allows enough inner gradient steps without calculating second-order derivatives. Experiments show that GMT2 achieves better efficiency and competitive accuracy comparing with popular meta-learning models.(c) 2023 Elsevier B.V. All rights reserved.

Meta weight learning via model-agnostic meta-learning

Meta-Weight-Net: Learning an Explicit Mapping For Sample Weighting.

Meta-Weight-Net: Learning an Explicit Mapping for Sample Weighting

Gradient-Based Meta-Learning Using Adaptive Multiple Loss Weighting and Homoscedastic Uncertainty

MetaInfoNet: Learning Task-Guided Information for Sample Reweighting

Learning to Learn by Jointly Optimizing Neural Architecture and Weights

MetaWeighting: Learning to Weight Tasks in Multi-Task Learning.

A Task-Aware Attention-Based Method for Improved Meta-Learning.

A Survey of Meta-learning for Classification Tasks

A Meta Understanding of Meta-Learning

Gradient-Based Meta-Learning Using Uncertainty to Weigh Loss for Few-Shot Learning

Connection-Adaptive Meta-Learning

A Probabilistic Formulation for Meta-Weight-Net

Rethinking Meta-Learning from a Learning Lens

Learning Symbolic Model-Agnostic Loss Functions via Meta-Learning

Meta-learning approaches for learning-to-learn in deep learning: A survey

Adversarial gradient-based meta learning with metric-based test

The meta-learning method for the ensemble model based on situational meta-task

ST-MAML: A Stochastic-Task based Method for Task-Heterogeneous Meta-Learning

Unsupervised Meta-Learning for Reinforcement Learning

Adaptive Gradient-Based Meta-Learning Methods