Abstract:Meta learning recently has been heavily researched and helped advance the contemporary machine learning. However, achieving well-performing meta-learning model requires a large amount of training tasks with high-quality meta-data representing the underlying task generalization goal, which is sometimes difficult and expensive to obtain for real applications. Current meta-data-driven meta-learning approaches, however, are fairly hard to train satisfactory meta-models with imperfect training tasks. To address this issue, we suggest a meta-knowledge informed meta-learning (MKIML) framework to improve meta-learning by additionally integrating compensated meta-knowledge into meta-learning process. We preliminarily integrate meta-knowledge into meta-objective via using an appropriate meta-regularization (MR) objective to regularize capacity complexity of the meta-model function class to facilitate better generalization on unseen tasks. As a practical implementation, we introduce data augmentation consistency to encode invariance as meta-knowledge for instantiating MR objective, denoted by DAC-MR. The proposed DAC-MR is hopeful to learn well-performing meta-models from training tasks with noisy, sparse or unavailable meta-data. We theoretically demonstrate that DAC-MR can be treated as a proxy meta-objective used to evaluate meta-model without high-quality meta-data. Besides, meta-data-driven meta-loss objective combined with DAC-MR is capable of achieving better meta-level generalization. 10 meta-learning tasks with different network architectures and benchmarks substantiate the capability of our DAC-MR on aiding meta-model learning. Fine performance of DAC-MR are obtained across all settings, and are well-aligned with our theoretical insights. This implies that our DAC-MR is problem-agnostic, and hopeful to be readily applied to extensive meta-learning problems and tasks.

Consistent MetaReg: Alleviating Intra-task Discrepancy for Better Meta-knowledge

Consistent Meta-Regularization for Better Meta-Knowledge in Few-Shot Learning

Improving Meta-Learning Model Via Meta-Contrastive Loss

Perturbing the Gradient for Alleviating Meta Overfitting

Class Regularization: Improve Few-shot Image Classification by Reducing Meta Shift

Adaptive Meta-learner via Gradient Similarity for Few-shot Text Classification

Gradient-Based Meta-Learning Using Uncertainty to Weigh Loss for Few-Shot Learning

Improving Generalization of Meta-Learning with Inverted Regularization at Inner-Level

Meta-learning of feature distribution alignment for enhanced feature sharing

Incremental Meta-Learning via Indirect Discriminant Alignment

The Role of Deconfounding in Meta-learning

Meta-Tasks: An alternative view on Meta-Learning Regularization

DAC-MR: Data Augmentation Consistency Based Meta-Regularization for Meta-Learning

Effectively learn how to learn: a novel few-shot learning with meta-gradient memory

Task Groupings Regularization: Data-Free Meta-Learning with Heterogeneous Pre-trained Models

The meta-learning method for the ensemble model based on situational meta-task

AG-Meta: Adaptive graph meta-learning via representation consistency over local subgraphs

Rethinking the Number of Shots in Robust Model-Agnostic Meta-Learning

Adaptive Gradient-Based Meta-Learning Methods

Is the Meta-Learning Idea Able to Improve the Generalization of Deep Neural Networks on the Standard Supervised Learning?

Task Attended Meta-Learning for Few-Shot Learning