Abstract:Meta learning recently has been heavily researched and helped advance the contemporary machine learning. However, achieving well-performing meta-learning model requires a large amount of training tasks with high-quality meta-data representing the underlying task generalization goal, which is sometimes difficult and expensive to obtain for real applications. Current meta-data-driven meta-learning approaches, however, are fairly hard to train satisfactory meta-models with imperfect training tasks. To address this issue, we suggest a meta-knowledge informed meta-learning (MKIML) framework to improve meta-learning by additionally integrating compensated meta-knowledge into meta-learning process. We preliminarily integrate meta-knowledge into meta-objective via using an appropriate meta-regularization (MR) objective to regularize capacity complexity of the meta-model function class to facilitate better generalization on unseen tasks. As a practical implementation, we introduce data augmentation consistency to encode invariance as meta-knowledge for instantiating MR objective, denoted by DAC-MR. The proposed DAC-MR is hopeful to learn well-performing meta-models from training tasks with noisy, sparse or unavailable meta-data. We theoretically demonstrate that DAC-MR can be treated as a proxy meta-objective used to evaluate meta-model without high-quality meta-data. Besides, meta-data-driven meta-loss objective combined with DAC-MR is capable of achieving better meta-level generalization. 10 meta-learning tasks with different network architectures and benchmarks substantiate the capability of our DAC-MR on aiding meta-model learning. Fine performance of DAC-MR are obtained across all settings, and are well-aligned with our theoretical insights. This implies that our DAC-MR is problem-agnostic, and hopeful to be readily applied to extensive meta-learning problems and tasks.

Dual Meta Calibration Mix for Improving Generalization in Meta-Learning

BlockMix: Meta Regularization and Self-Calibrated Inference for Metric-Based Meta-Learning

Towards Well-Generalizing Meta-Learning Via Adversarial Task Augmentation

DAC-MR: Data Augmentation Consistency Based Meta-Regularization for Meta-Learning

Patch Mix Augmentation with Dual Encoders for Meta-Learning

Improving Generalization in Meta-Learning Via Meta-Gradient Augmentation

Patch Mix Augmentation with Dual Encoders for Meta-Learning.

Set-based Meta-Interpolation for Few-Task Meta-Learning

Scalable Multi-Modal Continual Meta-Learning

MetaMixUp: Learning Adaptive Interpolation Policy of MixUp with Meta-Learning

A Meta Understanding of Meta-Learning

Meta-Learning Requires Meta-Augmentation

MAC: a meta-learning approach for feature learning and recombination

Revisit Multimodal Meta-Learning through the Lens of Multi-Task Learning

Model-Agnostic Learning to Meta-Learn

Conditional Mutual Information-Based Generalization Bound for Meta Learning

Theoretical Characterization of the Generalization Performance of Overfitted Meta-Learning

ST-MAML: A Stochastic-Task based Method for Task-Heterogeneous Meta-Learning

Learn To Learn More Precisely

MAML-en-LLM: Model Agnostic Meta-Training of LLMs for Improved In-Context Learning

More is Better: A Novel Multi-view Framework for Domain Generalization