Abstract: Data-Free Meta-Learning (DFML) aims to efficiently learn new tasks by leveraging multiple pre-trained models without requiring their original training data. Existing inversion-based DFML methods construct pseudo tasks from a learnable dataset, which is inversely generated from the pre-trained model pool. For the first time, we reveal two major challenges hindering their practical deployments: Task-Distribution Shift (TDS) and Task-Distribution Corruption (TDC). TDS leads to a biased meta-learner because of the skewed task distribution towards newly generated tasks. TDC occurs when untrusted models characterized by misleading labels or poor quality pollute the task distribution. To tackle these issues, we introduce a robust DFML framework that ensures task distributional robustness. We propose to meta-learn from a pseudo task distribution, diversified through task interpolation within a compact task-memory buffer. This approach reduces the meta-learner's overreliance on newly generated tasks by maintaining consistent performance across a broader range of interpolated memory tasks, thus ensuring its generalization for unseen tasks. Additionally, our framework seamlessly incorporates an automated model selection mechanism into the meta-training phase, parameterizing each model's reliability as a learnable weight. This is optimized with a policy gradient algorithm inspired by reinforcement learning, effectively addressing the non-differentiable challenge posed by model selection. Comprehensive experiments across various datasets demonstrate the framework's effectiveness in mitigating TDS and TDC, underscoring its potential to improve DFML in real-world scenarios.

ST-MAML: A Stochastic-Task based Method for Task-Heterogeneous Meta-Learning

Task Similarity Aware Meta Learning - Theory-Inspired Improvement on MAML.

Meta-Learning with Heterogeneous Tasks

Multimodal meta-learning through meta-learned task representations

Robust Fast Adaptation from Adversarially Explicit Task Distribution Generation

When MAML Can Adapt Fast and How to Assist When It Cannot

BI-MAML: Balanced Incremental Approach for Meta Learning

Set-based Meta-Interpolation for Few-Task Meta-Learning

How Does the Task Landscape Affect MAML Performance?

MAML2: meta reinforcement learning via meta-learning for task categories

Scalable Multi-Modal Continual Meta-Learning

Task-Agnostic Meta-Learning for Few-shot Learning

Model-agnostic Multi-Stage Loss Optimization Meta Learning

MAML-en-LLM: Model Agnostic Meta-Training of LLMs for Improved In-Context Learning

Memory-Based Optimization Methods for Model-Agnostic Meta-Learning and Personalized Federated Learning

A Task-Aware Attention-Based Method for Improved Meta-Learning.

Dif-MAML: Decentralized Multi-Agent Meta-Learning

Towards Task Sampler Learning for Meta-Learning

Towards Well-Generalizing Meta-Learning Via Adversarial Task Augmentation

Task-Distributionally Robust Data-Free Meta-Learning.