Abstract: Data-Free Meta-Learning (DFML) aims to efficiently learn new tasks by leveraging multiple pre-trained models without requiring their original training data. Existing inversion-based DFML methods construct pseudo tasks from a learnable dataset, which is inversely generated from the pre-trained model pool. For the first time, we reveal two major challenges hindering their practical deployments: Task-Distribution Shift (TDS) and Task-Distribution Corruption (TDC). TDS leads to a biased meta-learner because of the skewed task distribution towards newly generated tasks. TDC occurs when untrusted models characterized by misleading labels or poor quality pollute the task distribution. To tackle these issues, we introduce a robust DFML framework that ensures task distributional robustness. We propose to meta-learn from a pseudo task distribution, diversified through task interpolation within a compact task-memory buffer. This approach reduces the meta-learner's overreliance on newly generated tasks by maintaining consistent performance across a broader range of interpolated memory tasks, thus ensuring its generalization for unseen tasks. Additionally, our framework seamlessly incorporates an automated model selection mechanism into the meta-training phase, parameterizing each model's reliability as a learnable weight. This is optimized with a policy gradient algorithm inspired by reinforcement learning, effectively addressing the non-differentiable challenge posed by model selection. Comprehensive experiments across various datasets demonstrate the framework's effectiveness in mitigating TDS and TDC, underscoring its potential to improve DFML in real-world scenarios.

Domain Disentangled Meta-Learning

Multi-Domain Learning by Meta-Learning: Taking Optimal Steps in Multi-Domain Loss Landscapes by Inner-Loop Learning

Semi-supervised Meta-learning with Disentanglement for Domain-generalised Medical Image Segmentation

Multi-Domain Learning From Insufficient Annotations

Boosting urban prediction tasks with domain-sharing knowledge via meta-learning

Task-Distributionally Robust Data-Free Meta-Learning.

Learning to Generalize: Meta-Learning for Domain Generalization

Domain Mapping Network for Remote Sensing Cross-Domain Few-Shot Classification

Decoupled Training: Return of Frustratingly Easy Multi-Domain Learning

Meta-causal Learning for Single Domain Generalization

Maximal Domain Independent Representations Improve Transfer Learning

Open Domain Generalization with Domain-Augmented Meta-Learning

Meta Learning on a Sequence of Imbalanced Domains with Difficulty Awareness

Cross-Domain Extreme Learning Machines for Domain Adaptation

Learning Distribution-Matched Landmarks For Unsupervised Domain Adaptation

Domain Adaptive Meta-Learning for Dialogue State Tracking

Cluster, Split, Fuse, and Update: Meta-Learning for Open Compound Domain Adaptive Semantic Segmentation

EDRL: Entropy-guided disentangled representation learning for unsupervised domain adaptation in semantic segmentation

MLDGG: Meta-Learning for Domain Generalization on Graphs

Aggregation of Disentanglement: Reconsidering Domain Variations in Domain Generalization

Randomized Spectrum Transformations for Adapting Object Detector in Unseen Domains