Abstract:Deep learning models are often trained on datasets that are limited in size and distribution, which may not fully represent the entire range of data encountered in practice. Thus, making deep learning models generalize to out-of-distribution data has received a significant amount of attention in recent studies due to the critical importance of this ability in real-world applications. Meta learning as an effective knowledge transfer paradigm, which learns a base model with high generalization ability to adapt to new data distributions by minimizing domain shifts across tasks during meta-training. However, most existing meta learning methods assume that the base model can access the labels of different domains, and this assumption is demanding in many real application scenarios. In addition, these methods focus on narrowing data-level domain shifts, while ignoring task-level domain shifts, which may lead to inadequate or even negative transfer. Inspired by human learners who use induction to learn and master new tasks, we propose a novel domain-aware meta learning framework for out-of-distribution generalization, termed SMLG. This framework enables the base model to generalize effectively to unseen domains without relying on domain-specific labels. Specifically, we develop a domain-aware transformation module to obtain meta representation and pseudo domain labels. As a result, the base model can be trained robustly without the need for direct domain label input. Furthermore, to investigate the impact of domain shifts at different levels, we introduce a joint loss function that combines cross-entropy with a domain alignment constraint. Extensive experiments on benchmark datasets demonstrate the efficacy of our framework.

Task-Distributionally Robust Data-Free Meta-Learning.

Task Groupings Regularization: Data-Free Meta-Learning with Heterogeneous Pre-trained Models

Meta-Reinforcement Learning Robust to Distributional Shift Via Performing Lifelong In-Context Learning

Meta-learning without data via Wasserstein distributionally-robust model fusion.

FREE: Faster and Better Data-Free Meta-Learning

Learning to Learn from APIs: Black-Box Data-Free Meta-Learning

Meta-Learning Without Data Via Unconditional Diffusion Models

Towards Domain-Aware Stable Meta Learning for Out-of-Distribution Generalization

Clustered Task-Aware Meta-Learning by Learning From Learning Paths

ST-MAML: A Stochastic-Task based Method for Task-Heterogeneous Meta-Learning

Robust Fast Adaptation from Adversarially Explicit Task Distribution Generation

Decentralized federated meta‐learning framework for few‐shot multitask learning

Meta-learning of feature distribution alignment for enhanced feature sharing

Boosting urban prediction tasks with domain-sharing knowledge via meta-learning

Data-Efficient and Robust Task Selection for Meta-Learning

Meta Distant Transfer Learning for Pre-trained Language Models.

Set-based Meta-Interpolation for Few-Task Meta-Learning

DFML: Decentralized Federated Mutual Learning

MetaRM: Shifted Distributions Alignment via Meta-Learning

Rethinking Meta-Learning from a Learning Lens

Few-Shot Learning of Compact Models via Task-Specific Meta Distillation