Abstract:Deep learning models are often trained on datasets that are limited in size and distribution, which may not fully represent the entire range of data encountered in practice. Thus, making deep learning models generalize to out-of-distribution data has received a significant amount of attention in recent studies due to the critical importance of this ability in real-world applications. Meta learning as an effective knowledge transfer paradigm, which learns a base model with high generalization ability to adapt to new data distributions by minimizing domain shifts across tasks during meta-training. However, most existing meta learning methods assume that the base model can access the labels of different domains, and this assumption is demanding in many real application scenarios. In addition, these methods focus on narrowing data-level domain shifts, while ignoring task-level domain shifts, which may lead to inadequate or even negative transfer. Inspired by human learners who use induction to learn and master new tasks, we propose a novel domain-aware meta learning framework for out-of-distribution generalization, termed SMLG. This framework enables the base model to generalize effectively to unseen domains without relying on domain-specific labels. Specifically, we develop a domain-aware transformation module to obtain meta representation and pseudo domain labels. As a result, the base model can be trained robustly without the need for direct domain label input. Furthermore, to investigate the impact of domain shifts at different levels, we introduce a joint loss function that combines cross-entropy with a domain alignment constraint. Extensive experiments on benchmark datasets demonstrate the efficacy of our framework.

Meta-learning without data via Wasserstein distributionally-robust model fusion.

Task-Distributionally Robust Data-Free Meta-Learning.

Meta-Learning Without Data Via Unconditional Diffusion Models

FREE: Faster and Better Data-Free Meta-Learning

Learning to Learn from APIs: Black-Box Data-Free Meta-Learning

Task Groupings Regularization: Data-Free Meta-Learning with Heterogeneous Pre-trained Models

Meta-learning of feature distribution alignment for enhanced feature sharing

Architecture, Dataset and Model-Scale Agnostic Data-free Meta-Learning

MetaDiff: Meta-Learning with Conditional Diffusion for Few-Shot Learning

Towards Domain-Aware Stable Meta Learning for Out-of-Distribution Generalization

DFRD: Data-Free Robustness Distillation for Heterogeneous Federated Learning

Meta-DM: Applications of Diffusion Models on Few-Shot Learning.

ML-FDA: Meta-Learning via Feature Distribution Alignment for Few-Shot Learning

Robust Meta-Representation Learning via Global Label Inference and Classification

Rethinking Meta-Learning from a Learning Lens

Meta-KD: A Meta Knowledge Distillation Framework for Language Model Compression across Domains

Meta-Tasks: An alternative view on Meta-Learning Regularization

Meta-FL: A Novel Meta-Learning Framework for Optimizing Heterogeneous Model Aggregation in Federated Learning

Meta-Learning across Meta-Tasks for Few-Shot Learning

A Joint Optimization Framework of the Embedding Model and Classifier for Meta-Learning

Personalized Federated Learning: A Clustered Distributed Co-Meta-Learning Approach