Abstract:Domain generalization (DG) aims to generalize the knowledge learned from multiple source domains to unseen target domains. Existing DG techniques can be subsumed under two broad categories, i.e., domain-invariant representation learning and domain manipulation. Nevertheless, it is extremely difficult to explicitly augment or generate the unseen target data. And when source domain variety increases, developing a domain-invariant model by simply aligning more domain-specific information becomes more challenging. In this paper, we propose a simple yet effective method for domain generalization, named Knowledge Distillation based Domain-invariant Representation Learning (KDDRL), that learns domain-invariant representation while encouraging the model to maintain domain-specific features, which recently turned out to be effective for domain generalization. To this end, our method incorporates multiple auxiliary student models and one student leader model to perform a two-stage distillation. In the first-stage distillation, each domain-specific auxiliary student treats the ensemble of other auxiliary students' predictions as a target, which helps to excavate the domain-invariant representation. Also, we present an error removal module to prevent the transfer of faulty information by eliminating incorrect predictions compared to the true labels. In the second-stage distillation, the student leader model with domain-specific features combines the domain-invariant representation learned from the group of auxiliary students to make the final prediction. Extensive experiments and in-depth analysis on popular DG benchmark datasets demonstrate that our KDDRL significantly outperforms the current state-of-the-art methods.

Generalizable Label Distribution Learning

Label Distribution Learning by Exploiting Label Correlations.

Label Distribution Learning with High-Order Label Correlations.

Label Distribution Learning by Exploiting Label Distribution Manifold

Label Distribution Learning with Label Correlations on Local Samples

Label Distribution Learning by Exploiting Sample Correlations Locally.

Label Distribution Learning: A Local Collaborative Mechanism

Label Distribution Learning with Label-Specific Features.

Incomplete Label Distribution Learning by Exploiting Global Sample Correlation

Label Distribution Learning by Exploiting Feature-Label Correlations Locally

Knowledge Distillation-based Domain-invariant Representation Learning for Domain Generalization

Label Distribution Learning by Exploiting Fuzzy Label Correlation

Label Distribution Learning with Label Correlations Via Low-Rank Approximation

Two-stage Label Distribution Learning with Label-Independent Prediction Based on Label-Specific Features

Classification with Label Distribution Learning.

Label Distribution Learning Machine.

Imbalanced Label Distribution Learning.

Exploiting Multi-Label Correlation in Label Distribution Learning

Incomplete Label Distribution Learning Via Label Correlation Decomposition

Label Distribution Learning by Mining Local Label Correlations in Self-regulating Clusters Independent of Sample Distance

Label Enhancement for Label Distribution Learning