Abstract:Unsupervised domain adaptation (UDA) aims to alleviate domain shifts by transferring relevant domain information from a fully labeled source domain to an unknown target domain. Although significant progress has been made recently, the use of UDA in real-world applications is still limited owing to low-resource computers and privacy issues. To further extend the flexibility of the target model, this study attempts to address a new setting of UDA, where only a source model is provided, and target models with various network architectures can be learned according to the deployment environment. Therefore, we propose a novel two-stage Cross-domain Knowledge Distillation method via Teacher-Student Mutual Learning, termed CdKD-TSML, which enables the peer networks to employ pseudo labels assigned by one another as supplemental supervision. In the first stage, to depress the inevitable noise in hard pseudo labels generated by the self-supervised clustering procedure, we further propose to softly refine the pseudo labels with mutual learning, where networks and label predictions are online optimized cooperatively by distilling knowledge from each other. In the second stage, we jointly fine-tune the distilled models via a cooperative consistency learning strategy, which selects pseudo-labeled samples from the two peers to update the networks, thereby enhancing the generalization of the models. We theoretically demonstrate that the target error of CdKD-TSML is minimized by simultaneously decreasing the pseudo-label noise and alleviating the sample selection bias. Experiments on three challenging UDA datasets demonstrate that CdKD-TSML yields superior results compared to other state-of-the-art methods, proving its effectiveness in this novel setting.

Adaptive Mutual Learning for Unsupervised Domain Adaptation

Class-Level Adaptation Network with Self Training for Unsupervised Domain Adaptation

Joint Domain Alignment and Discriminative Feature Learning for Unsupervised Deep Domain Adaptation

Attention-based Cross-Layer Domain Alignment for Unsupervised Domain Adaptation

Teacher-Student Mutual Learning for efficient source-free unsupervised domain adaptation

Multi-metric Domain Adaptation for Unsupervised Transfer Learning

Information Maximizing Adaptation Network With Label Distribution Priors for Unsupervised Domain Adaptation

Improving Domain-Adapted Sentiment Classification by Deep Adversarial Mutual Learning

Joint Semantic Domain Alignment and Target Classifier Learning for Unsupervised Domain Adaptation

Deep Joint Discriminative Feature Learning and Class-Aware Domain Alignment for Unsupervised Domain Adaptation

Learning to Adapt to Evolving Domains.

Self-adaptive Re-weighted Adversarial Domain Adaptation

Unsupervised Domain Adaptation Using Robust Class-Wise Matching

Learning Semantic Representations for Unsupervised Domain Adaptation.

Class Discriminative Adversarial Learning for Unsupervised Domain Adaptation

Unsupervised Domain Adaptation With Class-Aware Memory Alignment

Knowledge Adaptation: Teaching to Adapt

Unsupervised Domain Adaptation via Domain-Adaptive Diffusion

Unsupervised Domain Adaptation with Unified Joint Distribution Alignment

Learning Distribution-Matched Landmarks For Unsupervised Domain Adaptation

Dual-Level Adaptive and Discriminative Knowledge Transfer for Cross-Domain Recognition