Abstract:Unsupervised domain adaptation (UDA) aims to alleviate domain shifts by transferring relevant domain information from a fully labeled source domain to an unknown target domain. Although significant progress has been made recently, the use of UDA in real-world applications is still limited owing to low-resource computers and privacy issues. To further extend the flexibility of the target model, this study attempts to address a new setting of UDA, where only a source model is provided, and target models with various network architectures can be learned according to the deployment environment. Therefore, we propose a novel two-stage Cross-domain Knowledge Distillation method via Teacher-Student Mutual Learning, termed CdKD-TSML, which enables the peer networks to employ pseudo labels assigned by one another as supplemental supervision. In the first stage, to depress the inevitable noise in hard pseudo labels generated by the self-supervised clustering procedure, we further propose to softly refine the pseudo labels with mutual learning, where networks and label predictions are online optimized cooperatively by distilling knowledge from each other. In the second stage, we jointly fine-tune the distilled models via a cooperative consistency learning strategy, which selects pseudo-labeled samples from the two peers to update the networks, thereby enhancing the generalization of the models. We theoretically demonstrate that the target error of CdKD-TSML is minimized by simultaneously decreasing the pseudo-label noise and alleviating the sample selection bias. Experiments on three challenging UDA datasets demonstrate that CdKD-TSML yields superior results compared to other state-of-the-art methods, proving its effectiveness in this novel setting.

Federated Domain Adaptation via Pseudo-label Refinement

A Framework for Self-Supervised Federated Domain Adaptation

MCKD: Mutually Collaborative Knowledge Distillation for Federated Domain Adaptation and Generalization

Attention-based Cross-Layer Domain Alignment for Unsupervised Domain Adaptation

Domain-Augmented Domain Adaptation

Source-free Domain Adaptation Via Dynamic Pseudo Labeling and Self-supervision

Domain Adaptation without Model Transferring

Generation, division and training: A promising method for source-free unsupervised domain adaptation

Pseudo-label refinement via hierarchical contrastive learning for source-free unsupervised domain adaptation

Uncertainty-Aware Pseudo-Label Filtering for Source-Free Unsupervised Domain Adaptation

Domain Adaptation Using Pseudo Labels

Unsupervised domain adaptation with weak source domain labels via bidirectional subdomain alignment

Dual Classifier Adaptation: Source-Free UDA via Adaptive Pseudo-Labels Learning

Universal Semi-Supervised Domain Adaptation by Mitigating Common-Class Bias

Label Disentangled Analysis for unsupervised visual domain adaptation

IDPL: Intra-subdomain adaptation adversarial learning segmentation method based on Dynamic Pseudo Labels

Teacher-Student Mutual Learning for efficient source-free unsupervised domain adaptation

Class-Incremental Unsupervised Domain Adaptation via Pseudo-Label Distillation

Adversarial Unsupervised Domain Adaptation with Conditional and Label Shift: Infer, Align and Iterate

A Free Lunch for Unsupervised Domain Adaptive Object Detection without Source Data

Federated Unsupervised Domain Adaptation for Face Recognition