Abstract:Unsupervised domain adaptation (UDA) aims to alleviate domain shifts by transferring relevant domain information from a fully labeled source domain to an unknown target domain. Although significant progress has been made recently, the use of UDA in real-world applications is still limited owing to low-resource computers and privacy issues. To further extend the flexibility of the target model, this study attempts to address a new setting of UDA, where only a source model is provided, and target models with various network architectures can be learned according to the deployment environment. Therefore, we propose a novel two-stage Cross-domain Knowledge Distillation method via Teacher-Student Mutual Learning, termed CdKD-TSML, which enables the peer networks to employ pseudo labels assigned by one another as supplemental supervision. In the first stage, to depress the inevitable noise in hard pseudo labels generated by the self-supervised clustering procedure, we further propose to softly refine the pseudo labels with mutual learning, where networks and label predictions are online optimized cooperatively by distilling knowledge from each other. In the second stage, we jointly fine-tune the distilled models via a cooperative consistency learning strategy, which selects pseudo-labeled samples from the two peers to update the networks, thereby enhancing the generalization of the models. We theoretically demonstrate that the target error of CdKD-TSML is minimized by simultaneously decreasing the pseudo-label noise and alleviating the sample selection bias. Experiments on three challenging UDA datasets demonstrate that CdKD-TSML yields superior results compared to other state-of-the-art methods, proving its effectiveness in this novel setting.

Class-Incremental Unsupervised Domain Adaptation via Pseudo-Label Distillation

Federated Domain Adaptation via Pseudo-label Refinement

Source-free Domain Adaptation Via Dynamic Pseudo Labeling and Self-supervision

Multi-Granularity Class Prototype Topology Distillation for Class-Incremental Source-Free Unsupervised Domain Adaptation

Boosting Unsupervised Domain Adaptation with Soft Pseudo-label and Curriculum Learning

IDPL: Intra-subdomain adaptation adversarial learning segmentation method based on Dynamic Pseudo Labels

Class Discriminative Adversarial Learning for Unsupervised Domain Adaptation

Generation, division and training: A promising method for source-free unsupervised domain adaptation

Teacher-Student Mutual Learning for efficient source-free unsupervised domain adaptation

Dual Classifier Adaptation: Source-Free UDA via Adaptive Pseudo-Labels Learning

Universal Semi-Supervised Domain Adaptation by Mitigating Common-Class Bias

CLDA: Collaborative Learning for Enhanced Unsupervised Domain Adaptation

Domain Adaptation via Prompt Learning

Uncertainty-Aware Pseudo-Label Filtering for Source-Free Unsupervised Domain Adaptation

Domain-Augmented Domain Adaptation

Semi-Supervised Domain Adaptation with Prototypical Alignment and Consistency Learning

Learning Transferable Conceptual Prototypes for Interpretable Unsupervised Domain Adaptation

Exploiting multi-level consistency learning for source-free domain adaptation

Learning Target Domain Specific Classifier for Partial Domain Adaptation

Learning From a Complementary-Label Source Domain: Theory and Algorithms