Abstract:In this work, we take a deeper look into the diverse factors that influence the efficacy of modern unsupervised domain adaptation (UDA) methods using a large-scale, controlled empirical study. To facilitate our analysis, we first develop UDA-Bench, a novel PyTorch framework that standardizes training and evaluation for domain adaptation enabling fair comparisons across several UDA methods. Using UDA-Bench, our comprehensive empirical study into the impact of backbone architectures, unlabeled data quantity, and pre-training datasets reveals that: (i) the benefits of adaptation methods diminish with advanced backbones, (ii) current methods underutilize unlabeled data, and (iii) pre-training data significantly affects downstream adaptation in both supervised and self-supervised settings. In the context of unsupervised adaptation, these observations uncover several novel and surprising properties, while scientifically validating several others that were often considered empirical heuristics or practitioner intuitions in the absence of a standardized training and evaluation framework. The UDA-Bench framework and trained models are publicly available at <a class="link-external link-https" href="https://github.com/ViLab-UCSD/UDABench_ECCV2024" rel="external noopener nofollow">this https URL</a>.

What problem does this paper attempt to address?

### Problems the paper attempts to solve This paper aims to conduct an in - depth exploration of various factors affecting the effectiveness of modern Unsupervised Domain Adaptation (UDA) methods through large - scale, controlled empirical research. Specifically, the paper develops a new PyTorch framework named UDA - Bench, which standardizes the training and evaluation processes of domain adaptation, thereby enabling a fair comparison between different UDA methods. Through this framework, the author conducts a comprehensive empirical study on the following three main factors: 1. **Backbone Architectures**: - Research on the performance of different backbone architectures (such as CNN, Vision Transformers, MLP, etc.) in domain adaptation. - Verify the compatibility of existing adaptation methods with these new backbone architectures. 2. **Amount of Unlabeled Data**: - Explore the efficiency of current UDA methods in utilizing unlabeled data in the target domain. - Analyze the impact of reducing the amount of unlabeled data on adaptation performance. 3. **Nature of Pre - training Data**: - Research the impact of the type of pre - training data (such as supervised pre - training and self - supervised pre - training) on downstream adaptation tasks. - Compare the effects of different pre - training datasets (such as ImageNet, CUB, Places - 205, etc.). ### Main Findings 1. **Selection of Backbone Architectures**: - Recent vision transformers (such as Swin, DeiT) perform excellently in cross - domain robustness, outperforming the traditional ResNet - 50. - However, integrating these advanced architectures into existing UDA methods will weaken the benefits of these methods, resulting in a change in relative rankings. 2. **Impact of the Amount of Unlabeled Data**: - Current UDA methods are inefficient in utilizing a large amount of unlabeled data, and the performance improvement is limited. - Reducing the amount of unlabeled data (by up to 75%) has little impact on the accuracy in the target domain (usually no more than 1%). 3. **Nature of Pre - training Data**: - Pre - training data has a significant impact on the performance of downstream adaptation tasks, but this impact differs between supervised pre - training and self - supervised pre - training. - In supervised pre - training, using data similar to the downstream task can significantly improve accuracy. - In self - supervised pre - training, object - centered pre - training data is suitable for object - centered tasks, and scene - centered pre - training data is suitable for scene - centered tasks. ### Conclusion Through this comprehensive empirical study, the author reveals several new insights into the practical application of UDA methods and scientifically validates some phenomena that were previously considered empirical intuitions. These findings help researchers identify opportunities for developing more effective adaptation algorithms in the future and guide practitioners to maximize the benefits of current UDA methods. The UDA - Bench framework and its training models have been publicly released to promote further understanding and improvement of UDA methods.

UDA-Bench: Revisiting Common Assumptions in Unsupervised Domain Adaptation Using a Standardized Framework

Attention-based Cross-Layer Domain Alignment for Unsupervised Domain Adaptation

Revisiting, Benchmarking and Understanding Unsupervised Graph Domain Adaptation

Deep Unsupervised Domain Adaptation: A Review of Recent Advances and Perspectives

SKADA-Bench: Benchmarking Unsupervised Domain Adaptation Methods with Realistic Validation

Incorporating Pre-training Data Matters in Unsupervised Domain Adaptation

Re-weighted Adversarial Adaptation Network for Unsupervised Domain Adaptation

Unsupervised Domain Adaptation Approach for Vision-Based Semantic Understanding of Bridge Inspection Scenes Without Manual Annotations

Online Unsupervised Domain Adaptation Via Reducing Inter- and Intra-Domain Discrepancies

A Collaborative Alignment Framework of Transferable Knowledge Extraction for Unsupervised Domain Adaptation

Deep Unsupervised Domain Adaptation for Time Series Classification: a Benchmark

Revisiting Deep Subspace Alignment for Unsupervised Domain Adaptation

Unveiling the Superior Paradigm: A Comparative Study of Source-Free Domain Adaptation and Unsupervised Domain Adaptation

Trust And Balance: Few Trusted Samples Pseudo-Labeling and Temperature Scaled Loss for Effective Source-Free Unsupervised Domain Adaptation

A Study of Unsupervised Evaluation Metrics for Practical and Automatic Domain Adaptation

Reducing Bi-Level Feature Redundancy for Unsupervised Domain Adaptation

NI-UDA: Graph Contrastive Domain Adaptation for Nonshared-and-Imbalanced Unsupervised Domain Adaptation

Unsupervised Domain Adaptation for Video Object Grounding with Cascaded Debiasing Learning

Universal Domain Adaptation from Foundation Models: A Baseline Study

Adversarial Unsupervised Domain Adaptation Via Real Data Augmentation and Uncertainty Penalization

Unsupervised Domain Adaptation for Anatomical Landmark Detection