Abstract:Recently, despite the unprecedented success of large pre-trained visual-language models (VLMs) on a wide range of downstream tasks, the real-world unsupervised domain adaptation (UDA) problem is still not well explored. Therefore, in this paper, we first experimentally demonstrate that the unsupervised-trained VLMs can significantly reduce the distribution discrepancy between source and target domains, thereby improving the performance of UDA. However, a major challenge for directly deploying such models on downstream UDA tasks is prompt engineering, which requires aligning the domain knowledge of source and target domains, since the performance of UDA is severely influenced by a good domain-invariant representation. We further propose a Prompt-based Distribution Alignment (PDA) method to incorporate the domain knowledge into prompt learning. Specifically, PDA employs a two-branch prompt-tuning paradigm, namely base branch and alignment branch. The base branch focuses on integrating class-related representation into prompts, ensuring discrimination among different classes. To further minimize domain discrepancy, for the alignment branch, we construct feature banks for both the source and target domains and propose image-guided feature tuning (IFT) to make the input attend to feature banks, which effectively integrates self-enhanced and cross-domain features into the model. In this way, these two branches can be mutually promoted to enhance the adaptation of VLMs for UDA. We conduct extensive experiments on three benchmarks to demonstrate that our proposed PDA achieves state-of-the-art performance. The code is available at <a class="link-external link-https" href="https://github.com/BaiShuanghao/Prompt-based-Distribution-Alignment" rel="external noopener nofollow">this https URL</a>.

Few-Shot Domain Adaptation Via Prompt-Guided Multi-prototype Alignment Network

Domain consensual contrastive learning for few-shot universal domain adaptation

Federated Domain Adaptation via Pseudo-label Refinement

EMPL: A novel Efficient Meta Prompt Learning Framework for Few-shot Unsupervised Domain Adaptation

Domain Adaptation via Prompt Learning

Prototypical Cross-domain Self-supervised Learning for Few-shot Unsupervised Domain Adaptation

Multi-Prompt Alignment for Multi-Source Unsupervised Domain Adaptation

Confidence-based Visual Dispersal for Few-shot Unsupervised Domain Adaptation

Prompt-based Distribution Alignment for Unsupervised Domain Adaptation

Few-Shot Unsupervised Domain Adaptation Via Meta Learning

Source-free Domain Adaptation Via Dynamic Pseudo Labeling and Self-supervision

Few-shot Domain Adaptation for Semantic Segmentation.

Progressive Feature Alignment for Unsupervised Domain Adaptation

Domain-Adaptive Few-Shot Learning

Prototype-Augmented Contrastive Learning for Few-Shot Unsupervised Domain Adaptation.

GrabDAE: An Innovative Framework for Unsupervised Domain Adaptation Utilizing Grab-Mask and Denoise Auto-Encoder

Enhancing Domain Adaptation through Prompt Gradient Alignment

Domain-Agnostic Mutual Prompting for Unsupervised Domain Adaptation

Semi-Supervised Domain Adaptation with Prototypical Alignment and Consistency Learning

Data-Efficient CLIP-Powered Dual-Branch Networks for Source-Free Unsupervised Domain Adaptation