Abstract:Recently, despite the unprecedented success of large pre-trained visual-language models (VLMs) on a wide range of downstream tasks, the real-world unsupervised domain adaptation (UDA) problem is still not well explored. Therefore, in this paper, we first experimentally demonstrate that the unsupervised-trained VLMs can significantly reduce the distribution discrepancy between source and target domains, thereby improving the performance of UDA. However, a major challenge for directly deploying such models on downstream UDA tasks is prompt engineering, which requires aligning the domain knowledge of source and target domains, since the performance of UDA is severely influenced by a good domain-invariant representation. We further propose a Prompt-based Distribution Alignment (PDA) method to incorporate the domain knowledge into prompt learning. Specifically, PDA employs a two-branch prompt-tuning paradigm, namely base branch and alignment branch. The base branch focuses on integrating class-related representation into prompts, ensuring discrimination among different classes. To further minimize domain discrepancy, for the alignment branch, we construct feature banks for both the source and target domains and propose image-guided feature tuning (IFT) to make the input attend to feature banks, which effectively integrates self-enhanced and cross-domain features into the model. In this way, these two branches can be mutually promoted to enhance the adaptation of VLMs for UDA. We conduct extensive experiments on three benchmarks to demonstrate that our proposed PDA achieves state-of-the-art performance. The code is available at <a class="link-external link-https" href="https://github.com/BaiShuanghao/Prompt-based-Distribution-Alignment" rel="external noopener nofollow">this https URL</a>.

EMPL: A novel Efficient Meta Prompt Learning Framework for Few-shot Unsupervised Domain Adaptation

Domain Adaptation via Prompt Learning

Task-Adaptive Prompted Transformer for Cross-Domain Few-Shot Learning

Self-supervised Meta-Prompt Learning with Meta-Gradient Regularization for Few-shot Generalization

Revisiting Unsupervised Meta-Learning via the Characteristics of Few-Shot Tasks

HybridPrompt: Domain-Aware Prompting for Cross-Domain Few-Shot Learning

Meta-Learning Adversarial Domain Adaptation Network for Few-Shot Text Classification.

Multi-Prompt Alignment for Multi-Source Unsupervised Domain Adaptation

Federated Adaptive Prompt Tuning for Multi-Domain Collaborative Learning

Convert Cross-Domain Classification into Few-Shot Learning: A Unified Prompt-Tuning Framework for Unsupervised Domain Adaptation

Learning to Prompt Your Domain for Vision-Language Models

Domain-Agnostic Mutual Prompting for Unsupervised Domain Adaptation

Domain consensual contrastive learning for few-shot universal domain adaptation

Prototypical Cross-domain Self-supervised Learning for Few-shot Unsupervised Domain Adaptation

Enhancing Domain Adaptation through Prompt Gradient Alignment

Prompt-based Distribution Alignment for Unsupervised Domain Adaptation

CLIP2UDA: Making Frozen CLIP Reward Unsupervised Domain Adaptation in 3D Semantic Segmentation

Meta-Exploiting Frequency Prior for Cross-Domain Few-Shot Learning