Abstract:Prompt learning stands out as one of the most efficient approaches for adapting powerful vision-language foundational models like CLIP to downstream datasets by tuning learnable prompt vectors with very few samples. However, despite its success in achieving remarkable performance on in-domain data, prompt learning still faces the significant challenge of effectively generalizing to novel classes and domains. Some existing methods address this concern by dynamically generating distinct prompts for different domains. Yet, they overlook the inherent potential of prompts to generalize across unseen domains. To address these limitations, our study introduces an innovative prompt learning paradigm, called MetaPrompt, aiming to directly learn domain invariant prompt in few-shot scenarios. To facilitate learning prompts for image and text inputs independently, we present a dual-modality prompt tuning network comprising two pairs of coupled encoders. Our study centers on an alternate episodic training algorithm to enrich the generalization capacity of the learned prompts. In contrast to traditional episodic training algorithms, our approach incorporates both in-domain updates and domain-split updates in a batch-wise manner. For in-domain updates, we introduce a novel asymmetric contrastive learning paradigm, where representations from the pre-trained encoder assume supervision to regularize prompts from the prompted encoder. To enhance performance on out-of-domain distribution, we propose a domain-split optimization on visual prompts for cross-domain tasks or textual prompts for cross-class tasks during domain-split updates. Extensive experiments across 11 datasets for base-to-new generalization and 4 datasets for domain generalization exhibit favorable performance. Compared with the state-of-the-art method, MetaPrompt achieves an absolute gain of 1.02% on the overall harmonic mean in base-to-new generalization and consistently demonstrates superiority over all benchmarks in domain generalization.

TransPrompt V2: Transferable Prompt-based Fine-tuning for Few-shot Text Classification

TransPrompt v2: A Transferable Prompting Framework for Cross-task Text Classification

TransPrompt - Towards an Automatic Transferable Prompting Framework for Few-shot Text Classification.

On Transferability of Prompt Tuning for Natural Language Processing

Towards Unified Prompt Tuning for Few-shot Text Classification

Cross-Lingual Transfer for Natural Language Inference via Multilingual Prompt Translator

STPrompt: Semantic-guided and Task-driven prompts for Effective Few-shot Classification

AdaPrompt: Adaptive Model Training for Prompt-based NLP

Match-Prompt: Improving Multi-task Generalization Ability for Neural Text Matching via Prompt Learning

Hierarchical Prompt Tuning for Few-Shot Multi-Task Learning

BayesPrompt: Prompting Large-Scale Pre-Trained Language Models on Few-shot Inference via Debiased Domain Abstraction

Task-Adaptive Prompted Transformer for Cross-Domain Few-Shot Learning

Multitask Pre-training of Modular Prompt for Chinese Few-Shot Learning

Exploring Prompts in Few-Shot Cross-Linguistic Topic Classification Scenarios

MetaPrompting: Learning to Learn Better Prompts

Multi-Task Pre-Training of Modular Prompt for Few-Shot Learning

Multitask Prompt Tuning Enables Parameter-Efficient Transfer Learning

Learning Domain Invariant Prompt for Vision-Language Models

TriMPL: Masked Multi-Prompt Learning with Knowledge Mixing for Vision-Language Few-shot Learning

Dynamic Prompting: A Unified Framework for Prompt Tuning

Multitask Vision-Language Prompt Tuning