Abstract:Prompt learning is a new learning paradigm which reformulates downstream tasks as similar pretraining tasks on pretrained models by leveraging textual prompts. Recent works have demonstrated that prompt learning is particularly useful for few-shot learning, where there is limited training data. Depending on the granularity of prompts, those methods can be roughly divided into task-level prompting and instance-level prompting. Task-level prompting methods learn one universal prompt for all input samples, which is efficient but ineffective to capture subtle differences among different classes. Instance-level prompting methods learn a specific prompt for each input, though effective but inefficient. In this work, we develop a novel prototype-based prompt learning method to overcome the above limitations. In particular, we focus on few-shot image recognition tasks on pretrained vision-language models (PVLMs) and develop a method of prompting through prototype (PTP), where we define $K$ image prototypes and $K$ prompt prototypes. In PTP, the image prototype represents a centroid of a certain image cluster in the latent space and a prompt prototype is defined as a soft prompt in the continuous space. The similarity between a query image and an image prototype determines how much this prediction relies on the corresponding prompt prototype. Hence, in PTP, similar images will utilize similar prompting ways. Through extensive experiments on seven real-world benchmarks, we show that PTP is an effective method to leverage the latent knowledge and adaptive to various PVLMs. Moreover, through detailed analysis, we discuss pros and cons for prompt learning and parameter-efficient fine-tuning under the context of few-shot learning.

Unified Prompt Learning Makes Pre-Trained Language Models Better Few-Shot Learners

Towards Unified Prompt Tuning for Few-shot Text Classification

What Makes Pre-trained Language Models Better Zero/Few-shot Learners?

Instance-aware Prompt Learning for Language Understanding and Generation

Multi-Task Pre-Training of Modular Prompt for Few-Shot Learning

Hierarchical Prompt Tuning for Few-Shot Multi-Task Learning

Adversarial Knowledge Stimulated Contrastive Prompting for Few-shot Language Learners

Prompting through Prototype: A Prototype-based Prompt Learning on Pretrained Vision-Language Models

Multitask Pre-training of Modular Prompt for Chinese Few-Shot Learning

Ontology-enhanced Prompt-tuning for Few-shot Learning

PPT: Pre-trained Prompt Tuning for Few-shot Learning

Cutting Down on Prompts and Parameters: Simple Few-Shot Learning with Language Models

Self-supervised Meta-Prompt Learning with Meta-Gradient Regularization for Few-shot Generalization

StablePT: Towards Stable Prompting for Few-shot Learning via Input Separation

STPrompt: Semantic-guided and Task-driven prompts for Effective Few-shot Classification

Unified Vision and Language Prompt Learning

AdaPrompt: Adaptive Model Training for Prompt-based NLP

ConsPrompt: Exploiting Contrastive Samples for Fewshot Prompt Learning

Helping Language Models Learn More: Multi-dimensional Task Prompt for Few-shot Tuning

ConsPrompt: Easily Exploiting Contrastive Samples for Few-shot Prompt Learning

Consprompt: Exploiting Contrastive Samples for Few-Shot Prompt Learning