Abstract:Recent studies have shown that prompt-based fine-tuning improves the performance of large Pre-trained Language Models (PLMs) for few-shot text classification. Specifically, this type of method transforms the text classification task into the inherent Masked Language Modeling (MLM) with some natural language prompts and verbalizers to bridge the gap between the pre-training stage and the fine-tuning stage. Yet, it is unclear how the prompting knowledge can be transferred across similar or distant NLP tasks, for the purpose of mutual reinforcement. Based on continuous prompt embeddings, we propose TransPrompt v2 , a novel transferable prompting framework for few-shot learning across similar or distant text classification tasks. For transferable prompt learning across similar tasks, we employ a multi-task meta-knowledge acquisition procedure to train a meta-learner that captures the cross-task transferable knowledge. For learning across distant tasks, we further introduce the task type descriptions and propose intra-type and inter-type prompt encoders to capture implicit mutual relations among multiple distant tasks. Additionally, two de-biasing techniques are further designed to make the trained meta-learner more task-agnostic and unbiased towards any tasks. After that, the meta-learner can be fine-tuned for specific tasks with better parameters initialization. Extensive experiments show that TransPrompt v2 outperforms single-task and cross-task strong baselines over multiple NLP tasks and datasets. We further show that the meta-learner can effectively improve the performance of PLMs on previously unseen tasks. In addition, TransPrompt v2 also outperforms strong fine-tuning baselines when learning with full training sets.

ConsPrompt: Easily Exploiting Contrastive Samples for Few-shot Prompt Learning

ConsPrompt: Exploiting Contrastive Samples for Fewshot Prompt Learning

STPrompt: Semantic-guided and Task-driven prompts for Effective Few-shot Classification

Learning from Contrastive Prompts: Automated Optimization and Adaptation

ConKgPrompt: Contrastive Sample Method Based on Knowledge-Guided Prompt Learning for Text Classification

Unified Prompt Learning Makes Pre-Trained Language Models Better Few-Shot Learners

Making Pre-trained Language Models End-to-end Few-shot Learners with Contrastive Prompt Tuning

Instance-aware Prompt Learning for Language Understanding and Generation

Self-supervised Meta-Prompt Learning with Meta-Gradient Regularization for Few-shot Generalization

Consistency-guided Prompt Learning for Vision-Language Models

TransPrompt V2: Transferable Prompt-based Fine-tuning for Few-shot Text Classification

AdaPrompt: Adaptive Model Training for Prompt-based NLP

MetaPrompting: Learning to Learn Better Prompts

PREFER: Prompt Ensemble Learning via Feedback-Reflect-Refine

Prompting through Prototype: A Prototype-based Prompt Learning on Pretrained Vision-Language Models

Multi-Task Pre-Training of Modular Prompt for Few-Shot Learning

HybridPrompt: Domain-Aware Prompting for Cross-Domain Few-Shot Learning

TransPrompt - Towards an Automatic Transferable Prompting Framework for Few-shot Text Classification.

Multitask Pre-training of Modular Prompt for Chinese Few-Shot Learning

StablePT: Towards Stable Prompting for Few-shot Learning via Input Separation