Abstract:Recent studies have shown that prompt-based fine-tuning improves the performance of large Pre-trained Language Models (PLMs) for few-shot text classification. Specifically, this type of method transforms the text classification task into the inherent Masked Language Modeling (MLM) with some natural language prompts and verbalizers to bridge the gap between the pre-training stage and the fine-tuning stage. Yet, it is unclear how the prompting knowledge can be transferred across similar or distant NLP tasks, for the purpose of mutual reinforcement. Based on continuous prompt embeddings, we propose TransPrompt v2 , a novel transferable prompting framework for few-shot learning across similar or distant text classification tasks. For transferable prompt learning across similar tasks, we employ a multi-task meta-knowledge acquisition procedure to train a meta-learner that captures the cross-task transferable knowledge. For learning across distant tasks, we further introduce the task type descriptions and propose intra-type and inter-type prompt encoders to capture implicit mutual relations among multiple distant tasks. Additionally, two de-biasing techniques are further designed to make the trained meta-learner more task-agnostic and unbiased towards any tasks. After that, the meta-learner can be fine-tuned for specific tasks with better parameters initialization. Extensive experiments show that TransPrompt v2 outperforms single-task and cross-task strong baselines over multiple NLP tasks and datasets. We further show that the meta-learner can effectively improve the performance of PLMs on previously unseen tasks. In addition, TransPrompt v2 also outperforms strong fine-tuning baselines when learning with full training sets.

Prompt-Based Meta-Learning for Few-shot Text Classification.

When Low Resource NLP Meets Unsupervised Language Model: Meta-Pretraining then Meta-Learning for Few-Shot Text Classification (Student Abstract)

Self-supervised Meta-Prompt Learning with Meta-Gradient Regularization for Few-shot Generalization

Investigating Prompt Learning for Chinese Few-Shot Text Classification with Pre-Trained Language Models

Prompt-Based Graph Convolution Adversarial Meta-Learning for Few-Shot Text Classification

Prompt-Based Label-Aware Framework for Few-Shot Multi-Label Text Classification

STPrompt: Semantic-guided and Task-driven prompts for Effective Few-shot Classification

Knowledge-Enhanced Prompt Learning for Few-Shot Text Classification

Meta-Learning Adversarial Domain Adaptation Network for Few-Shot Text Classification.

Label-template based Few-Shot Text Classification with Contrastive Learning

Knowledge-Guided Prompt Learning for Few-Shot Text Classification

Towards Unified Prompt Tuning for Few-shot Text Classification

Meta-learning Siamese Network for Few-Shot Text Classification.

Improving Few-shot Text Classification via Pretrained Language Representations

TransPrompt - Towards an Automatic Transferable Prompting Framework for Few-shot Text Classification.

Ontology-enhanced Prompt-tuning for Few-shot Learning

Meta-Prompt Tuning Vision-Language Model for Multi-Label Few-Shot Image Recognition

TransPrompt V2: Transferable Prompt-based Fine-tuning for Few-shot Text Classification

Improving Meta-learning for Few-Shot Text Classification Via Label Propagation

PromptDA: Label-guided Data Augmentation for Prompt-based Few-shot Learners

Automatic Multi-Label Prompting: Simple and Interpretable Few-Shot Classification