Abstract: Text classification is one of the most imperative tasks in natural language processing (NLP). Recent advances with pre-trained language models (PLMs) have shown remarkable success on this task. However, the satisfying results obtained by PLMs heavily depend on the large amounts of task-specific labeled data, which may not be feasible in many application scenarios due to data access and privacy constraints. The recently-proposed prompt-based fine-tuning paradigm improves the performance of PLMs for few-shot text classification with task-specific templates. Yet, it is unclear how the prompting knowledge can be transferred across tasks, for the purpose of mutual reinforcement. We propose TransPrompt v2, a novel transferable prompting framework for few-shot learning across similar or distant text classification tasks. For learning across similar tasks, we employ a multi-task meta-knowledge acquisition (MMA) procedure to train a meta-learner that captures the cross-task transferable knowledge. For learning across distant tasks, we further inject the task type descriptions into the prompt, and capture the intra-type and inter-type prompt embeddings among multiple distant tasks. Additionally, two de-biasing techniques are further designed to make the trained meta-learner more task-agnostic and unbiased towards any tasks. After that, the meta-learner can be adapted to each specific task with better parameters initialization. Extensive experiments show that TransPrompt v2 outperforms single-task and cross-task strong baselines over multiple NLP tasks and datasets. We further show that the meta-learner can effectively improve the performance of PLMs on previously unseen tasks. In addition, TransPrompt v2 also outperforms strong fine-tuning baselines when learning with full training sets.

UniPCM: Universal Pre-trained Conversation Model with Task-aware Automatic Prompt

OmniDialog: An Omnipotent Pre-training Model for Task-Oriented Dialogue System

A Task-oriented Dialog Model with Task-progressive and Policy-aware Pre-training

MuDPT: Multi-modal Deep-symphysis Prompt Tuning for Large Pre-trained Vision-Language Models

UniDCP: Unifying Multiple Medical Vision-language Tasks via Dynamic Cross-modal Learnable Prompts

Helping Language Models Learn More: Multi-dimensional Task Prompt for Few-shot Tuning

Zero-shot Cross-lingual Transfer of Prompt-based Tuning with a Unified Multilingual Prompt

Making Pre-trained Language Models End-to-end Few-shot Learners with Contrastive Prompt Tuning

Multi-Prompt with Depth Partitioned Cross-Modal Learning

Unified Knowledge Prompt Pre-training for Customer Service Dialogues

Improving Task Generalization via Unified Schema Prompt

AdaPrompt: Adaptive Model Training for Prompt-based NLP

TransPrompt v2: A Transferable Prompting Framework for Cross-task Text Classification

TransPrompt V2: Transferable Prompt-based Fine-tuning for Few-shot Text Classification

Unified Vision and Language Prompt Learning

Mastering Robot Manipulation with Multimodal Prompts through Pretraining and Multi-task Fine-tuning

Multi-Task Pre-Training for Plug-and-Play Task-Oriented Dialogue System

MPrompt: Exploring Multi-level Prompt Tuning for Machine Reading Comprehension

Unified Multimodal Pre-training and Prompt-based Tuning for Vision-Language Understanding and Generation

Modality-invariant and Specific Prompting for Multimodal Human Perception Understanding