A Novel Prompt-tuning Method: Incorporating Scenario-specific Concepts into a Verbalizer

Yong Ma,Senlin Luo,Yu-Ming Shang,Zhengjun Li,Yong Liu

2024-01-10

Abstract:The verbalizer, which serves to map label words to class labels, is an essential component of prompt-tuning. In this paper, we present a novel approach to constructing verbalizers. While existing methods for verbalizer construction mainly rely on augmenting and refining sets of synonyms or related words based on class names, this paradigm suffers from a narrow perspective and lack of abstraction, resulting in limited coverage and high bias in the label-word space. To address this issue, we propose a label-word construction process that incorporates scenario-specific concepts. Specifically, we extract rich concepts from task-specific scenarios as label-word candidates and then develop a novel cascade calibration module to refine the candidates into a set of label words for each class. We evaluate the effectiveness of our proposed approach through extensive experiments on {five} widely used datasets for zero-shot text classification. The results demonstrate that our method outperforms existing methods and achieves state-of-the-art results.

Computation and Language,Artificial Intelligence

What problem does this paper attempt to address?

The problem that this paper attempts to solve is to construct a more effective verbalizer in the prompt - tuning method. Existing methods for constructing verbalizers mainly rely on the enhancement and refinement of synonyms or related words based on category names. This method has the problems of narrow perspective and low level of abstraction, resulting in a limited coverage range and high deviation in the label word space. To solve these problems, the paper proposes a new method, that is, constructing a verbalizer by combining concepts in specific scenarios (the ISCV method), in order to increase the diversity and abstraction level of label words, thereby improving the performance of zero - shot text classification tasks.

Specifically, the ISCV method contains two main steps:
1. **Concept Mining**: Randomly select a set of samples from a specific task scenario, and extract relevant concepts based on these samples as a candidate set of label words.
2. **Cascade Calibration**: Remove irrelevant or invalid label word candidates through a novel cascade calibration module, and finally refine the set of label words for each category.

Through extensive experiments on five widely - used datasets, the results show that the ISCV method is superior to existing methods in zero - shot text classification tasks and reaches the state - of - the - art level. In addition, the ISCV method can also reduce the standard deviation of results between different prompt templates and improve the stability of experimental results.