Abstract:Stance detection is an active task in natural language processing (NLP) that aims to identify the author's stance towards a particular target within a text. Given the remarkable language understanding capabilities and encyclopedic prior knowledge of large language models (LLMs), how to explore the potential of LLMs in stance detection has received significant attention. Unlike existing LLM-based approaches that focus solely on fine-tuning with large-scale datasets, we propose a new prompting method, called \textit{Chain of Stance} (CoS). In particular, it positions LLMs as expert stance detectors by decomposing the stance detection process into a series of intermediate, stance-related assertions that culminate in the final judgment. This approach leads to significant improvements in classification performance. We conducted extensive experiments using four SOTA LLMs on the SemEval 2016 dataset, covering the zero-shot and few-shot learning setups. The results indicate that the proposed method achieves state-of-the-art results with an F1 score of 79.84 in the few-shot setting.
What problem does this paper attempt to address?
The problem that this paper attempts to solve is how to utilize large - language models (LLMs) in natural language processing (NLP) to improve the performance of stance detection. Specifically, the paper proposes a new prompting method, called "Chain of Stance (CoS)", which decomposes the stance - detection process into a series of intermediate, stance - related assertions, and finally forms a final judgment. This method aims to significantly improve classification performance.
### Background of the Paper
With the rapid development of social media platforms such as X, WeChat, and TikTok, the amount of user - generated content has surged, and it has become particularly important to automatically analyze opinions, emotions, and stances in texts. Understanding the public's stance tendencies is of great significance for political analysis, public opinion polls, and rumor detection, etc. Therefore, stance detection has become an important topic in NLP.
### Stance - Detection Task
The goal of stance detection is to identify the author's attitude towards a specific target (such as an entity, concept, or event), and it is usually divided into three categories: support, oppose, or neutral.
### Application of Large - Language Models (LLMs)
In recent years, large - language models (such as ChatGPT, GPT - 4, Qwen, etc.) have performed excellently in multiple NLP tasks. Their powerful language - understanding and reasoning abilities give them great potential in stance - detection tasks. However, the existing LLMs - based methods mainly focus on fine - tuning on large - scale datasets, which requires high computational and time costs.
### Proposed Method: Chain of Stance (CoS)
To fill this gap, the paper proposes a new stance - detection method - Chain of Stance (CoS). This method utilizes the encyclopedic prior knowledge of LLMs, by decomposing the stance - detection process into a series of intermediate, stance - related assertions, and finally forms a final judgment. This method not only improves classification performance but also enhances the model's interpretability and transparency.
### Specific Steps
1. **Understand Contextual Information**: Given text \(S\), understand the contextual information \(I\) of the text, including the topic, author's identity, target audience, and relevant sociocultural background.
\[
I=\arg\max p(i | S, t)
\]
2. **Parse Main Views**: Based on \(S\), \(t\), and \(i\), parse the main views \(V\) of the text.
\[
V = \arg\max p(v | S, t, i)
\]
3. **Analyze Linguistic Expressions and Emotional Attitudes**: Analyze the linguistic expressions and emotional tendencies \(E\) of the text, identify emotional words and rhetorical devices, and analyze the author's tone.
\[
E=\arg\max p(e | S, t, i, v)
\]
4. **Compare Stance Similarities and Contrasts**: Compare the text \(S\) with three possible stances (support, oppose, neutral), and calculate the probability of each stance.
\[
A = \{ \text{support}: P(\text{support} | S, t, i, v, e), \text{oppose}: P(\text{oppose} | S, t, i, v, e), \text{neutral}: P(\text{neutral} | S, t, i, v, e) \}
\]
5. **Confirm the Consistency and Rationality of Stances**: Based on the context and other relevant information, conduct logical reasoning to confirm the consistency and rationality of stances.
\[
L=\arg\max p(l | S, t, i, v, e, a)
\]
6. **Make a Final Decision**: Based on the above information, determine the stance polarity towards the target \(t\).
\[
\hat{y}=\arg\max p(y | S, t, i, v, e, a, l)
\]
### Experimental Results
The paper conducted zero - shot and few - shot experiments on the SemEval 2016 dataset, using four state - of - the - art LLMs (Mistral - 7B, Qwen 1.5 - 7B, LLaMA 3 - 8B, LLaMA 2)