Abstract:Stance detection automatically detects the stance in a text towards a target, vital for content analysis in web and social media research. Despite their promising capabilities, LLMs encounter challenges when directly applied to stance detection. First, stance detection demands multi-aspect knowledge, from deciphering event-related terminologies to understanding the expression styles in social media platforms. Second, stance detection requires advanced reasoning to infer authors' implicit viewpoints, as stance are often subtly embedded rather than overtly stated in the text. To address these challenges, we design a three-stage framework COLA (short for Collaborative rOle-infused LLM-based Agents) in which LLMs are designated distinct roles, creating a collaborative system where each role contributes uniquely. Initially, in the multidimensional text analysis stage, we configure the LLMs to act as a linguistic expert, a domain specialist, and a social media veteran to get a multifaceted analysis of texts, thus overcoming the first challenge. Next, in the reasoning-enhanced debating stage, for each potential stance, we designate a specific LLM-based agent to advocate for it, guiding the LLM to detect logical connections between text features and stance, tackling the second challenge. Finally, in the stance conclusion stage, a final decision maker agent consolidates prior insights to determine the stance. Our approach avoids extra annotated data and model training and is highly usable. We achieve state-of-the-art performance across multiple datasets. Ablation studies validate the effectiveness of each design role in handling stance detection. Further experiments have demonstrated the explainability and the versatility of our approach. Our approach excels in usability, accuracy, effectiveness, explainability and versatility, highlighting its value.

Language Models Learn Metadata: Political Stance Detection Case Study

Examining the Influence of Political Bias on Large Language Model Performance in Stance Classification

Advancing Annotation of Stance in Social Media Posts: A Comparative Analysis of Large Language Models and Crowd Sourcing

Robust Stance Detection: Understanding Public Perceptions in Social Media

Stance Detection: A Practical Guide to Classifying Political Beliefs in Text

Automated stance detection in complex topics and small languages: The challenging case of immigration in polarizing news media

The Power of LLM-Generated Synthetic Data for Stance Detection in Online Political Discussions

Legal and Political Stance Detection of SCOTUS Language

Chain of Stance: Stance Detection with Large Language Models

A Survey of Stance Detection on Social Media: New Directions and Perspectives

Whose Side Are You On? Investigating the Political Stance of Large Language Models

Multilingual Multi-Target Stance Recognition in Online Public Consultations

Inducing Political Bias Allows Language Models Anticipate Partisan Reactions to Controversies

Prompting and Fine-Tuning Open-Sourced Large Language Models for Stance Classification

Stance Detection on Social Media with Background Knowledge

(Mis)alignment Between Stance Expressed in Social Media Data and Public Opinion Surveys

From Pretraining Data to Language Models to Downstream Tasks: Tracking the Trails of Political Biases Leading to Unfair NLP Models

Stance Detection on Social Media with Fine-Tuned Large Language Models

Stance Detection with Collaborative Role-Infused LLM-Based Agents

Metadata Might Make Language Models Better