Abstract:Sarcasm detection is a significant challenge in sentiment analysis due to the nuanced and context-dependent nature of verbiage. We introduce Pragmatic Metacognitive Prompting (PMP) to improve the performance of Large Language Models (LLMs) in sarcasm detection, which leverages principles from pragmatics and reflection helping LLMs interpret implied meanings, consider contextual cues, and reflect on discrepancies to identify sarcasm. Using state-of-the-art LLMs such as LLaMA-3-8B, GPT-4o, and Claude 3.5 Sonnet, PMP achieves state-of-the-art performance on GPT-4o on MUStARD and SemEval2018. This study demonstrates that integrating pragmatic reasoning and metacognitive strategies into prompting significantly enhances LLMs' ability to detect sarcasm, offering a promising direction for future research in sentiment analysis.

What problem does this paper attempt to address?

The problem that this paper attempts to solve is: **Improve the performance of large language models (LLMs) in sarcasm detection**. Specifically, sarcasm detection is an important challenge in sentiment analysis because sarcastic language usually has subtle, context - dependent characteristics. Existing large language models perform poorly in sarcasm detection, especially when dealing with non - sequential and implicit meanings. To solve this problem, the author introduced a new prompting method - **Pragmatic Metacognitive Prompting (PMP)**. ### Main problems and solutions 1. **Limitations of existing methods**: - The performance of large language models on sarcasm detection tasks is not as good as that of specially trained Transformer - Encoder models. - The logical pipeline structure of LLMs contradicts the non - sequential nature of sarcasm, resulting in their poor performance in sarcasm detection. 2. **Proposal of PMP**: - **PMP combines pragmatics and reflection strategies** to help LLMs better understand implicit meanings, consider context clues, and reflect on inconsistencies to identify sarcasm. - By using state - of - the - art LLMs (such as LLaMA - 3 - 8B, GPT - 4o and Claude 3.5 Sonnet), PMP has achieved state - of - the - art performance on the MUStARD and SemEval2018 datasets. 3. **Specific implementation**: - **Preliminary analysis**: The LLM first analyzes the basic pragmatic factors of the given text (such as meaning, presupposition, intention, polarity, pretense, and the relationship between literal and implicit meanings). - **Reflection stage**: Another LLM is invoked to reflect on the preliminary analysis and finally generate a more accurate judgment. ### Summary This research shows that integrating pragmatic reasoning and metacognitive strategies into the prompting method can significantly improve the ability of LLMs in sarcasm detection, providing a promising direction for future sentiment analysis research.

Pragmatic Metacognitive Prompting Improves LLM Performance on Sarcasm Detection

SarcasmBench: Towards Evaluating Large Language Models on Sarcasm Understanding

Is Sarcasm Detection A Step-by-Step Reasoning Process in Large Language Models?

Metacognitive Prompting Improves Understanding in Large Language Models

An Evaluation of State-of-the-Art Large Language Models for Sarcasm Detection

Enhancing Cross-Lingual Sarcasm Detection by a Prompt Learning Framework with Data Augmentation and Contrastive Learning

Enhanced Semantic Representation Learning for Sarcasm Detection by Integrating Context-Aware Attention and Fusion Network

Evaluating statistical language models as pragmatic reasoners

Automated Sarcasm Recognition Using Applied Linguistics Driven Deep Learning With Large Language Model

Sarcasm in Sight and Sound: Benchmarking and Expansion to Improve Multimodal Sarcasm Detection

Prompt Design Matters for Computational Social Science Tasks but in Unpredictable Ways

RankPrompt: Step-by-Step Comparisons Make Language Models Better Reasoners

UPAR: A Kantian-Inspired Prompting Framework for Enhancing Large Language Model Capabilities

Sarcasm Analysis Using Conversation Context

Unlocking Structured Thinking in Language Models with Cognitive Prompting

PHAnToM: Persona-based Prompting Has An Effect on Theory-of-Mind Reasoning in Large Language Models

Question-Analysis Prompting Improves LLM Performance in Reasoning Tasks

Reading with Intent

Rethinking Pragmatics in Large Language Models: Towards Open-Ended Evaluation and Preference Tuning

Meta Reasoning for Large Language Models

Sememe knowledge and auxiliary information enhanced approach for sarcasm detection