Abstract:Large Language Models (LLMs) have recently displayed their extraordinary capabilities in language understanding. However, how to comprehensively assess the sentiment capabilities of LLMs continues to be a challenge. This paper investigates the ability of LLMs to detect and react to sentiment in text modal. As the integration of LLMs into diverse applications is on the rise, it becomes highly critical to comprehend their sensitivity to emotional tone, as it can influence the user experience and the efficacy of sentiment-driven tasks. We conduct a series of experiments to evaluate the performance of several prominent LLMs in identifying and responding appropriately to sentiments like positive, negative, and neutral emotions. The models' outputs are analyzed across various sentiment benchmarks, and their responses are compared with human evaluations. Our discoveries indicate that although LLMs show a basic sensitivity to sentiment, there are substantial variations in their accuracy and consistency, emphasizing the requirement for further enhancements in their training processes to better capture subtle emotional cues. Take an example in our findings, in some cases, the models might wrongly classify a strongly positive sentiment as neutral, or fail to recognize sarcasm or irony in the text. Such misclassifications highlight the complexity of sentiment analysis and the areas where the models need to be refined. Another aspect is that different LLMs might perform differently on the same set of data, depending on their architecture and training datasets. This variance calls for a more in-depth study of the factors that contribute to the performance differences and how they can be optimized.

A Comparative Study of Different Pre-trained Language Models for Sentiment Analysis of Human-Computer Negotiation Dialogue

Investigating the Impact of Pre-trained Language Models on Dialog Evaluation

The Model Arena for Cross-lingual Sentiment Analysis: A Comparative Study in the Era of Large Language Models

DialogLM: Pre-trained Model for Long Dialogue Understanding and Summarization

Sentiment Analysis through LLM Negotiations

DialogueBERT: A Self-Supervised Learning based Dialogue Pre-training Encoder

Comparative Study of Language Models on Cross-Domain Data with Model Agnostic Explainability

A Comparative Analysis of Pretrained Language Models for Text-to-Speech

DialogBench: Evaluating LLMs as Human-like Dialogue Systems

Emotion Classification In Software Engineering Texts: A Comparative Analysis of Pre-trained Transformers Language Models

ESG Sentiment Analysis: comparing human and language model performance including GPT

Fine-Tuning Language Models for Ethical Ambiguity: A Comparative Study of Alignment with Human Responses

NegotiationToM: A Benchmark for Stress-testing Machine Theory of Mind on Negotiation Surrounding

Performance evaluation of Reddit Comments using Machine Learning and Natural Language Processing methods in Sentiment Analysis

Fractal-Inspired Sentiment Analysis: Evaluation Of Large Language Models And Deep Learning Methods

Improving Dialog Systems for Negotiation with Personality Modeling

xDial-Eval: A Multilingual Open-Domain Dialogue Evaluation Benchmark

Sentiment Analysis in the Era of Large Language Models: A Reality Check

Do Large Language Models Possess Sensitive to Sentiment?

GLM-Dialog: Noise-tolerant Pre-training for Knowledge-grounded Dialogue Generation

Comparative Analysis of Libraries for the Sentimental Analysis