Abstract:Large Language Models (LLMs) have recently displayed their extraordinary capabilities in language understanding. However, how to comprehensively assess the sentiment capabilities of LLMs continues to be a challenge. This paper investigates the ability of LLMs to detect and react to sentiment in text modal. As the integration of LLMs into diverse applications is on the rise, it becomes highly critical to comprehend their sensitivity to emotional tone, as it can influence the user experience and the efficacy of sentiment-driven tasks. We conduct a series of experiments to evaluate the performance of several prominent LLMs in identifying and responding appropriately to sentiments like positive, negative, and neutral emotions. The models' outputs are analyzed across various sentiment benchmarks, and their responses are compared with human evaluations. Our discoveries indicate that although LLMs show a basic sensitivity to sentiment, there are substantial variations in their accuracy and consistency, emphasizing the requirement for further enhancements in their training processes to better capture subtle emotional cues. Take an example in our findings, in some cases, the models might wrongly classify a strongly positive sentiment as neutral, or fail to recognize sarcasm or irony in the text. Such misclassifications highlight the complexity of sentiment analysis and the areas where the models need to be refined. Another aspect is that different LLMs might perform differently on the same set of data, depending on their architecture and training datasets. This variance calls for a more in-depth study of the factors that contribute to the performance differences and how they can be optimized.

Using a Generative Model for Sentiment Analysis.

A Language Modeling Approach to Sentiment Analysis

A Method Based on Generation Models for Analyzing Sentiment-Topic in Texts

A Mixture Language Model for the Classification of Chinese Online Reviews.

Reflections on Sentiment/Opinion Analysis

SBTM: A Joint Sentiment and Behaviour Topic Model for Online Course Discussion Forums

SGMR: Sentiment-Aligned Generative Model for Reviews.

A Generative Model for Structured Sentiment Analysis.

Generative Sentiment Classification Model Affiliating Domain-Specific Sentiment Lexicons

What is Sentiment Meant to Mean to Language Models?

Fine-Grained Sentiment-Controlled Text Generation Approach Based on Pre-Trained Language Model

Mixture model-based subject-sentiment analysis in texts

Semi-supervised Learning with Generative Model for Sentiment Classification of Stock Messages

JST-RR Model: Joint Modeling of Ratings and Reviews in Sentiment-Topic Prediction

Do Large Language Models Possess Sensitive to Sentiment?

Modeling Sentiment and Aspect Using Syntax: A Topic Model Approach

Cross-Lingual Mixture Model for Sentiment Classification.

An effective hybrid model for opinion mining and sentiment analysis

Reducing Sentiment Bias in Language Models via Counterfactual Evaluation

Using Bilingual Lexicon to Judge Sentiment Orientation of Chinese Words

Language Model Based Chinese Financial News Sentiment Classification.