Abstract:Sentiment classification is a form of data analytics where people's feelings and attitudes toward a topic are mined from data. This tantalizing power to "predict the zeitgeist" means that sentiment classification has long attracted interest, but with mixed results. However, the recent development of the BERT framework and its pretrained neural language models is seeing new-found success for sentiment classification. BERT models are trained to capture word-level information via mask language modeling and sentence-level contexts via next sentence prediction tasks. Out of the box, they are adequate models for some natural language processing tasks. However, most models are further fine-tuned with domain-specific information to increase accuracy and usefulness. Motivated by the idea that a further fine-tuning step would improve the performance for downstream sentiment classification tasks, we developed TopicBERT-a BERT model fine-tuned to recognize topics at the corpus level in addition to the word and sentence levels. TopicBERT comprises two variants: TopicBERT-ATP (aspect topic prediction), which captures topic information via an auxiliary training task, and TopicBERT-TA, where topic representation is directly injected into a topic augmentation layer for sentiment classification. With TopicBERT-ATP, the topics are predetermined by an LDA mechanism and collapsed Gibbs sampling. With TopicBERT-TA, the topics can change dynamically during the training. Experimental results show that both approaches deliver the state-of-the-art performance in two different domains with SemEval 2014 Task 4. However, in a test of methods, direct augmentation outperforms further training. Comprehensive analyses in the form of ablation, parameter, and complexity studies accompany the results.

Optimization Techniques for Sentiment Analysis Based on LLM (GPT-3)

SentimentGPT: Exploiting GPT for Advanced Sentiment Analysis and its Departure from Current Machine Learning

Optimizing Performance: How Compact Models Match or Exceed GPT's Classification Capabilities through Fine-Tuning

Text Sentiment Analysis Based on Transformer and Augmentation

TopicBERT: A Topic-Enhanced Neural Language Model Fine-Tuned for Sentiment Classification

Instruct-FinGPT: Financial Sentiment Analysis by Instruction Tuning of General-Purpose Large Language Models

Using Large Language Models for sentiment analysis of health-related social media data: empirical evaluation and practical tips

Summary of ChatGPT-Related Research and Perspective Towards the Future of Large Language Models

Optimizing Multi-Task Learning for Enhanced Performance in Large Language Models

Enhancing Large Language Model Performance To Answer Questions and Extract Information More Accurately

A Comprehensive Capability Analysis of GPT-3 and GPT-3.5 Series Models

Research on the Application of Deep Learning-based BERT Model in Sentiment Analysis

A fine-tuning approach research of pre-trained model with two stage

Large language models for aspect-based sentiment analysis

Towards Optimal NLP Solutions: Analyzing GPT and LLaMA-2 Models Across Model Scale, Dataset Size, and Task Diversity

Fine-Tuning Large Language Models in Education

A Fine-grained Sentiment Analysis of App Reviews using Large Language Models: An Evaluation Study

From Artificial Needles to Real Haystacks: Improving Retrieval Capabilities in LLMs by Finetuning on Synthetic Data

Chinese Short-Text Sentiment Prediction: A Study of Progressive Prediction Techniques and Attentional Fine-Tuning

How Robust is GPT-3.5 to Predecessors? A Comprehensive Study on Language Understanding Tasks

From Text to Emotion: Unveiling the Emotion Annotation Capabilities of LLMs