Abstract:Sentiment classification is a form of data analytics where people's feelings and attitudes toward a topic are mined from data. This tantalizing power to "predict the zeitgeist" means that sentiment classification has long attracted interest, but with mixed results. However, the recent development of the BERT framework and its pretrained neural language models is seeing new-found success for sentiment classification. BERT models are trained to capture word-level information via mask language modeling and sentence-level contexts via next sentence prediction tasks. Out of the box, they are adequate models for some natural language processing tasks. However, most models are further fine-tuned with domain-specific information to increase accuracy and usefulness. Motivated by the idea that a further fine-tuning step would improve the performance for downstream sentiment classification tasks, we developed TopicBERT-a BERT model fine-tuned to recognize topics at the corpus level in addition to the word and sentence levels. TopicBERT comprises two variants: TopicBERT-ATP (aspect topic prediction), which captures topic information via an auxiliary training task, and TopicBERT-TA, where topic representation is directly injected into a topic augmentation layer for sentiment classification. With TopicBERT-ATP, the topics are predetermined by an LDA mechanism and collapsed Gibbs sampling. With TopicBERT-TA, the topics can change dynamically during the training. Experimental results show that both approaches deliver the state-of-the-art performance in two different domains with SemEval 2014 Task 4. However, in a test of methods, direct augmentation outperforms further training. Comprehensive analyses in the form of ablation, parameter, and complexity studies accompany the results.

Sentiment Analysis of Political Posts on Hong Kong Local Forums Using Fine-Tuned Mbert

An Analysis of the Emotional Evolution of Large-Scale Internet Public Opinion Events Based on the BERT-LDA Hybrid Model

Aspect-Level Sentiment Analysis Approach via BERT and Aspect Feature Location Model

TopicBERT: A Topic-Enhanced Neural Language Model Fine-Tuned for Sentiment Classification

Fine-Tuning BERT for Sentiment Analysis of Vietnamese Reviews

Fine-Tuning BERT Based Approach for Multi-Class Sentiment Analysis on Twitter Emotion Data

HinglishNLP: Fine-tuned Language Models for Hinglish Sentiment Detection

Public Sentiment Analysis of Social Security Emergencies Based on Feature Fusion Model of BERT and TextLevelGCN

Research on the Application of Deep Learning-based BERT Model in Sentiment Analysis

On Robustness and Bias Analysis of BERT-Based Relation Extraction

Aspect-Based Sentiment Analysis using Local Context Focus Mechanism with DeBERTa

Sentiment Analysis on Chinese Health Forums: A Preliminary Study of Different Language Models.

Weibo Text Sentiment Analysis Based on BERT and Deep Learning

A BERT Framework to Sentiment Analysis of Tweets

Utilizing BERT Intermediate Layers for Aspect Based Sentiment Analysis and Natural Language Inference

CM-BERT

Fine-tuning Pretrained Multilingual BERT Model for Indonesian Aspect-based Sentiment Analysis

Sentiment analysis classification system using hybrid BERT models

Research on sentiment classification for netizens based on the BERT-BiLSTM-TextCNN model

An improved aspect-category sentiment analysis model for text sentiment analysis based on RoBERTa

Financial Sentiment Analysis on News and Reports Using Large Language Models and FinBERT