Abstract:Sentiment classification is a form of data analytics where people's feelings and attitudes toward a topic are mined from data. This tantalizing power to "predict the zeitgeist" means that sentiment classification has long attracted interest, but with mixed results. However, the recent development of the BERT framework and its pretrained neural language models is seeing new-found success for sentiment classification. BERT models are trained to capture word-level information via mask language modeling and sentence-level contexts via next sentence prediction tasks. Out of the box, they are adequate models for some natural language processing tasks. However, most models are further fine-tuned with domain-specific information to increase accuracy and usefulness. Motivated by the idea that a further fine-tuning step would improve the performance for downstream sentiment classification tasks, we developed TopicBERT-a BERT model fine-tuned to recognize topics at the corpus level in addition to the word and sentence levels. TopicBERT comprises two variants: TopicBERT-ATP (aspect topic prediction), which captures topic information via an auxiliary training task, and TopicBERT-TA, where topic representation is directly injected into a topic augmentation layer for sentiment classification. With TopicBERT-ATP, the topics are predetermined by an LDA mechanism and collapsed Gibbs sampling. With TopicBERT-TA, the topics can change dynamically during the training. Experimental results show that both approaches deliver the state-of-the-art performance in two different domains with SemEval 2014 Task 4. However, in a test of methods, direct augmentation outperforms further training. Comprehensive analyses in the form of ablation, parameter, and complexity studies accompany the results.

Revealing the power of BERT for text sentiment classification

A BERT Framework to Sentiment Analysis of Tweets

Bidirectional Encoder Representations from Transformers (BERT): A sentiment analysis odyssey

T-BERT -- Model for Sentiment Analysis of Micro-blogs Integrating Topic Model and BERT

Research on sentiment classification for netizens based on the BERT-BiLSTM-TextCNN model

Adapting BERT for Target-Oriented Multimodal Sentiment Classification

TopicBERT: A Topic-Enhanced Neural Language Model Fine-Tuned for Sentiment Classification

Improving BERT-Based Text Classification With Auxiliary Sentence and Domain Knowledge

Fine-Tuning BERT Based Approach for Multi-Class Sentiment Analysis on Twitter Emotion Data

BERT-Based Sentiment Analysis: A Software Engineering Perspective

Research on the Application of Deep Learning-based BERT Model in Sentiment Analysis

Text Sentiment Analysis of Douban Film Short Comments Based on BERT-CNN-BiLSTM-Att Model

Transfer Learning for Sentiment Classification Using Bidirectional Encoder Representations from Transformers (BERT) Model

Sentiment recognition and analysis method of official document text based on BERT–SVM model

Incorporating Pre-trained Transformer Models into TextCNN for Sentiment Analysis on Software Engineering Texts

CM-BERT

Sentiment Analysis of Text Reviews Using Lexicon-Enhanced Bert Embedding (LeBERT) Model with Convolutional Neural Network

Sentiment Analysis About Investors and Consumers in Energy Market Based on BERT-BiLSTM

RoBERTa-BiLSTM: A Context-Aware Hybrid Model for Sentiment Analysis

A Study of Sentiment Analysis Algorithms for Agricultural Product Reviews Based on Improved BERT Model

Weibo Text Sentiment Analysis Based on BERT and Deep Learning