Abstract:Sentiment classification is a form of data analytics where people's feelings and attitudes toward a topic are mined from data. This tantalizing power to "predict the zeitgeist" means that sentiment classification has long attracted interest, but with mixed results. However, the recent development of the BERT framework and its pretrained neural language models is seeing new-found success for sentiment classification. BERT models are trained to capture word-level information via mask language modeling and sentence-level contexts via next sentence prediction tasks. Out of the box, they are adequate models for some natural language processing tasks. However, most models are further fine-tuned with domain-specific information to increase accuracy and usefulness. Motivated by the idea that a further fine-tuning step would improve the performance for downstream sentiment classification tasks, we developed TopicBERT-a BERT model fine-tuned to recognize topics at the corpus level in addition to the word and sentence levels. TopicBERT comprises two variants: TopicBERT-ATP (aspect topic prediction), which captures topic information via an auxiliary training task, and TopicBERT-TA, where topic representation is directly injected into a topic augmentation layer for sentiment classification. With TopicBERT-ATP, the topics are predetermined by an LDA mechanism and collapsed Gibbs sampling. With TopicBERT-TA, the topics can change dynamically during the training. Experimental results show that both approaches deliver the state-of-the-art performance in two different domains with SemEval 2014 Task 4. However, in a test of methods, direct augmentation outperforms further training. Comprehensive analyses in the form of ablation, parameter, and complexity studies accompany the results.

Ranking Based Multi-Label Classification for Sentiment Analysis

Multi-feature fusion and dual-channel networks for sentiment analysis

A multi-label classification based approach for sentiment classification

Text Sentiment Classification Based on BERT Embedding and Sliced Multi-Head Self-Attention Bi-GRU

A Multi-Classification Sentiment Analysis Model of Chinese Short Text Based on Gated Linear Units and Attention Mechanism

Fine-Tuning BERT for Multi-Label Sentiment Analysis in Unbalanced Code-Switching Text.

A Fusion Model for Multi-Label Emotion Classification Based on BERT and Topic Clustering

A Joint Segmentation and Classification Framework for Sentence Level Sentiment Classification

Sentiment analysis classification system using hybrid BERT models

Text multi-label sentiment analysis based on Bi-LSTM

Multi-Task Learning Model Based on Multi-Scale CNN and LSTM for Sentiment Classification

Fine-Tuning BERT Based Approach for Multi-Class Sentiment Analysis on Twitter Emotion Data

Research on sentiment classification for netizens based on the BERT-BiLSTM-TextCNN model

A Multi-Channel Text Sentiment Analysis Model Integrating Pre-training Mechanism

Domain Attention Model for Multi-Domain Sentiment Classification.

A Two-Level Classifier Model for Sentiment Analysis.

TopicBERT: A Topic-Enhanced Neural Language Model Fine-Tuned for Sentiment Classification

A Sentiment Classification Approach Based on Sentiment Sentence Framework

A Fine-Grained Modal Label-Based Multi-Stage Network for Multimodal Sentiment Analysis.

Research on Fine-Grained Sentiment Classification.

A BERT Framework to Sentiment Analysis of Tweets