Abstract:Sentiment classification is a form of data analytics where people's feelings and attitudes toward a topic are mined from data. This tantalizing power to "predict the zeitgeist" means that sentiment classification has long attracted interest, but with mixed results. However, the recent development of the BERT framework and its pretrained neural language models is seeing new-found success for sentiment classification. BERT models are trained to capture word-level information via mask language modeling and sentence-level contexts via next sentence prediction tasks. Out of the box, they are adequate models for some natural language processing tasks. However, most models are further fine-tuned with domain-specific information to increase accuracy and usefulness. Motivated by the idea that a further fine-tuning step would improve the performance for downstream sentiment classification tasks, we developed TopicBERT-a BERT model fine-tuned to recognize topics at the corpus level in addition to the word and sentence levels. TopicBERT comprises two variants: TopicBERT-ATP (aspect topic prediction), which captures topic information via an auxiliary training task, and TopicBERT-TA, where topic representation is directly injected into a topic augmentation layer for sentiment classification. With TopicBERT-ATP, the topics are predetermined by an LDA mechanism and collapsed Gibbs sampling. With TopicBERT-TA, the topics can change dynamically during the training. Experimental results show that both approaches deliver the state-of-the-art performance in two different domains with SemEval 2014 Task 4. However, in a test of methods, direct augmentation outperforms further training. Comprehensive analyses in the form of ablation, parameter, and complexity studies accompany the results.

Sentiment Analysis Using Bert Model

BERT-Based Sentiment Analysis: A Software Engineering Perspective

Research on the Application of Deep Learning-based BERT Model in Sentiment Analysis

Fine-grained Sentiment Classification using BERT

Sentiment analysis classification system using hybrid BERT models

A BERT Framework to Sentiment Analysis of Tweets

T-BERT -- Model for Sentiment Analysis of Micro-blogs Integrating Topic Model and BERT

TopicBERT: A Topic-Enhanced Neural Language Model Fine-Tuned for Sentiment Classification

Sentiment recognition and analysis method of official document text based on BERT–SVM model

Bidirectional Encoder Representations from Transformers (BERT): A sentiment analysis odyssey

BERT: A Review of Applications in Sentiment Analysis

Fine-Tuning BERT Based Approach for Multi-Class Sentiment Analysis on Twitter Emotion Data

Leveraging the BERT Model for Enhanced Sentiment Analysis in Multicontextual Social Media Content

Sentiment Analysis of Text Reviews Using Lexicon-Enhanced Bert Embedding (LeBERT) Model with Convolutional Neural Network

Optimizing Customer Satisfaction Through Sentiment Analysis: A BERT-Based Machine Learning Approach to Extract Insights

Detection of extreme sentiments on social networks with BERT

SemEval-2017 Task 4: Sentiment Analysis in Twitter using BERT

Aspect-Based Sentiment Analysis using Local Context Focus Mechanism with DeBERTa

Senti-BSAS: A BERT-based Classification Model with Sentiment Calculating for Happiness Research.

Improving Bert Model Accuracy for Uni-modal Aspect-Based Sentiment Analysis Task

Does BERT look at sentiment lexicon?