Abstract:Sentiment classification is a form of data analytics where people's feelings and attitudes toward a topic are mined from data. This tantalizing power to "predict the zeitgeist" means that sentiment classification has long attracted interest, but with mixed results. However, the recent development of the BERT framework and its pretrained neural language models is seeing new-found success for sentiment classification. BERT models are trained to capture word-level information via mask language modeling and sentence-level contexts via next sentence prediction tasks. Out of the box, they are adequate models for some natural language processing tasks. However, most models are further fine-tuned with domain-specific information to increase accuracy and usefulness. Motivated by the idea that a further fine-tuning step would improve the performance for downstream sentiment classification tasks, we developed TopicBERT-a BERT model fine-tuned to recognize topics at the corpus level in addition to the word and sentence levels. TopicBERT comprises two variants: TopicBERT-ATP (aspect topic prediction), which captures topic information via an auxiliary training task, and TopicBERT-TA, where topic representation is directly injected into a topic augmentation layer for sentiment classification. With TopicBERT-ATP, the topics are predetermined by an LDA mechanism and collapsed Gibbs sampling. With TopicBERT-TA, the topics can change dynamically during the training. Experimental results show that both approaches deliver the state-of-the-art performance in two different domains with SemEval 2014 Task 4. However, in a test of methods, direct augmentation outperforms further training. Comprehensive analyses in the form of ablation, parameter, and complexity studies accompany the results.

ContextBert: Enhanced Implicit Sentiment Analysis Using Implicit-sentiment-query Attention

Implicit Sentiment Analysis of Chinese Texts based on Contextual Information and Knowledge Enhancement

Context-Guided BERT for Targeted Aspect-Based Sentiment Analysis

Implicit sentiment analysis based on multi-feature neural network model

A Mutual Implicit Sentiment Analysis Model with Bundle-Aware Contrastive Learning

BERT-ASC: Auxiliary-Sentence Construction for Implicit Aspect Learning in Sentiment Analysis

Chinese Implicit Sentiment Analysis based on Hybrid Neural Networks

Dynamic commonsense knowledge fused method for Chinese implicit sentiment analysis

SentiBERT: A Transferable Transformer-Based Architecture for Compositional Sentiment Semantics

Chinese Implicit Sentiment Analysis Based on Hierarchical Knowledge Enhancement and Multi-Pooling

Research on the Application of Deep Learning-based BERT Model in Sentiment Analysis

An Effective Contextual Language Modeling Framework for Speech Summarization with Augmented Features

Leveraging the BERT Model for Enhanced Sentiment Analysis in Multicontextual Social Media Content

Interactive capsule network for implicit sentiment analysis

TopicBERT: A Topic-Enhanced Neural Language Model Fine-Tuned for Sentiment Classification

Aspect-Level Sentiment Analysis Approach via BERT and Aspect Feature Location Model

A Syntax-based BSGCN Model for Chinese Implicit Sentiment Analysis with Multi-classification

CM-BERT

Aspect-Based Sentiment Analysis using Local Context Focus Mechanism with DeBERTa

Implicit Sentiment Analysis Based on Chain of Thought Prompting

Implicit sentiment analysis based on affective knowledge and event information