Abstract:Multi-emotion sentiment classification is a natural language processing (NLP) problem with valuable use cases on real-world data. We demonstrate that large-scale unsupervised language modeling combined with finetuning offers a practical solution to this task on difficult datasets, including those with label class imbalance and domain-specific context. By training an attention-based Transformer network (Vaswani et al. 2017) on 40GB of text (Amazon reviews) (McAuley et al. 2015) and fine-tuning on the training set, our model achieves a 0.69 F1 score on the SemEval Task 1:E-c multi-dimensional emotion classification problem (Mohammad et al. 2018), based on the Plutchik wheel of emotions (Plutchik 1979). These results are competitive with state of the art models, including strong F1 scores on difficult (emotion) categories such as Fear (0.73), Disgust (0.77) and Anger (0.78), as well as competitive results on rare categories such as Anticipation (0.42) and Surprise (0.37). Furthermore, we demonstrate our application on a real world text classification task. We create a narrowly collected text dataset of real tweets on several topics, and show that our finetuned model outperforms general purpose commercially available APIs for sentiment and multidimensional emotion classification on this dataset by a significant margin. We also perform a variety of additional studies, investigating properties of deep learning architectures, datasets and algorithms for achieving practical multidimensional sentiment classification. Overall, we find that unsupervised language modeling and finetuning is a simple framework for achieving high quality results on real-world sentiment classification.

Leveraging Sparse and Dense Feature Combinations for Sentiment Classification

Audio Sentiment Analysis by Heterogeneous Signal Features Learned from Utterance-Based Parallel Neural Network.

Utterance-Based Audio Sentiment Analysis Learned by a Parallel Combination of CNN and LSTM.

Sentiment Analysis Using Deep Robust Complementary Fusion of Multi-Features and Multi-Modalities.

Sentiment Lexicon Enhanced Neural Sentiment Classification

Sentiment Lexicon Enhanced Attention-Based LSTM for Sentiment Classification

Leveraging Multi-grained Sentiment Lexicon Information for Neural Sequence Models

Multi-feature fusion and dual-channel networks for sentiment analysis

Leveraging multiple features for document sentiment classification

Sentiment Commonsense Induced Sequential Neural Networks for Sentiment Classification.

Transfer Pretrained Sentence Encoder to Sentiment Classification.

Sequential Multi-Kernel Convolutional Recurrent Network for Sentiment Classification

Aspect-Level Sentiment Analysis Based on Deep Learning

Practical Text Classification With Large Pre-Trained Language Models

Leveraging distant supervision and deep learning for twitter sentiment and emotion classification

Combining Vector Space Features and Convolution Neural Network for Text Sentiment Analysis.

Sentiment Embeddings with Applications to Sentiment Analysis

Neuro or Symbolic? Fine-tuned Transformer with Unsupervised LDA Topic Clustering for Text Sentiment Analysis

Linguistically Regularized LSTM for Sentiment Classification

A Multi-sentiment-resource Enhanced Attention Network for Sentiment Classification

Lightweight Scheme to Capture Stock Market Sentiment on Social Media Using Sparse Attention Mechanism: A Case Study on Twitter