Abstract:We propose learning sentiment-specific word embeddings dubbed sentiment embeddings in this paper. Existing word embedding learning algorithms typically only use the contexts of words but ignore the sentiment of texts. It is problematic for sentiment analysis because the words with similar contexts but opposite sentiment polarity, such as good and bad, are mapped to neighboring word vectors. We address this issue by encoding sentiment information of texts (e.g., sentences and words) together with contexts of words in sentiment embeddings. By combining context and sentiment level evidences, the nearest neighbors in sentiment embedding space are semantically similar and it favors words with the same sentiment polarity. In order to learn sentiment embeddings effectively, we develop a number of neural networks with tailoring loss functions, and collect massive texts automatically with sentiment signals like emoticons as the training data. Sentiment embeddings can be naturally used as word features for a variety of sentiment analysis tasks without feature engineering. We apply sentiment embeddings to word-level sentiment analysis, sentence level sentiment classification, and building sentiment lexicons. Experimental results show that sentiment embeddings consistently outperform context-based embeddings on several benchmark datasets of these tasks. This work provides insights on the design of neural networks for learning task-specific word embeddings in other natural language processing tasks.

Learning Bilingual Sentiment-Specific Word Embeddings without Cross-Lingual Supervision

Bilingual Word Embedding: A Survey from Supervised to Unsupervised

Bilingual Sentiment Embeddings: Joint Projection of Sentiment Across Languages

Learning Bilingual Embedding Model for Cross-Language Sentiment Classification

Exploring Implicit Semantic Constraints for Bilingual Word Embeddings

Don't Forget Cheap Training Signals Before Building Unsupervised Bilingual Word Embeddings

Unsupervised Cross-Lingual Sentence Representation Learning via Linguistic Isomorphism

Bilingual word embedding fusion for robust unsupervised bilingual lexicon induction

Jointly Learning Bilingual Word Embeddings and Alignments

Jointly Learning Bilingual Sentiment and Semantic Representations for Cross-Language Sentiment Classification.

Cross-Lingual Sentiment Classification with Bilingual Document Representation Learning

Adversarial Training for Unsupervised Bilingual Lexicon Induction

Joint Bilingual Sentiment Classification with Unlabeled Parallel Corpora.

Sentiment Embeddings with Applications to Sentiment Analysis

Learning Sentiment-Specific Word Embedding for Twitter Sentiment Classification.

Towards a Unified End-to-End Approach for Fully Unsupervised Cross-Lingual Sentiment Analysis.

Cross-Lingual Sentiment Lexicon Learning with Bilingual Word Graph Label Propagation

Unsupervised Bilingual Word Embedding Agreement for Unsupervised Neural Machine Translation

Improving Multilingual Sentence Embedding using Bi-directional Dual Encoder with Additive Margin Softmax

Beyond Bilingual: Multi-sense Word Embeddings using Multilingual Context

Towards a Universal Sentiment Classifier in Multiple Languages