Abstract:Learning a distinct representation for each sense of an ambiguous word could lead to more powerful and fine-grained models of vector-space representations. Yet while `multi-sense' methods have been proposed and tested on artificial word-similarity tasks, we don't know if they improve real natural language understanding tasks. In this paper we introduce a multi-sense embedding model based on Chinese Restaurant Processes that achieves state of the art performance on matching human word similarity judgments, and propose a pipelined architecture for incorporating multi-sense embeddings into language understanding. We then test the performance of our model on part-of-speech tagging, named entity recognition, sentiment analysis, semantic relation identification and semantic relatedness, controlling for embedding dimensionality. We find that multi-sense embeddings do improve performance on some tasks (part-of-speech tagging, semantic relation identification, semantic relatedness) but not on others (named entity recognition, various forms of sentiment analysis). We discuss how these differences may be caused by the different role of word sense information in each of the tasks. The results highlight the importance of testing embedding models in real applications.

Integrating Word Embeddings and Traditional NLP Features to Measure Textual Entailment and Semantic Relatedness of Sentence Pairs

Improve Word Embedding Using Both Writing and Pronunciation.

Do Multi-Sense Embeddings Improve Natural Language Understanding?

Chinese Textual Entailment Recognition Enhanced With Word Embedding

An Exploration Of Semantic Relations In Neural Word Embeddings Using Extrinsic Knowledge

Task-oriented Word Embedding for Text Classification.

Embedding WordNet Knowledge for Textual Entailment.

ECNU: Using Traditional Similarity Measurements and Word Embedding for Semantic Textual Similarity Estimation.

Revisiting Embedding Features for Simple Semi-supervised Learning.

ECNU: Leveraging Word Embeddings to Boost Performance for Paraphrase in Twitter.

Leveraging Pattern Associations For Word Embedding Models

Enhanced Word Embedding Method in Text Classification

The Expressive Power of Word Embeddings

A Survey of Word Embeddings Based on Deep Learning

Ontology Matching With Word Embeddings

ECNU: One Stone Two Birds: Ensemble of Heterogenous Measures for Semantic Relatedness and Textual Entailment

Extending Embedding Representation by Incorporating Latent Relations.

Training and Evaluating Improved Dependency-Based Word Embeddings.

Study on the Chinese Word Semantic Relation Classification with Word Embedding.

Learning Sentiment-Inherent Word Embedding for Word-Level and Sentence-Level Sentiment Analysis

Learning word embeddings from dependency relations