Abstract:Multi-sense word embedding is an important extension of neural word embeddings. By leveraging context of each word instance, multi-prototype version of word embeddings were accomplished to represent the multi-senses. Unfortunately, this kind of context based approach inevitably produces multiple senses which should actually be a single one, suffering from the various context of a word. (Shi et al., 2016) used WordNet to evaluate the neighborhood similarity of each sense pair to detect such pseudo multi-senses. In this paper, a novel framework for unsupervised corpus sense tagging is presented, which mainly contains four steps: (a) train multi-sense word embeddings on the given corpus, using existing multi-sense word embedding frameworks; (b) detect pseudo multi-senses in the obtained embeddings, without requirement to any extra language resources; (c) label each word in the corpus with a specific sense tag, with respect to the result of pseudo multi-sense detection; (d) re-train multi-sense word embeddings with the pre-selected sense tags. We evaluate our framework by training word embeddings with the obtained sense specific corpus. On the tasks of word similarity, word analogy as well as sentence understanding, the embeddings trained on sense-specific corpus obtain better results than the basic strategy which is applied in step (a).

Real Multi-Sense or Pseudo Multi-Sense: an Approach to Improve Word Representation

Do Multi-Sense Embeddings Improve Natural Language Understanding?

Understanding and Improving Multi-Sense Word Embeddings via Extended Robust Principal Component Analysis

Constructing High Quality Sense-specific Corpus and Word Embedding Via Unsupervised Elimination of Pseudo Multi-sense.

Addressing the Polysemy Problem in Language Modeling with Attentional Multi-Sense Embeddings

Leveraging Human Prior Knowledge to Learn Sense Representations

Beyond Bilingual: Multi-sense Word Embeddings using Multilingual Context

On Modeling Sense Relatedness in Multi-prototype Word Embedding.

Multi-sense Definition Modeling using Word Sense Decompositions

Implicit Subjective and Sentimental Usages in Multi-sense Word Embeddings.

Multi-phase Word Sense Embedding Retrofitting with Lexical Ontology

A Probabilistic Model for Learning Multi-Prototype Word Embeddings.

Multi-phase Word Sense Embedding Learning Using a Corpus and a Lexical Ontology.

Modeling multi-prototype Chinese word representation learning for word similarity

Learning Sense-specific Word Embeddings By Exploiting Bilingual Resources.

Using Multi-Sense Vector Embeddings for Reverse Dictionaries

Discovering and Distinguishing Multiple Visual Senses for Polysemous Words

Context-Specific and Multi-Prototype Character Representations.

Learning Word Sense Embeddings from Word Sense Definitions

Semantic Representations of Word Senses and Concepts

Different Contexts Lead to Different Word Embeddings