Abstract:Word embedding aims to learn a continuous representation for each word. It attracts increasing attention due to its effectiveness in various tasks such as named entity recognition and language modeling. Most existing word embedding results are generally trained on one individual data source such as news pages or Wikipedia articles. However, when we apply them to other tasks such as web search, the performance suffers. To obtain a robust word embedding for different applications, multiple data sources could be leveraged. In this paper, we proposed a two-side multimodal neural network to learn a robust word embedding from multiple data sources including free text, user search queries and search click-through data. This framework takes the word embeddings learned from different data sources as pre-train, and then uses a two-side neural network to unify these embeddings. The pre-trained embeddings are obtained by adapting the recently proposed CBOW algorithm. Since the proposed neural network does not need to re-train word embeddings for a new task, it is highly scalable in real world problem solving. Besides, the network allows weighting different sources differently when applied to different application tasks. Experiments on two real-world applications including web search ranking and word similarity measuring show that our neural network with multiple sources outperforms state-of-the-art word embedding algorithm with each individual source. It also outperforms other competitive baselines using multiple sources.

Learning Multi-Prototype Word Embedding from Single-Prototype Word Embedding with Integrated Knowledge.

A Probabilistic Model for Learning Multi-Prototype Word Embeddings.

Do Multi-Sense Embeddings Improve Natural Language Understanding?

On Modeling Sense Relatedness in Multi-prototype Word Embedding.

Going Beyond Multi-Task Dense Prediction with Synergy Embedding Models

Multi-phase Word Sense Embedding Retrofitting with Lexical Ontology

Context-Specific and Multi-Prototype Character Representations.

Multi-phase Word Sense Embedding Learning Using a Corpus and a Lexical Ontology.

Bridging Text and Knowledge with Multi-Prototype Embedding for Few-Shot Relational Triple Extraction.

Beyond Bilingual: Multi-sense Word Embeddings using Multilingual Context

Real Multi-Sense or Pseudo Multi-Sense: an Approach to Improve Word Representation

Learning Context-Specific Word/Character Embeddings.

Bridge Text and Knowledge by Learning Multi-Prototype Entity Mention Embedding

Constructing High Quality Sense-specific Corpus and Word Embedding Via Unsupervised Elimination of Pseudo Multi-sense.

Gaussian Mixture Embeddings for Multiple Word Prototypes.

Modeling multi-prototype Chinese word representation learning for word similarity

A knowledge-enriched ensemble method for word embedding and multi-sense embedding

Understanding and Improving Multi-Sense Word Embeddings via Extended Robust Principal Component Analysis

Learning Context-Sensitive Word Embeddings with Neural Tensor Skip-Gram Model

Pre-Trained Multi-View Word Embedding Using Two-Side Neural Network

Joint Learning of Character and Word Embeddings.