Abstract:Current breakthroughs in natural language processing have benefited dramatically from neural language models, through which distributional semantics can leverage neural data representations to facilitate downstream applications. Since neural embeddings use context prediction on word co-occurrences to yield dense vectors, they are inevitably prone to capture more semantic association than semantic similarity. To improve vector space models in deriving semantic similarity, we post-process neural word embeddings through deep metric learning, through which we can inject lexical-semantic relations, including syn/antonymy and hypo/hypernymy, into a distributional space. We introduce hierarchy-fitting, a novel semantic specialization approach to modelling semantic similarity nuances inherently stored in the IS-A hierarchies. Hierarchy-fitting attains state-of-the-art results on the common- and rare-word benchmark datasets for deriving semantic similarity from neural word embeddings. It also incorporates an asymmetric distance function to specialize hypernymy's directionality explicitly, through which it significantly improves vanilla embeddings in multiple evaluation tasks of detecting hypernymy and directionality without negative impacts on semantic similarity judgement. The results demonstrate the efficacy of hierarchy-fitting in specializing neural embeddings with semantic relations in late fusion, potentially expanding its applicability to aggregating heterogeneous data and various knowledge resources for learning multimodal semantic spaces.

Improved Learning of Word Embeddings with Word Definitions and Semantic Injection.

Enhanced Double-Carrier Word Embedding Via Phonetics and Writing

Learning Word Sense Embeddings from Word Sense Definitions

Improved Learning of Chinese Word Embeddings with Semantic Knowledge.

Knowledge-Powered Deep Learning for Word Embedding

Learning Effective Word Embedding Using Morphological Word Similarity

An Exploration Of Semantic Relations In Neural Word Embeddings Using Extrinsic Knowledge

Improving Word Vector with Prior Knowledge in Semantic Dictionary.

Improving interpretability of word embeddings by generating definition and usage

Learning Word Embeddings from Intrinsic and Extrinsic Views

Learning word embeddings from dependency relations

Delta Embedding Learning

Exploiting external knowledge sources to improve kernel-based Word Sense Disambiguation

A knowledge-enriched ensemble method for word embedding and multi-sense embedding

Visually Aligned Word Embeddings for Improving Zero-shot Learning.

Learning Semantic Word Embeddings Based on Ordinal Knowledge Constraints.

Word embeddings: A semi-supervised learning method for slot-filling in spoken dialog systems

Improving Word Embeddings for Antonym Detection Using Thesauri and SentiWordNet.

Lexical semantics enhanced neural word embeddings

Learning Chinese Word Embeddings from Stroke, Structure and Pinyin of Characters

Learning Sense-specific Word Embeddings By Exploiting Bilingual Resources.