Abstract:Current breakthroughs in natural language processing have benefited dramatically from neural language models, through which distributional semantics can leverage neural data representations to facilitate downstream applications. Since neural embeddings use context prediction on word co-occurrences to yield dense vectors, they are inevitably prone to capture more semantic association than semantic similarity. To improve vector space models in deriving semantic similarity, we post-process neural word embeddings through deep metric learning, through which we can inject lexical-semantic relations, including syn/antonymy and hypo/hypernymy, into a distributional space. We introduce hierarchy-fitting, a novel semantic specialization approach to modelling semantic similarity nuances inherently stored in the IS-A hierarchies. Hierarchy-fitting attains state-of-the-art results on the common- and rare-word benchmark datasets for deriving semantic similarity from neural word embeddings. It also incorporates an asymmetric distance function to specialize hypernymy's directionality explicitly, through which it significantly improves vanilla embeddings in multiple evaluation tasks of detecting hypernymy and directionality without negative impacts on semantic similarity judgement. The results demonstrate the efficacy of hierarchy-fitting in specializing neural embeddings with semantic relations in late fusion, potentially expanding its applicability to aggregating heterogeneous data and various knowledge resources for learning multimodal semantic spaces.

Multi-phase Word Sense Embedding Retrofitting with Lexical Ontology

Multi-phase Word Sense Embedding Learning Using a Corpus and a Lexical Ontology.

Do Multi-Sense Embeddings Improve Natural Language Understanding?

Learning Word Sense Embeddings from Word Sense Definitions

Learning Multi-Prototype Word Embedding from Single-Prototype Word Embedding with Integrated Knowledge.

Beyond Bilingual: Multi-sense Word Embeddings using Multilingual Context

Real Multi-Sense or Pseudo Multi-Sense: an Approach to Improve Word Representation

Learning Sense-specific Word Embeddings By Exploiting Bilingual Resources.

Using a Chinese Lexicon to Learn Sense Embeddings and Measure Semantic Similarity.

Constructing High Quality Sense-specific Corpus and Word Embedding Via Unsupervised Elimination of Pseudo Multi-sense.

On Modeling Sense Relatedness in Multi-prototype Word Embedding.

A Probabilistic Model for Learning Multi-Prototype Word Embeddings.

A knowledge-enriched ensemble method for word embedding and multi-sense embedding

Lexical semantics enhanced neural word embeddings

Multi-sense Definition Modeling using Word Sense Decompositions

Semantic Structure-Based Word Embedding by Incorporating Concept Convergence and Word Divergence

Leveraging Human Prior Knowledge to Learn Sense Representations

Understanding and Improving Multi-Sense Word Embeddings via Extended Robust Principal Component Analysis

Pre-Trained Multi-View Word Embedding Using Two-Side Neural Network

Together We Make Sense -- Learning Meta-Sense Embeddings from Pretrained Static Sense Embeddings

Modeling multi-prototype Chinese word representation learning for word similarity