Abstract:Current breakthroughs in natural language processing have benefited dramatically from neural language models, through which distributional semantics can leverage neural data representations to facilitate downstream applications. Since neural embeddings use context prediction on word co-occurrences to yield dense vectors, they are inevitably prone to capture more semantic association than semantic similarity. To improve vector space models in deriving semantic similarity, we post-process neural word embeddings through deep metric learning, through which we can inject lexical-semantic relations, including syn/antonymy and hypo/hypernymy, into a distributional space. We introduce hierarchy-fitting, a novel semantic specialization approach to modelling semantic similarity nuances inherently stored in the IS-A hierarchies. Hierarchy-fitting attains state-of-the-art results on the common- and rare-word benchmark datasets for deriving semantic similarity from neural word embeddings. It also incorporates an asymmetric distance function to specialize hypernymy's directionality explicitly, through which it significantly improves vanilla embeddings in multiple evaluation tasks of detecting hypernymy and directionality without negative impacts on semantic similarity judgement. The results demonstrate the efficacy of hierarchy-fitting in specializing neural embeddings with semantic relations in late fusion, potentially expanding its applicability to aggregating heterogeneous data and various knowledge resources for learning multimodal semantic spaces.

Improving Word Vector with Prior Knowledge in Semantic Dictionary.

Learning Effective Word Embedding Using Morphological Word Similarity

Word vector computation based on implicit expression

An Exploration Of Semantic Relations In Neural Word Embeddings Using Extrinsic Knowledge

Lexical semantics enhanced neural word embeddings

Improved Learning of Chinese Word Embeddings with Semantic Knowledge.

Improved Learning of Word Embeddings with Word Definitions and Semantic Injection.

Improving Word Embeddings for Antonym Detection Using Thesauri and SentiWordNet.

Improving Word Vector Model with Part-of-speech and Dependency Grammar Information

Co-learning of Word Representations and Morpheme Representations.

Morph-fitting: Fine-Tuning Word Vector Spaces with Simple Language-Specific Rules

Knowledge-Powered Deep Learning for Word Embedding

Research on Chinese Semantic Similarity Algorithm

KNET: A General Framework for Learning Word Embedding using Morphological Knowledge

Word Vector Enrichment of Low Frequency Words in the Bag-of-Words Model for Short Text Multi-class Classification Problems

Improving Vector Space Word Representations Via Kernel Canonical Correlation Analysis

Enhancing Semantic Word Representations by Embedding Deeper Word Relationships

Exploiting external knowledge sources to improve kernel-based Word Sense Disambiguation

Adapting Word Embeddings to New Languages with Morphological and Phonological Subword Representations

An Improved Historical Embedding without Alignment

Deriving Word Vectors from Contextualized Language Models using Topic-Aware Mention Selection