Abstract:It is widely accepted that traditional word embedding models, which rely on distributional semantics hypothesis, are relatively limited for contrast meaning problem. Distributional semantics hypothesis indicates that words lying in similar contexts have similar representations in vector space. Nevertheless, synonyms and antonyms often locate in similar contexts, which means they appear close to each other in vector space. Hence, it is of great difficulty to distinguish antonyms from synonyms. To address this challenge, we propose an optimization model, named Lexicon-based Word Embedding Tuning (LWET) model. The goal of LWET is to incorporate reliable semantic lexicons to tune the distributions of pre-trained word embeddings in the vector space so as to improve their ability of distinguishing antonyms from synonyms. To speed up the training process of LWET, we propose two approximation algorithms, including positive sampling and quasi-hierarchical softmax. Compared with quasi-hierarchical softmax, positive sampling is faster, however, at the cost of worse performance. In experiments, LWET and other state-of-the-art models are tested on antonyms recognition, distinguishing antonyms from synonyms and word similarity. The results of the first two experiments show that LWET significantly improves the ability of word embeddings to detect antonyms, thus achieving the state-of-the-art performance. On word similarity, LWET gets slightly better performance than the state-of-the-art models. It means that LWET can remain and strengthen the semantic structure rather than destroy it when tuning word distributions in vector space. In general, compared with related work, LWET can not only achieve similar or even better performance, but also speed up the training process.

Exploiting Multiple Resources For Word-Phrase Semantic Similarity Evaluation

Application-Oriented Comparison and Evaluation of Six Semantic Similarity Measures Based on Wordnet

A Large Probabilistic Semantic Network Based Approach to Compute Term Similarity

Measuring Semantic Similarity Between Words Based On Multiple Relational Information

Combining Multiple Resources to Improve SMT-based Paraphrasing Model

Automatic Extraction of Multiword Expressions Combining Statistical and Similarity Approaches

Using a Chinese Lexicon to Learn Sense Embeddings and Measure Semantic Similarity.

Computing Semantic Text Similarity Using Rich Features.

ECNU: Leveraging on Ensemble of Heterogeneous Features and Information Enrichment for Cross Level Semantic Similarity Estimation

Improving Keyphrase Extraction Using Wikipedia Semantics

Semantic Word-formation Based Chinese Word Similarity Computing

A semi-supervised key phrase extraction approach: learning from title phrases through a document semantic network

Improving Word Similarity Computation Accuracy by Multiple Parameter Optimization Based on Ontology Knowledge

Word Similarity Computation with Extreme-Similar Method.

Semantic similarity between complex named entities: An approach using multiple web resources

A semantic matching approach addressing multidimensional representations for web service discovery

ECNU: Using Traditional Similarity Measurements and Word Embedding for Semantic Textual Similarity Estimation.

Research on Chinese Semantic Similarity Algorithm

Semantic Similarity Computation Based on Multi-feature Combination Using HowNet.

Revisit Word Embeddings with Semantic Lexicons for Modeling Lexical Contrast

Improving Multilingual Semantic Textual Similarity with Shared Sentence Encoder for Low-resource Languages