Abstract:It is widely accepted that traditional word embedding models, which rely on distributional semantics hypothesis, are relatively limited for contrast meaning problem. Distributional semantics hypothesis indicates that words lying in similar contexts have similar representations in vector space. Nevertheless, synonyms and antonyms often locate in similar contexts, which means they appear close to each other in vector space. Hence, it is of great difficulty to distinguish antonyms from synonyms. To address this challenge, we propose an optimization model, named Lexicon-based Word Embedding Tuning (LWET) model. The goal of LWET is to incorporate reliable semantic lexicons to tune the distributions of pre-trained word embeddings in the vector space so as to improve their ability of distinguishing antonyms from synonyms. To speed up the training process of LWET, we propose two approximation algorithms, including positive sampling and quasi-hierarchical softmax. Compared with quasi-hierarchical softmax, positive sampling is faster, however, at the cost of worse performance. In experiments, LWET and other state-of-the-art models are tested on antonyms recognition, distinguishing antonyms from synonyms and word similarity. The results of the first two experiments show that LWET significantly improves the ability of word embeddings to detect antonyms, thus achieving the state-of-the-art performance. On word similarity, LWET gets slightly better performance than the state-of-the-art models. It means that LWET can remain and strengthen the semantic structure rather than destroy it when tuning word distributions in vector space. In general, compared with related work, LWET can not only achieve similar or even better performance, but also speed up the training process.

A novel model for semantic similarity measurement based on wordnet and word embedding

A Novel Comprehensive Approach for Estimating Concept Semantic Similarity in WordNet

A New Hypred Improved Method for Measuring Concept Semantic Similarity in WordNet.

Evaluation of taxonomic and neural embedding methods for calculating semantic similarity

Measuring Semantic Similarity Between Words Based On Multiple Relational Information

Application-Oriented Comparison and Evaluation of Six Semantic Similarity Measures Based on Wordnet

Semantic Similarity Computing Model Based on Multi Model Fine-Grained Nonlinear Fusion

Research on the Semantic Measurement in Co-word Analysis.

A comparative study for wordnet guided text representation

An Exploration Of Semantic Relations In Neural Word Embeddings Using Extrinsic Knowledge

Research on the Modeling of Semantic-Based Web Resources Feature.

Bridging the Semantic Latent Space Between Brain and Machine: Similarity is All You Need

Lexical semantics enhanced neural word embeddings

Chinese Word Sense Embedding with SememeWSD and Synonym Set

A Hybrid Semantic Similarity Measurement for Geospatial Entities

Revisit Word Embeddings with Semantic Lexicons for Modeling Lexical Contrast

Research on Chinese Semantic Similarity Algorithm

Evaluation method of word embedding by roots and affixes

Research on Hownet-Based Chinese Word Lexical Semantic Similarity Measurement

A new similarity measure for vector space models in text classification and information retrieval

Improving Word Embeddings for Antonym Detection Using Thesauri and SentiWordNet.