Abstract:It is widely accepted that traditional word embedding models, which rely on distributional semantics hypothesis, are relatively limited for contrast meaning problem. Distributional semantics hypothesis indicates that words lying in similar contexts have similar representations in vector space. Nevertheless, synonyms and antonyms often locate in similar contexts, which means they appear close to each other in vector space. Hence, it is of great difficulty to distinguish antonyms from synonyms. To address this challenge, we propose an optimization model, named Lexicon-based Word Embedding Tuning (LWET) model. The goal of LWET is to incorporate reliable semantic lexicons to tune the distributions of pre-trained word embeddings in the vector space so as to improve their ability of distinguishing antonyms from synonyms. To speed up the training process of LWET, we propose two approximation algorithms, including positive sampling and quasi-hierarchical softmax. Compared with quasi-hierarchical softmax, positive sampling is faster, however, at the cost of worse performance. In experiments, LWET and other state-of-the-art models are tested on antonyms recognition, distinguishing antonyms from synonyms and word similarity. The results of the first two experiments show that LWET significantly improves the ability of word embeddings to detect antonyms, thus achieving the state-of-the-art performance. On word similarity, LWET gets slightly better performance than the state-of-the-art models. It means that LWET can remain and strengthen the semantic structure rather than destroy it when tuning word distributions in vector space. In general, compared with related work, LWET can not only achieve similar or even better performance, but also speed up the training process.

PolyUCOMP: Combining Semantic Vectors with Skip Bigrams for Semantic Textual Similarity

PolyUCOMP-CORE_TYPED: Computing Semantic Textual Similarity Using Overlapped Senses.

Sentence Similarity Computation in Question Answering Robot

Overview Of The Nlpcc-Iccpol 2016 Shared Task: Chinese Word Similarity Measurement

Incorporate Semantic Structures into Machine Translation Evaluation via UCCA

MIXCD: System Description for Evaluating Chinese Word Similarity at SemEval-2012.

SemEval-2012 Task 4: Evaluating Chinese Word Similarity.

Application-Oriented Comparison and Evaluation of Six Semantic Similarity Measures Based on Wordnet

Chinese Word Sense Embedding with SememeWSD and Synonym Set

A survey on the techniques, applications, and performance of short text semantic similarity

Knowledge-Enhanced Bilingual Textual Representations for Cross-Lingual Semantic Textual Similarity

Research on Chinese Semantic Similarity Algorithm

Sentence Similarity Computation by Integrating Shallow and Deep Information

Chinese Word Similarity Computing Based on Combination Strategy

Semantic Similarity Based on Compact Concept Ontology

Revisit Word Embeddings with Semantic Lexicons for Modeling Lexical Contrast

Word Sense Disambiguation using Knowledge-based Word Similarity

Semantic Similarity Based on Corpus Statistics and Lexical Taxonomy

PolyUCOMP in TAC 2011 Entity Linking and Slot-Filling.

Semantic Similarity Computing Model Based on Multi Model Fine-Grained Nonlinear Fusion

Semantic similarity information discrimination for video captioning