A Chinese Short Text Similarity Method Integrating Sentence-Level and Phrase-Level Semantics
Zhenji Shen,Zhiyong Xiao
DOI: https://doi.org/10.3390/electronics13244868
IF: 2.9
2024-12-11
Electronics
Abstract:Short text similarity, as a pivotal research domain within Natural Language Processing (NLP), has been extensively utilized in intelligent search, recommendation systems, and question-answering systems. Most existing short-text similarity models focus on aligning the overall semantic content of an entire sentence, often ignoring the semantic associations between individual phrases in the sentence. It is particular in the Chinese context, as synonyms and near-synonyms can cause serious interference in the computation of text similarity. To overcome these limitations, a novel short text similarity computation method integrating both sentence-level and phrase-level semantics was proposed. By harnessing vector representations of Chinese words/phrases as external knowledge, this approach amalgamates global sentence characteristics with local phrase features to compute short text similarity from diverse perspectives, spanning from the global to the local level. Experimental results demonstrate that the proposed model outperforms previous methods in the Chinese short text similarity task. Specifically, the model achieves an accuracy of 90.16% in LCQMC, which is 2.23% and 1.46%, respectively, better than ERNIE and Glyce + BERT.
engineering, electrical & electronic,computer science, information systems,physics, applied