Abstract:In natural language processing (NLP), text similarity calculation is widely used in information retrieval, machine translation, text mining etc. The definition of the similarity between texts may not just refer to words with similar meanings. Domain similarity, which evaluates the similarity on basis of domain reference, is becoming a promising approach in dealing with large documents. By adopting domain similarity calculation, the degree of similarity could be controlled at different semantic levels, and extract texts in different domain granularity. For example, web pages of Lakers, NBA, basketball and sports could be retrieved respectively with different settings of domain similarity. LSI (Latent Semantic Indexing) is a feasible approach that can be applied to calculate text domain similarity. By controlling the number of topics, the domain similarity can be determined in different granularity. However, the performance is greatly affected by the number of specified topics, which is required for LSI algorithm. In this paper, an adaptive method was applied to word similarity calculation. TF-IDF was used to get the word frequency in the text, and the number of topics in the mixed text, set by dimensionality reduction and clustering was automatically obtained. According to the number of clusters, the similarity between text domains was calculated as the number of topics mapped to the subspace in the LSI. Accordingly, experimental results have shown that the method proposed in this paper is superior to other algorithms in the accuracy of text similarity calculation.

A Deep Learning Based Method to Measure the Similarity of Long Text

An adaptive method for text domain similarity calculation

Chinese long text similarity calculation of semantic progressive fusion based on Bert

Short Text Similarity Calculation Using Semantic Information

A survey on the techniques, applications, and performance of short text semantic similarity

Text Similarity Measurement Method Based on BiLSTM-SECapsNet Model

Fine-grained Semantic Textual Similarity Measurement Via a Feature Separation Network

Using Multiple Features and Statistical Model to Calculate Text Units Similarity

Deep Text Understanding Model for Similar Case Matching

Measurement of Text Similarity: A Survey

Re-LSTM: A long short-term memory network text similarity algorithm based on weighted word embedding

A text classification network model combining machine learning and deep learning

Document Similarity for Texts of Varying Lengths via Hidden Topics

A Survey on Deep Text Matching

A Short Texts Matching Method Using Shallow Features and Deep Features

Attentive Siamese LSTM Network for Semantic Textual Similarity Measure

A Combined Measure for Text Semantic Similarity

Similarity algorithm of text based on semantic understanding

A Chinese Short Text Similarity Method Integrating Sentence-Level and Phrase-Level Semantics

ECNU: Using Traditional Similarity Measurements and Word Embedding for Semantic Textual Similarity Estimation.

Hybrid Attention Based Neural Architecture for Text Semantics Similarity Measurement