Abstract:In natural language processing (NLP), text similarity calculation is widely used in information retrieval, machine translation, text mining etc. The definition of the similarity between texts may not just refer to words with similar meanings. Domain similarity, which evaluates the similarity on basis of domain reference, is becoming a promising approach in dealing with large documents. By adopting domain similarity calculation, the degree of similarity could be controlled at different semantic levels, and extract texts in different domain granularity. For example, web pages of Lakers, NBA, basketball and sports could be retrieved respectively with different settings of domain similarity. LSI (Latent Semantic Indexing) is a feasible approach that can be applied to calculate text domain similarity. By controlling the number of topics, the domain similarity can be determined in different granularity. However, the performance is greatly affected by the number of specified topics, which is required for LSI algorithm. In this paper, an adaptive method was applied to word similarity calculation. TF-IDF was used to get the word frequency in the text, and the number of topics in the mixed text, set by dimensionality reduction and clustering was automatically obtained. According to the number of clusters, the similarity between text domains was calculated as the number of topics mapped to the subspace in the LSI. Accordingly, experimental results have shown that the method proposed in this paper is superior to other algorithms in the accuracy of text similarity calculation.

Text Similarity Computing Based On Attribute Center Of Gravity Model And Lsa

An adaptive method for text domain similarity calculation

Improvement of Text Similarity Computing Algorithm Based on Attribute

A New Similarity Computing Method Based on Concept Similarity in Chinese Text Processing.

Study on Spatial Geometric Similarity Based on Conformal Geometric Algebra.

Statutes Recommendation Based on Text Similarity.

Research on Chinese Semantic Similarity Algorithm

STSG: A Short Text Semantic Graph Model for Similarity Computing Based on Dependency Parsing and Pre-trained Language Models

A Concept Similarity Based Text Classification Algorithm

A Hybrid Semantic Similarity Measurement for Geospatial Entities

Description-Based Text Similarity

Learning Similarity Measures in Non-Orthogonal Space.

A similarity computing algorithm for volumetric data sets

Query similarity computing based on system similarity measurement

Bridging the Semantic Latent Space Between Brain and Machine: Similarity is All You Need

Semantic Similarity Computing Model Based on Multi Model Fine-Grained Nonlinear Fusion

Using Link-Based Content Analysis to Measure Document Similarity Effectively

A Short-Text Similarity Model Combining Semantic and Syntactic Information

Text Metric Method on Statistical Manifold Learning

Graph-Based Text Similarity Measurement by Exploiting Wikipedia As Background Knowledge

Research of Chinese Text Classification Methods Based on Semantic Vector and Semantic Similarity