Abstract:In natural language processing (NLP), text similarity calculation is widely used in information retrieval, machine translation, text mining etc. The definition of the similarity between texts may not just refer to words with similar meanings. Domain similarity, which evaluates the similarity on basis of domain reference, is becoming a promising approach in dealing with large documents. By adopting domain similarity calculation, the degree of similarity could be controlled at different semantic levels, and extract texts in different domain granularity. For example, web pages of Lakers, NBA, basketball and sports could be retrieved respectively with different settings of domain similarity. LSI (Latent Semantic Indexing) is a feasible approach that can be applied to calculate text domain similarity. By controlling the number of topics, the domain similarity can be determined in different granularity. However, the performance is greatly affected by the number of specified topics, which is required for LSI algorithm. In this paper, an adaptive method was applied to word similarity calculation. TF-IDF was used to get the word frequency in the text, and the number of topics in the mixed text, set by dimensionality reduction and clustering was automatically obtained. According to the number of clusters, the similarity between text domains was calculated as the number of topics mapped to the subspace in the LSI. Accordingly, experimental results have shown that the method proposed in this paper is superior to other algorithms in the accuracy of text similarity calculation.

The Dictionary-Based Quantified Conceptual Relations For Hard And Soft Chinese Text Clustering

An adaptive method for text domain similarity calculation

A Clustering Algorithm for Short Documents Based On Concept Similarity

A Fast and Effective Method for Clustering Large-Scale Chinese Question Dataset

Semantic smoothing of document models for agglomerative clustering

A Novel Text Clustering Algorithm Based on Inner Product Space Model of Semantic

A Comparative Study on Representing Units in Chinese Text Clustering

Text Clustering on Oral Conversation Corpus.

A New Similarity Computing Method Based on Concept Similarity in Chinese Text Processing.

Concept chain based text clustering

Constrained Coclustering for Textual Documents.

Semantic Smoothing for Model-based Document Clustering

Semantic Word-formation Based Chinese Word Similarity Computing

Document Clustering Based on Semantic Smoothing Approach

A Concept Similarity Based Text Classification Algorithm

Semantic Correlation Network Based Text Clustering

CDW: A Text Clustering Model for Diverse Versions Discovery.

Clustering web images by correlation mining of image-text

Clustering of Chinese Adjectives Based on the Machine Tractable Dictionary of Contemporary Chinese Predicate Adjectives

A Linguistic Feature Based Text Clustering Method.

Fuzzy C-Means Text Clustering Based on Topic Concept Sub-Space