Abstract:In natural language processing (NLP), text similarity calculation is widely used in information retrieval, machine translation, text mining etc. The definition of the similarity between texts may not just refer to words with similar meanings. Domain similarity, which evaluates the similarity on basis of domain reference, is becoming a promising approach in dealing with large documents. By adopting domain similarity calculation, the degree of similarity could be controlled at different semantic levels, and extract texts in different domain granularity. For example, web pages of Lakers, NBA, basketball and sports could be retrieved respectively with different settings of domain similarity. LSI (Latent Semantic Indexing) is a feasible approach that can be applied to calculate text domain similarity. By controlling the number of topics, the domain similarity can be determined in different granularity. However, the performance is greatly affected by the number of specified topics, which is required for LSI algorithm. In this paper, an adaptive method was applied to word similarity calculation. TF-IDF was used to get the word frequency in the text, and the number of topics in the mixed text, set by dimensionality reduction and clustering was automatically obtained. According to the number of clusters, the similarity between text domains was calculated as the number of topics mapped to the subspace in the LSI. Accordingly, experimental results have shown that the method proposed in this paper is superior to other algorithms in the accuracy of text similarity calculation.

An analysis method of cross-lingual literature similarity

An adaptive method for text domain similarity calculation

Similarity Calculations Of Academic Articles Using Topic Events And Domain Knowledge

Cross-Lingual Document Clustering Based on Similarity Space Model

A Model for Chinese Sentence Similarity Computing

Cross-Language Similar Document Retrieval

Research on Author's Semantic Similarity Based on Collaborative Network

Research on the Extraction and Alignment of Tibetan-Chinese Cross-language Topics

Identifying Word Translations in Scientific Literature Based on Labeled Bilingual Topic Model and Co-occurrence Features

Cross-Lingual Taxonomy Alignment With Bilingual Biterm Topic Model

English-Chinese Cross Language Word Embedding Similarity Calculation

Analysis and Research on Cross Language Topic Discovery in Chinese and English

An English-Chinese Cross-lingual Word Semantic Similarity Measure Exploring Attributes and Relations.

Understanding Cross-Lingual Alignment -- A Survey

Nonparametric Symmetric Correspondence Topic Models for Multilingual Text Analysis.

English-Chinese translation strategies based on BERT and multi-similarity fusion of English and American literature

A Chinese Short Text Similarity Method Integrating Sentence-Level and Phrase-Level Semantics

Similarity of Sentence Representations in Multilingual LMs: Resolving Conflicting Literature and Case Study of Baltic Languages

Using Link-Based Content Analysis to Measure Document Similarity Effectively

Chinese Sentence Similarity Computational Model Based on Multi-Features Combination

A New Method for Calculating Similarity Between Sentences and Application on Automatic Abstracting