Abstract:Document similarity search is to find documents similar to a given query document and return a ranked list of similar documents to users, which is widely used in many text and web systems, such as digital library, search engine, etc. Traditional retrieval models, including the Okapi's BM25 model and the Smart's vector space model with length normalization, could handle this problem to some extent by taking the query document as a long query. In practice, the Cosine measure is considered as the best model for document similarity search because of its good ability to measure similarity between two documents. In this paper, the quantitative performances of the above models are compared using experiments. Because the Cosine measure is not able to reflect the structural similarity between documents, a new retrieval model based on TextTiling is proposed in the paper. The proposed model takes into account the subtopic structures of documents. It first splits the documents into text segments with TextTiling and calculates the similarities for different pairs of text segments in the documents. Lastly the overall similarity between the documents is returned by combining the similarities of different pairs of text segments with optimal matching method. Experiments are performed and results show: 1) the popular retrieval models (the Okapi's BM25 model and the Smart's vector space model with length normalization) do not perform well for document similarity search; 2) the proposed model based on TextTiling is effective and outperforms other models, including the Cosine measure; 3) the methods for the three components in the proposed model are validated to be appropriately employed.

An algorithm of document refinement based on sentence similarity computation

Document Refinement Based on Semantic Query Expansion

A Semantic Query Expansion-Based Patent Retrieval Approach

Searching Semantic Web Documents Based on RDF Sentences

Using query expansion in graph-based approach for query-focused multi-document summarization

Semantic Similarity Computing Model Based on Multi Model Fine-Grained Nonlinear Fusion

Document similarity search based on generic summaries

Constructing virtual documents for ontology matching.

Improving Sentence Similarity Estimation for Unsupervised Extractive Summarization

Semantic Query Expansion and Context-Based Discriminative Term Modeling for Spoken Document Retrieval

Using Proximity in Query Focused Multi-document Extractive Summarization

A New Retrieval Model Based on TextTiling for Document Similarity Search

Cross-Language Similar Document Retrieval

Automatic multi-document summarization based on new sentence similarity measures

Double-Hypergraph Based Sentence Ranking for Query-Focused Multi-document Summarizaton

Concept Based Query Expansion Using Wordnet

ON SENTENCE-BASED INFORMATION RETRIEVAL MODEL

A New Document Retrieval Model Using Dempster-Shafer Theory of Evidence

Combining document representations for known-item search

Diversified and Verbalized Result Summarization for Semantic Association Search

Query by document via a decomposition-based two-level retrieval approach.