Abstract:Search engine is a common tool to retrieve the information in the Web. But the current status of returned results is still far from satisfaction. Users have to be confronted with searching for a long result list to get the information really wanted. Many works focused on the post processing search results to facilitate users to examine the results. One of the common ways of post processing search result is clustering. Term-based clustering appears as first way to cluster the results. But this method is suffering from the poor quality while the processed pages have little text. Link-based clustering can conquer this problem. But the quality of clusters heavily depends on the number of in-links and out-links in common. In this paper, we propose that the short text attached to in-link is valuable information and it is helpful to reach high clustering quality. To distinguish them with general snippet, we name it as in-snippet. Based on the in-snippet, we propose a new clustering method that combines the links and the in-snippets together. In our method, similarity between pages consists of two parts : link similarity and term similarity. We designed related algorithm to implement clustering. In order to prevent bias from human judgments, the experiment datasets are collected from Open Directory Project(DMOZ). Due to DMOZ is human-edited directory, the datasets from DMOZ has higher quality and larger scale. We use entropy and f-measure to evaluate the quality of the final clusters. By being compared with the link-based and the pure term-based algorithms, our method outperforms others in clustering quality.

An Improved Measuring Similarity For Short Text Snippets And Its Application In Clustering Search Engine

An adaptive method for text domain similarity calculation

Improving short text classification using public search engines

Similarity Measure Based on Improved Optimal Assignment Model

Short Text Similarity Calculation Using Semantic Information

Short Text Clustering for Search Results

An Efficient Similarity Measure Algorithm of Chinese Sentence

Structure And Content Similarity For Clustering Xml Documents

A Short Text Modeling Method Combining Semantic and Statistical Information

A Semantics-Based Method For Clustering Of Chinese Web Search Results

Clustering of Web Search Results Based on Combination of Links and In-Snippets

A survey on the techniques, applications, and performance of short text semantic similarity

Short Text Similarity Calculation Based on Jaccard and Semantic Mixture

A New Model for Measuring Similarity of Web Queries and Its Application in Query Expansion

Measurement of Text Similarity: A Survey

A Chinese Short Text Similarity Algorithm Based on Semantic and Syntax

An Efficient Token-based Approach for Web-Snippet Clustering

A Text Similarity Measurement Based on Semantic Fingerprint of Characteristic Phrases

Text Similarity Measurement of Semantic Cognition Based on Word Vector Distance Decentralization with Clustering Analysis

An Improved Similarity Measure Method in Collaborative Filtering Recommendation Algorithm

A Comparative Study Of Ontology Based Term Similarity Measures On Pubmed Document Clustering