Study on Meaningful String Extraction Algorithm for Improving Webpage Classification

Jie Chen,Jian Li,Hao Liao,Qingsheng Yuan,Xiuguo Bao

DOI: https://doi.org/10.1109/nlpke.2011.6138182

2011-01-01

Abstract:This paper shows that the accuracy of webpage classifiers can be improved by extracting meaningful strings with an unsupervised clustering method. Since webpage classification is different from original document classification with its words and phrases irregular, massive and unlabeled features, to cope with these features, we introduce two scenarios for extracting meaningful strings based on document clustering and term clustering with multi-strategies to optimize a vector space model (VSM). First, some candidate strings are used to build VSM; then, two scenarios performance on VSM respectively; last, we can extract meaningful strings from each cluster. So, theses meaningful string may represent documents comprehensively. The proposed method has been applied to webpage document classification and the results show that document clustering works better than term clustering. They also demonstrate that spectral clustering method outperforms k-means in document clustering case, conversely, make reduce performance in term clustering. However, a better synthetic performance can be obtained by spectral clustering with document clustering from some experiments.