Efficient Text Classification Using Term Projection

Yabin Zheng,Zhiyuan Liu,Shaohua Teng,Maosong Sun
DOI: https://doi.org/10.1007/978-3-642-04769-5_20
2009-01-01
Abstract:In this paper, we propose an efficient text classification method using term projection. Firstly, we use a modified chi(2) statistic to project terms into pre-defined categories, which is more efficient compared to other clustering methods. Afterwards, we utilize the generated clusters as features to represent the documents. The classification is then performed in a rule-based manner or via SVM. Experiment results show that our modified chi(2) statistic feature selection method outperforms traditional chi(2) statistic especially at lower dimensionalities. And our method is also more efficient than Latent Semantic Analysis (LSA) on homogeneous dataset. Meanwhile, we can reduce the feature dimensionality by three orders of magnitude to save training and testing cost, and maintain comparable accuracy. Moreover, we could use a small training set to gain an approximately 4.3% improvement on heterogeneous dataset as compared to traditional method, which indicates that our method has better generalization capability.
What problem does this paper attempt to address?