Abstract:In vector space model (VSM), text representation is the task of transforming the content of a textual document into a vector in the term space so that the document could be recognized and classified by a computer or a classifier. Different terms (i.e. words, phrases, or any other indexing units used to identify the contents of a text) have different importance in a text. The term weighting methods assign appropriate weights to the terms to improve the performance of text categorization. In this study, we investigate several widely-used unsupervised (traditional) and supervised term weighting methods on benchmark data collections in combination with SVM and kNN algorithms. In consideration of the distribution of relevant documents in the collection, we propose a new simple supervised term weighting method, i.e. tf.rf, to improve the terms' discriminating power for text categorization task. From the controlled experimental results, these supervised term weighting methods have mixed performance. Specifically, our proposed supervised term weighting method, tf.rf, has a consistently better performance than other term weighting methods while other supervised term weighting methods based on information theory or statistical metric perform the worst in all experiments. On the other hand, the popularly used tf.idf method has not shown a uniformly good performance in terms of different data sets.

Text Categorization Using SVM with Exponent Weighted ACO

A multiclass classification framework for document categorization

Application of SVM to Text Categorization

Feature Selection for Support Vector Machines in Text Categorization

A Text Categorization Method Based on SVM and Improved K-Means

Term Weighting Algorithm in Text Categorization Based on VSM

Aggressive Dimensionality Reduction With Reinforcement Local Feature Selection For Text Categorization

Term Selection and Weighting Approach Based on Key Words in Text Categorization

Supervised and Traditional Term Weighting Methods for Automatic Text Categorization

An On-line Learning Algorithm for Text Categorization with Support Vector Machines

A comprehensive comparative study on term weighting schemes for text categorization with support vector machines.

Web Search with Text Categorization Using Probabilistic Framework of SVM

A New Text Categorization Method Based on HMM and SVM

Study on SVM Compared with the other Text Classification Methods

An Improved Text Categorization Algorithm Based on VSM

Text Representations for Text Categorization: A Case Study in Biomedical Domain

Improved VSM Based on Chinese Text Categorization

A Text Categorization Method Using Extended Vector Space Model by Frequent Term Sets

A Comparati E Study on Term Eighting Schemes for Te T Categori Ation

A Grid-Based ACO Algorithm for Parameters Optimization in Support Vector Machines

Entropy-Based Term Weighting Schemes for Text Categorization in VSM