Abstract:In vector space model (VSM), text representation is the task of transforming the content of a textual document into a vector in the term space so that the document could be recognized and classified by a computer or a classifier. Different terms (i.e. words, phrases, or any other indexing units used to identify the contents of a text) have different importance in a text. The term weighting methods assign appropriate weights to the terms to improve the performance of text categorization. In this study, we investigate several widely-used unsupervised (traditional) and supervised term weighting methods on benchmark data collections in combination with SVM and kNN algorithms. In consideration of the distribution of relevant documents in the collection, we propose a new simple supervised term weighting method, i.e. tf.rf, to improve the terms' discriminating power for text categorization task. From the controlled experimental results, these supervised term weighting methods have mixed performance. Specifically, our proposed supervised term weighting method, tf.rf, has a consistently better performance than other term weighting methods while other supervised term weighting methods based on information theory or statistical metric perform the worst in all experiments. On the other hand, the popularly used tf.idf method has not shown a uniformly good performance in terms of different data sets.

A Novel Term Weighting Scheme with Distributional Coefficient for Text Categorization with Support Vector Machine

A Novel Term Weighting Scheme for Automated Text Categorization

Efficient text representation via weighted co-contributions of terms on class tendency

Term Selection and Weighting Approach Based on Key Words in Text Categorization

Entropy-Based Term Weighting Schemes for Text Categorization in VSM

Term Weighting Scheme with Enhanced Category Contribution for Text Categorization

Supervised and Traditional Term Weighting Methods for Automatic Text Categorization

Term Weighting Algorithm in Text Categorization Based on VSM

A comprehensive comparative study on term weighting schemes for text categorization with support vector machines.

A Semantic Term Weighting Scheme for Text Categorization.

A Novel Feature Weight Algorithm For Text Categorization

Efficient Representation of Text with Multiple Perspectives

A Supervised Term Weighting Scheme for Multi-class Text Categorization

Proposing a new term weighting scheme for text categorization

An improved supervised term weighting scheme for text representation and classification

Inverse-Category-Frequency based Supervised Term Weighting Schemes for Text Categorization.

A Study On Feature Weighting In Chinese Text Categorization

An Improved Feature Weighting Strategy in Chinese Text Categorization

A Redundancy Based Term Weighting Approach for Text Categorization

A Comparati E Study on Term Eighting Schemes for Te T Categori Ation

Three New Feature Weighting Methods for Text Categorization