Abstract:In text categorization, term selection is an important step for the sake of both categorization accuracy and computational efficiency. Different dimensionalities are expected under different practical resource restrictions of time or space. Traditionally in text categorization, the same scoring or ranking criterion is adopted for all target dimensionalities, which considers both the discriminability and the coverage of a term, such as ´ 2 or IG. In this paper, the poor accuracy at a low dimensionality is imputed to the small average vector length of the documents. Scalable term selection is proposed to optimize the term set at a given dimensionality according to an expected average vector length. Discriminability and coverage are separately measured; by adjusting the ratio of their weights in a combined criterion, the expected average vector length can be reached, which means a good compromise between the specificity and the exhaustivity of the term subset. Experiments show that the accuracy is considerably improved at lower dimensionalities, and larger term subsets have the possibility to lower the average vector length for a lower computational cost. The interesting observations might inspire further investigations.

Select Strong Information Features to Improve Text Categorization Effectiveness

CLDA: Feature Selection for Text Categorization Based on Constrained LDA

Feature Selection Method on Imbalanced Text

Learning Effective Features for Chinese Text Categorization

Aggressive Dimensionality Reduction With Reinforcement Local Feature Selection For Text Categorization

Improving Short Text Classification Through Better Feature Space Selection

A New Approach of Feature Selection for Text Categorization

A Study On Feature Weighting In Chinese Text Categorization

Feature Selection with Conditional Mutual Information Considering Feature Interaction

Dimensionality Reduction With Category Information Fusion And Non-Negative Matrix Factorization For Text Categorization

An Effective Feature Selection Method For Text Categorization

Relative Term-Frequency Based Feature Selection for Text Categorization

Eliminating High-Degree Biased Character Bigrams for Dimensionality Reduction in Chinese Text Categorization

Feature selection method based on backward cloud model in text classification

A multiclass classification framework for document categorization

A comprehensive unsupervised feature selection method of two-stage strategy

An Evaluation on Feature Selection for Text Clustering

Scalable Term Selection for Text Categorization.

A High Performance Two-Class Chinese Text Categorization Method

A General Framework of Feature Selection for Text Categorization

Chinese Text Categorization Based On The Binary Weighting Model With Non-Binary Smoothing