Abstract:In text categorization, term selection is an important step for the sake of both categorization accuracy and computational efficiency. Different dimensionalities are expected under different practical resource restrictions of time or space. Traditionally in text categorization, the same scoring or ranking criterion is adopted for all target dimensionalities, which considers both the discriminability and the coverage of a term, such as ´ 2 or IG. In this paper, the poor accuracy at a low dimensionality is imputed to the small average vector length of the documents. Scalable term selection is proposed to optimize the term set at a given dimensionality according to an expected average vector length. Discriminability and coverage are separately measured; by adjusting the ratio of their weights in a combined criterion, the expected average vector length can be reached, which means a good compromise between the specificity and the exhaustivity of the term subset. Experiments show that the accuracy is considerably improved at lower dimensionalities, and larger term subsets have the possibility to lower the average vector length for a lower computational cost. The interesting observations might inspire further investigations.

A Method of Feature Selection Based on Word2Vec in Text Categorization

CLDA: Feature Selection for Text Categorization Based on Constrained LDA

Text Feature Selection Methods Based on Word Vector

Improving Short Text Classification Through Better Feature Space Selection

Aggressive Dimensionality Reduction With Reinforcement Local Feature Selection For Text Categorization

A Discriminative and Semantic Feature Selection Method for Text Categorization

A Feature Selection Method Based on Class Feature Domains for Text Categorization

An Effective Feature Selection Method For Text Categorization

A comprehensive unsupervised feature selection method of two-stage strategy

Relative Term-Frequency Based Feature Selection for Text Categorization

Feature Selection for Support Vector Machines in Text Categorization

Efficient Method for Feature Selection in Text Classification

A Study On Feature Weighting In Chinese Text Categorization

Learning Effective Features for Chinese Text Categorization

Chi-square Statistics Feature Selection Based on Term Frequency and Distribution for Text Categorization

A feature selection method based on synonym merging in text classification system

A New Approach of Feature Selection for Text Categorization

Scalable Term Selection for Text Categorization.

A General Framework of Feature Selection for Text Categorization

Select Strong Information Features to Improve Text Categorization Effectiveness

Dimensionality Reduction With Category Information Fusion And Non-Negative Matrix Factorization For Text Categorization