Abstract:In text classification, it is necessary to perform feature selection to alleviate the curse of dimensionality caused by high-dimensional text data. In this paper, we utilize class term frequency (CTF) and class document frequency (CDF) to characterize the relevance between terms and categories in the level of term frequency (TF) and document frequency (DF). On the basis of relevance measurement above, three feature selection methods (ADF based on CTF (ADF-CTF), ADF based on CDF (ADF-CDF), and ADF based on both CTF and CDF (ADF-CTDF)) are proposed to identify relevant and discriminant terms by introducing absolute deviation factors (ADFs). Absolute deviation, a statistic concept, is first adopted to measure the relevance divergence characterized by CTF and CDF. In addition, ADF-CTF and ADF-CDF can be combined with existing DF-based and TF-based methods, respectively, which results in new ADF-based methods. Experimental results on six high-dimensional textual datasets using three classifiers indicate that ADF-based methods outperform original DF-based and TF-based ones in 89% cases in terms of Micro-F1 and Macro-F1, which demonstrates the role of ADF integrated in existing methods to boost the classification performance. In addition, findings also show that ADF-CTDF ranks first averagely among multiple datasets and significantly outperforms other methods in 99% cases.

New Feature Selection Approach(cdf) for Text Categorization

CLDA: Feature Selection for Text Categorization Based on Constrained LDA

Aggressive Dimensionality Reduction With Reinforcement Local Feature Selection For Text Categorization

A New Feature Selection Method for Text Categorization

An Effective Feature Selection Method For Text Categorization

Feature selection method based on category discriminability

A New Feature Selection Method in Text Categorization

A Feature Selection Method Based on Class Feature Domains for Text Categorization

A General Framework of Feature Selection for Text Categorization

A New Feature Selection Based on Comprehensive Measurement Both in Inter-Category and Intra-Category for Text Categorization

Application of Feature Selection Method to Text Categorization Based on Feature Contribution Degree

A New Approach of Feature Selection for Text Categorization

Dimensionality Reduction With Category Information Fusion And Non-Negative Matrix Factorization For Text Categorization

Learning Effective Features for Chinese Text Categorization

Text Feature Selection Based on Class Subspace

Relative Term-Frequency Based Feature Selection for Text Categorization

Feature Selection Based on Absolute Deviation Factor for Text Classification

New Approach to Feature Selection for Text Categorization Using Class Correlation

Improved Comprehensive Measurement Feature Selection Method for Text Categorization

Feature reduction methods for text classification

Efficient Method for Feature Selection in Text Classification