Abstract:In text classification field, many classifiers cannot deal with the features with large dimensions, thus it is very important to filter the redundant information from the original feature space efficiently and achieve the features with best qualities. On this basis, a new two-step based feature selection method is proposed in this paper. Firstly, some definitions (word semantic correlation, set semantic correlation, semantic correlative and semantic correlative set) are introduced, and the algorithm of generating the semantic correlative sets is given. Secondly, the process of the two-step based feature selection method is described: in the first step, a feature subset is obtained by using an optimal feature selection method, and a set of semantic correlative sets is generated by using the selected feature subset; in the second step, the redundant information of the selected features is filtered by using the generated semantic correlative sets. Finally, in order to avoid local optimum when searching the best threshold, the conception of memory recall position is introduced and an improved memory recall mechanism based fruit fly optimization algorithm is proposed. In the experiments, two typical classifiers: support vector machine and naïve bayes are used on four datasets: Reuters50, SMSSPAS, WebKB and 20-Newsgroups, and the 10-cross validation is carried out when the measurements of F 1 and receiver operating curve are used. Experimental results show that the proposed method achieves higher accuracy than several representative traditional feature selection methods and runs faster than typical mutual information based feature selection methods, illustrating its effectiveness on achieving the best features in text classification filed.

Method of Feature Reduction in Chinese Text Classification Based on Semantics

Improving Short Text Classification Through Better Feature Space Selection

Semantic Feature Reduction in Chinese Document Clustering

Dimensionality Reduction With Category Information Fusion And Non-Negative Matrix Factorization For Text Categorization

Feature Selection Method on Imbalanced Text

Feature Dimension Reduction Short Text Clustering Combined with Semantic and Statistics

Aggressive Dimensionality Reduction With Reinforcement Local Feature Selection For Text Categorization

A New Feature Selection Method Based On Concept Extraction In Automatic Chinese Text Classification

Text Classification Method Based on Semantic Distance

Semantic Similarity Metric and Its Application in Text Classification

Efficient Method for Feature Selection in Text Classification

Combining Lexical and Semantic Features for Short Text Classification.

A Method of Feature Selection Based on Word2Vec in Text Categorization

Research of Chinese Text Classification Methods Based on Semantic Vector and Semantic Similarity

Learning Effective Features for Chinese Text Categorization

Text Classification Via Learning Semantic Dependency and Association

A Discriminative and Semantic Feature Selection Method for Text Categorization

Two-step Based Feature Selection Method for Filtering Redundant Information

Automatic Text Classification Based On Rough Set And Improved Quick-Reduce Algorithm

A feature combination method for semantic role classification

An effective multi-modal adaptive contextual feature information fusion method for Chinese long text classification