Abstract:Text classification has been highlighted as the key process to organize online texts for better communication in the Digital Media Age. Text classification establishes classification rules based on text features, so the accuracy of feature selection is the basis of text classification. Facing fast-increasing Chinese electronic documents in the digital environment, scholars have accumulated quite a few algorithms for the feature selection for the automatic classification of Chinese texts in recent years. However, discussion about how to adapt existing feature selection algorithms for various types of Chinese texts is still inadequate. To address this, this study proposes three improved feature selection algorithms and tests their performance on different types of Chinese texts. These include an enhanced CHI square with mutual information (MI) algorithm, which simultaneously introduces word frequency and term adjustment (CHMI); a term frequency–CHI square (TF–CHI) algorithm, which enhances weight calculation; and a term frequency–inverse document frequency (TF–IDF) algorithm enhanced with the extreme gradient boosting (XGBoost) algorithm, which improves the algorithm’s ability of word filtering (TF–XGBoost). This study randomly chooses 3000 texts from six different categories of the Sogou news corpus to obtain the confusion matrix and evaluate the performance of the new algorithms with precision and the F1-score. Experimental comparisons are conducted on support vector machine (SVM) and naive Bayes (NB) classifiers. The experimental results demonstrate that the feature selection algorithms proposed in this paper improve performance across various news corpora, although the best feature selection schemes for each type of corpus are different. Further studies of the application of the improved feature selection methods in other languages and the improvement in classifiers are suggested.

Chinese text classification method using FastText and term frequency-inverse document frequency optimization

A Deep Investigation into fastText.

An Improved TF-IDF Approach for Text Classification

Research of Text Classification Based on Improved TF-IDF Algorithm

Fast text categorization based on collaborative work in the semantic and class spaces

Low-Resource Fast Text Classification Based on Intra-Class and Inter-Class Distance Calculation

FastClass: A Time-Efficient Approach to Weakly-Supervised Text Classification

Text Classification with Improved Word Embedding and Adaptive Segmentation

Research of Text Classification Based on TF-IDF and CNN-LSTM

Supervised Contrastive Learning with Term Weighting for Improving Chinese Text Classification

An Improved Term Weighting Scheme for Text Classification

Centroid Training to Achieve Effective Text Classification

A Chinese Short Text Classification Method Based on TF-IDF and Gradient Boosting Decision Tree

Text Classification Method Based On Convolution Neural Network

Dimensionality Reduction With Category Information Fusion And Non-Negative Matrix Factorization For Text Categorization

An Improved Chinese Text Classification Algorithm Based On Multiple Feature Factors

Chinese text classification method based on sentence information enhancement and feature fusion

Adapting Feature Selection Algorithms for the Classification of Chinese Texts

A Method for Chinese Text Classification Based on Three-Dimensional Vector Space Model

Optimizing News Text Classification with Bi-LSTM and Attention Mechanism for Efficient Data Processing