Abstract:In this paper, we propose a new random forest (RF) based ensemble method, ForesTexter, to solve the imbalanced text categorization problems. RF has shown great success in many real-world applications. However, the problem of learning from text data with class imbalance is a relatively new challenge that needs to be addressed. A RF algorithm tends to use a simple random sampling of features in building their decision trees. As a result, it selects many subspaces that contain few, if any, informative features for the minority class. Furthermore, the Gini measure for data splitting is considered to be skew sensitive and bias towards the majority class. Due to the inherent complex characteristics of imbalanced text datasets, learning RF from such data requires new approaches to overcome challenges related to feature subspace selection and cut-point choice while performing node splitting. To this end, we propose a new tree induction method that selects splits, both feature subspace selection and splitting criterion, for RF on imbalanced text data. The key idea is to stratify features into two groups and to generate effective term weighting for the features. One group contains positive features for the minority class and the other one contains the negative features for the majority class. Then, for feature subspace selection, we effectively select features from each group based on the term weights. The advantage of our approach is that each subspace contains adequate informative features for both minority and majority classes. One difference between our proposed tree induction method and the classical RF method is that our method uses Support Vector Machines (SVM) classifier to split the training data into smaller and more balance subsets at each tree node, and then successively retrains the SVM classifiers on the data partitions to refine the model while moving down the tree. In this way, we force the classifiers to learn from refined feature subspaces and data subsets to fit the imbalanced data better. Hence, the tree model becomes more robust for text categorization task with imbalanced dataset. Experimental results on various benchmark imbalanced text datasets (Reuters-21578, Ohsumed, and imbalanced 20 newsgroup) consistently demonstrate the effectiveness of our proposed ForesTexter method. The performance of our proposed approach is competitive against the standard random forest and different variants of SVM algorithms.

AN IMPROVED RANDOM FOREST BOOST MULTI-LABEL TEXT CLASSIFICATION ALGORITHM

An Improved Random Forest Classifier for Text Categorization.

An improved floatboost algorithm for Naïve bayes text classification

An Improved FloatBoost Algorithm for Naïve Bayes Text Classification.

ForesTexter: an Efficient Random Forest Algorithm for Imbalanced Text Categorization

A New Rotation Forest Ensemble Algorithm

Improved Parallel Random Forest Algorithm Combining Information Theory and Norm

MLRF: Multi-label classification through random forest with label-set partition

Stacking Bagged and Boosted Forests for Effective Automated Classification

A Multi-Label Chinese Text Categorization System Based on Boosting Algorithm

Two-Stage Multi-Class Adaboost for Facial Expression Recognition

An Improved Random Forest Classifier for Image Classification

Improved Boosting Algorithm With Adaptive Filtration

An Efficient Modified Boosting Method for Solving Classification Problems

Binary Classification: Is Boosting stronger than Bagging?

RotBoost: A technique for combining Rotation Forest and AdaBoost

Lda Boost Classification: Boosting by Topics

An improved random forest based on the classification accuracy and correlation measurement of decision trees

Random Bits Forest: a Strong Classifier/Regressor for Big Data

An Improved Chinese Text Classification Algorithm Based On Multiple Feature Factors

A Dynamic Boosted Ensemble Learning Method Based on Random Forest