Abstract:In this paper, we propose a new random forest (RF) based ensemble method, ForesTexter, to solve the imbalanced text categorization problems. RF has shown great success in many real-world applications. However, the problem of learning from text data with class imbalance is a relatively new challenge that needs to be addressed. A RF algorithm tends to use a simple random sampling of features in building their decision trees. As a result, it selects many subspaces that contain few, if any, informative features for the minority class. Furthermore, the Gini measure for data splitting is considered to be skew sensitive and bias towards the majority class. Due to the inherent complex characteristics of imbalanced text datasets, learning RF from such data requires new approaches to overcome challenges related to feature subspace selection and cut-point choice while performing node splitting. To this end, we propose a new tree induction method that selects splits, both feature subspace selection and splitting criterion, for RF on imbalanced text data. The key idea is to stratify features into two groups and to generate effective term weighting for the features. One group contains positive features for the minority class and the other one contains the negative features for the majority class. Then, for feature subspace selection, we effectively select features from each group based on the term weights. The advantage of our approach is that each subspace contains adequate informative features for both minority and majority classes. One difference between our proposed tree induction method and the classical RF method is that our method uses Support Vector Machines (SVM) classifier to split the training data into smaller and more balance subsets at each tree node, and then successively retrains the SVM classifiers on the data partitions to refine the model while moving down the tree. In this way, we force the classifiers to learn from refined feature subspaces and data subsets to fit the imbalanced data better. Hence, the tree model becomes more robust for text categorization task with imbalanced dataset. Experimental results on various benchmark imbalanced text datasets (Reuters-21578, Ohsumed, and imbalanced 20 newsgroup) consistently demonstrate the effectiveness of our proposed ForesTexter method. The performance of our proposed approach is competitive against the standard random forest and different variants of SVM algorithms.

Tcm-Rf : Hedging The Predictions Of Random Forest

A novel classification method of microarray with reliability and confidence

Heterogeneous Random Forest

Using random forest for reliable classification and cost-sensitive learning for medical diagnosis

A New Rotation Forest Ensemble Algorithm

Multinomial random forest

A Novel Consistent Random Forest Framework: Bernoulli Random Forests

Multinomial Random Forest: Toward Consistency and Privacy-Preservation

Three-way Selection Random Forest Algorithm Based on Decision Boundary Entropy

Hedging Forecast Combinations With an Application to the Random Forest

Predicting RNA Molecular Specific Hybridization via Random Forest

Stock Price Prediction Based on Optimized Random Forest Model.

Unsupervised Learning With Random Forest Predictors

Optimization of RF to Alloy Elastic Modulus Prediction Based on Cuckoo Algorithm

A New Random Forest Ensemble of Intuitionistic Fuzzy Decision Trees

ForesTexter: an Efficient Random Forest Algorithm for Imbalanced Text Categorization

MTForest: Ensemble Decision Trees based on Multi-Task Learning

A Modified Random Forest Based on Kappa Measure and Binary Artificial Bee Colony Algorithm

A two-phase random forest with differential privacy

Extreme random forest method for machine fault classification

Probabilistic Random Forest: A machine learning algorithm for noisy datasets