Abstract:Random forest is one of the most widely used machine learning algorithms. Decision trees used to construct the random forest may have low classification accuracies or high correlations, which affects the comprehensive performance of the random forest. Aiming at these problems, the authors proposed an improved random forest based on the classification accuracy and correlation measurement of decision trees in this paper. Its main idea includes two parts, one is retaining the classification and regression trees (CARTs) with better classification effects, the other is reducing the correlations between the CARTs. Specifically, in the classification effect evaluation part, each CART was applied to make predictions on three reserved data sets, then the average classification accuracies were achieved, respectively. Thus, all the CARTs were sorted in descending order according to their achieved average classification accuracies. In the correlation measurement part, the improved dot product method was proposed to calculate the cosine similarity, i.e., the correlation, between CARTs in the feature space. By using the achieved average classification accuracy as reference, the grid search method was used to find the inner product threshold. On this basis, the CARTs with low average classification accuracy among CART pairs whose inner product values are higher than the inner product threshold were marked as deletable. The achieved average classification accuracies and correlations of CARTs were comprehensively considered, those with high correlation and weak classification effect were deleted, and those with better quality were retained to construct the random forest. Multiple experiments show that, the proposed improved random forest achieved higher average classification accuracy than the five random forests used for comparison, and the lead was stable. The G-means and out-of-bag data (OBD) score obtained by the proposed improved random forest were also higher than the five random forests, and the lead was more obvious. In addition, the test results of three non-parametric tests show that, there were significant diversities between the proposed improved random forest and the other five random forests. This effectively proves the superiority and practicability of the proposed improved random forest.

Optimizing the number of branches in a decision forest using association rule metrics

Random Forest Pruning Techniques: A Recent Review

On Extreme Pruning of Random Forest Ensembles for Real-time Predictive Applications

Optimally Pruning Decision Tree Ensembles With Feature Cost

An Ensemble Framework to Improve the Accuracy of Prediction Using Clustered Random-Forest and Shrinkage Methods

Ensemble of optimal trees, random forest and random projection ensemble classification

A New Rotation Forest Ensemble Algorithm

Prediction of Student Performance Using Random Forest Combined With Naïve Bayes

Alpha-Trimming: Locally Adaptive Tree Pruning for Random Forests

Forest-ORE: Mining Optimal Rule Ensemble to interpret Random Forest models

Theoretical and Empirical Advances in Forest Pruning

Trees, Forests, Chickens, and Eggs: When and Why to Prune Trees in a Random Forest

MTForest: Ensemble Decision Trees based on Multi-Task Learning

An improved random forest based on the classification accuracy and correlation measurement of decision trees

A Modified Random Forest Based on Kappa Measure and Binary Artificial Bee Colony Algorithm

Mondrian Forests: Efficient Online Random Forests

MEPSI: an MDL-Based Ensemble Pruning Approach with Structural Information

On Computing Optimal Tree Ensembles

Free Lunch in the Forest: Functionally-Identical Pruning of Boosted Tree Ensembles

An optimal pruning algorithm of classifier ensembles: dynamic programming approach

VisRuler: Visual Analytics for Extracting Decision Rules from Bagged and Boosted Decision Trees