Abstract:Random forest (RF) is a very important classifier with applications in various machine learning tasks, but its promising performance heavily relies on the size of labeled training data. In this paper, we investigate constructing of RFs with a small size of labeled data and find that the performance bottleneck is located in the node splitting procedures; hence, existing solutions fail to properly partition the feature space if there are insufficient training data. To achieve robust node splitting with insufficient data, we present semisupervised splitting to overcome this limitation by splitting nodes with the guidance of both labeled and abundant unlabeled data. In particular, an accurate quality measure of node splitting is obtained by carrying out the kernel-based density estimation, whereby a multiclass version of asymptotic mean integrated squared error criterion is proposed to adaptively select the optimal bandwidth of the kernel. To avoid the curse of dimensionality, we project the data points from the original high-dimensional feature space onto a low-dimensional subspace before estimation. A unified optimization framework is proposed to select a coupled pair of subspace and separating hyperplane such that the smoothness of the subspace and the quality of the splitting are guaranteed simultaneously. Our algorithm efficiently avoids overfitting caused by bad initialization and local maxima when compared with conventional margin maximization-based semisupervised methods. We demonstrate the effectiveness of the proposed algorithm by comparing it with state-of-the-art supervised and semisupervised algorithms for typical computer vision applications, such as object categorization, face recognition, and image segmentation, on publicly available data sets.

On the Robust Splitting Criterion of Random Forest.

Adaptive Split Balancing for Optimal Random Forest

Best-scored Random Forest Classification

Distributional Adaptive Soft Regression Trees

Statistical analysis of various splitting criteria for decision trees

The random subspace method for constructing decision forests

Semi-supervised Node Splitting for Random Forest Construction

Statistical Advantages of Oblique Randomized Decision Trees and Forests

A Forest of Trees with Principal Direction Specified Oblique Split on Random Subspace.

Riemann-Lebesgue Forest for Regression

Risk bounds for purely uniformly random forests

Asymptotic Properties of High-Dimensional Random Forests

Unifying Decision Trees Split Criteria Using Tsallis Entropy

Ordinal Trees and Random Forests: Score-Free Recursive Partitioning and Improved Ensembles

Stochastic Optimization Forests

Multinomial random forest

Random forest construction with robust semisupervised node splitting.

Era Splitting -- Invariant Learning for Decision Trees

Unifying Attribute Splitting Criteria of Decision Trees by Tsallis Entropy.

Weighted Oblique Decision Trees

Randomization Can Reduce Both Bias and Variance: A Case Study in Random Forests