Abstract:Random Forest is an ensemble of decision trees based on the bagging and random subspace concepts. As suggested by Breiman, the strength of unstable learners and the diversity among them are the ensemble models' core strength. In this paper, we propose two approaches known as oblique and rotation double random forests. In the first approach, we propose rotation based double random forest. In rotation based double random forests, transformation or rotation of the feature space is generated at each node. At each node different random feature subspace is chosen for evaluation, hence the transformation at each node is different. Different transformations result in better diversity among the base learners and hence, better generalization performance. With the double random forest as base learner, the data at each node is transformed via two different transformations namely, principal component analysis and linear discriminant analysis. In the second approach, we propose oblique double random forest. Decision trees in random forest and double random forest are univariate, and this results in the generation of axis parallel split which fails to capture the geometric structure of the data. Also, the standard random forest may not grow sufficiently large decision trees resulting in suboptimal performance. To capture the geometric properties and to grow the decision trees of sufficient depth, we propose oblique double random forest. The oblique double random forest models are multivariate decision trees. At each non-leaf node, multisurface proximal support vector machine generates the optimal plane for better generalization performance. Also, different regularization techniques (Tikhonov regularisation, axis-parallel split regularisation, Null space regularisation) are employed for tackling the small sample size problems in the decision trees of oblique double random forest. The proposed ensembles of decision trees produce trees with bigger size compared to the standard ensembles of decision trees as bagging is used at eah which results in improved performance. The evaluation of the baseline models and the proposed oblique and rotation double random forest models is performed on benchmark 121 UCI datasets and real-world fisheries datasets. Both statistical analysis and the experimental results demonstrate the efficacy of the proposed oblique and rotation double random forest models compared to the baseline models on the benchmark datasets.

Random Ensemble of Fuzzy Rule-Based Models.

A New Rotation Forest Ensemble Algorithm

Fuzzy Rule-Based Models with Randomized Development Mechanisms

A New Random Forest Ensemble of Intuitionistic Fuzzy Decision Trees

An Integrated Fusion Framework for Ensemble Learning Leveraging Gradient Boosting and Fuzzy Rule-Based Models

MTForest: Ensemble Decision Trees based on Multi-Task Learning

Oblique Random Forest Ensemble Via Least Square Estimation for Time Series Forecasting.

BoostTree and BoostForest for Ensemble Learning

Constructing Multiple Support Vector Machines Ensemble Based On Fuzzy Integral And Rough Reducts

A Variant of Rotation Forest for Constructing Ensemble Classifiers

Parameter-Free Robust Ensemble Framework of Fuzzy Clustering

A Dynamic Boosted Ensemble Learning Method Based on Random Forest

A Development of Fuzzy Rule-based Regression Models through Using Decision Trees

Oblique and rotation double random forest

A Modular Neural Networks Ensembling Method Based on Fuzzy Decision-Making

Random Forests with Ensemble of Feature Spaces.

Towards generating random forests via extremely randomized trees

Beyond Discriminant Patterns: On the Robustness of Decision Rule Ensembles

Design of Fuzzy Ensemble Architecture Realized with the Aid of FCM-Based Fuzzy Partition and NN with Weighted LSE Estimation

A Federated F-score Based Ensemble Model for Automatic Rule Extraction

An Ensemble Extended Belief Rule Base Decision Model for Imbalanced Classification Problems