Abstract:Classification can be regarded as dividing the data space into decision regions separated by decision boundaries. In this paper we analyze decision tree algorithms and the NBTree algorithm from this perspective. Thus, a decision tree can be regarded as a classifier tree, in which each classifier on a non-root node is trained in decision regions of the classifier on the parent node. Meanwhile, the NBTree algorithm, which generates a classifier tree with the C4.5 algorithm and the naive Bayes classifier as the root and leaf classifiers respectively, can also be regarded as training naive Bayes classifiers in decision regions of the C4.5 algorithm. We propose a second division (SD) algorithm and three soft second division (SD-soft) algorithms to train classifiers in decision regions of the naive Bayes classifier. These four novel algorithms all generate two-level classifier trees with the naive Bayes classifier as root classifiers. The SD and three SD-soft algorithms can make good use of both the information contained in instances near decision boundaries, and those that may be ignored by the naive Bayes classifier. Finally, we conduct experiments on 30 data sets from the UC Irvine (UCI) repository. Experiment results show that the SD algorithm can obtain better generalization abilities than the NBTree and the averaged one-dependence estimators (AODE) algorithms when using the C4.5 algorithm and support vector machine (SVM) as leaf classifiers. Further experiments indicate that our three SD-soft algorithms can achieve better generalization abilities than the SD algorithm when argument values are selected appropriately.

for Further Innproving the Performance of Decision Tree Learning

Integrating Boosting and Stochastic Attribute Selection Committees for Further Improving the Performance of Decision Tree Learning

Boosting Trees for Cost-Sensitive Classifications.

Improving the Performance of Boosting for Naive Bayesian Classification

Improved AdaBoost.M1 of Decision Trees with Confidence-Rated Predictions

Tree-Structured Boosting: Connections Between Gradient Boosted Stumps and Full Decision Trees

Improving Classification Performance By Combining Multiple Tan Classifiers

Boosting Cost-Sensitive Trees

Boosting Support Vector Machines Successfully.

Boosting-Based Sequential Meta-Tree Ensemble Construction for Improved Decision Trees

Effective Boosting of Naïve Bayesian Classifiers by Local Accuracy Estimation

Dbboost: Enhancing Imbalanced Classification by A Novel Ensemble Based Technique

Improving The Performance Of Decision Tree: A Hybrid Approach

Experiments with a New Boosting Algorithm

A New Method for Learning Decision Tree Classifier

Improving Naive Bayes Classifier by Dividing Its Decision Regions

RotBoost: A technique for combining Rotation Forest and AdaBoost

Study on GA-based SVM multi-class classification decision-tree optimization algorithm

A Method to Boost Naïve Bayesian Classifiers

Improved boosting algorithm through weighted k-nearest neighbors classifier

ISABoost: A weak classifier inner structure adjusting based AdaBoost algorithm-ISABoost based application in scene categorization