Abstract:Classification can be regarded as dividing the data space into decision regions separated by decision boundaries. In this paper we analyze decision tree algorithms and the NBTree algorithm from this perspective. Thus, a decision tree can be regarded as a classifier tree, in which each classifier on a non-root node is trained in decision regions of the classifier on the parent node. Meanwhile, the NBTree algorithm, which generates a classifier tree with the C4.5 algorithm and the naive Bayes classifier as the root and leaf classifiers respectively, can also be regarded as training naive Bayes classifiers in decision regions of the C4.5 algorithm. We propose a second division (SD) algorithm and three soft second division (SD-soft) algorithms to train classifiers in decision regions of the naive Bayes classifier. These four novel algorithms all generate two-level classifier trees with the naive Bayes classifier as root classifiers. The SD and three SD-soft algorithms can make good use of both the information contained in instances near decision boundaries, and those that may be ignored by the naive Bayes classifier. Finally, we conduct experiments on 30 data sets from the UC Irvine (UCI) repository. Experiment results show that the SD algorithm can obtain better generalization abilities than the NBTree and the averaged one-dependence estimators (AODE) algorithms when using the C4.5 algorithm and support vector machine (SVM) as leaf classifiers. Further experiments indicate that our three SD-soft algorithms can achieve better generalization abilities than the SD algorithm when argument values are selected appropriately.

NeC4.5: Neural Ensemble Based C4.5

Artificial Neural Network Ensemble For Land Cover Classification

Selective Ensemble Of Decision Trees

Medical Diagnosis with C4.5 Rule Preceded by Artificial Neural Network Ensemble

MTForest: Ensemble Decision Trees based on Multi-Task Learning

Design Neural Network Based on Decision Tree Rules and Its Application Research

Deep Forest: Towards an Alternative to Deep Neural Networks.

Deep Forest: Towards an Alternative to Deep Neural Networks

Rule Learning Based on Neural Network Ensemble

A generalized decision tree ensemble based on the NeuralNetworks architecture: Distributed Gradient Boosting Forest (DGBF)

Genetic Algorithm based Selective Neural Network Ensemble.

Taiga: Performance Optimization of the C4.5 Decision Tree Construction Algorithm

Extracting Symbolic Rules from Trained Neural Network Ensembles

A New Rotation Forest Ensemble Algorithm

Inherently Interpretable Tree Ensemble Learning

Interpretability Diversity for Decision-Tree-Initialized Dendritic Neuron Model Ensemble

Selectively ensembling neural classifiers

Improving Naive Bayes Classifier by Dividing Its Decision Regions

A Survey of Neural Trees: Co-Evolving Neural Networks and Decision Trees

Improving The Performance Of Decision Tree: A Hybrid Approach

The Max-Cut Decision Tree: Improving on the Accuracy and Running Time of Decision Trees