Abstract:The performance of ID3 algorithm in decision tree depends on the information gain but it has a drawback because of tending to select attributes with many values as the branching attributes. The gain ratio (especially in C4.5) is proposed to improve the information gain, but it does not always improve the performance, nor is it always defined. Some scientists use normalized information distance to improve the gain ratio, however, it is ineffective. In this paper, we investigate two non-normalized information distance selection criteria to replace the information gain and the gain ratio and conduct detailed experiments on 13 datasets classified into four types with theoretical analysis. Surprisingly, on the datasets where the number of values of each attribute differ greatly i.e. in Type1 and Type2, non-normalized information distance-based algorithms can increase the accuracy of about 15-25% of ID3 algorithm. The first reason is that more values for an attribute does not reduce the distances, which is suggested by Mantaras. The second reason is that the conditional entropy which is the opposite one used in the information gain can bring balance to the multi-valued biased values. Furthermore, our methods can maintain results comparable to those of existing algorithms on other cases. Compared to the gain ratio, the algorithms with non-normalized information distances conquer the drawback much better on Type1 datasets, which is strongly confirmed by experiments and corresponding analysis. It can be presumed that "normalization" improvement methods such as normalized information distance and the gain ratio are not always effective.

The research and realization of decision trees based on credibility

Decision Tree Inductive Learning Algorithm Based on Removing Noise Gradually

Analysis and application of new shares based on decision tree

A Kind Of Inductive Learning Classification Algorithm Based On Statistics Rule

Heuristic Mode Research and Application of Decision Tree Algorithm

On the Optimality of Trees Generated by ID3

Application of Improved Decision Tree Algorithm

A modified ID3 decision tree algorithm based on cumulative residual entropy

Decision Tree Construction Method Based on Principal Component Analysis

Research Method of Custom Forfeit Crisis Based on Decision Tree

Building Fuzzy Neural Classifiers by Fuzzy Id3 Algorithm

Heuristic Model and Application of Decision Tree Based on Quasi-Linear Criterion

Optimization of the Decision Tree Algorithm about of Interest Degree about Customers

Application of variable precision rough set model in building decision tree

Fuzzy Id3 Algorithm Based On Generating Hartley Measure

Performance Improvement Validation of Decision Tree Algorithms with Non-normalized Information Distance in Experiments

Construction of College Chinese Blended Teaching Mode Based on Decision Tree Classification Model in New Media Context

An Improvement On The Algorithm Of Decision Tree

Heuristic Mode Research Of Decision Tree Algorithm Based On Synthesizing Effect

Research on Evaluation Model of Hospital Informatization Level Based on Decision Tree Algorithm

A New ID3 Algorithm Based on Revised Information Gain