Abstract:Decision Tree is a well understood Machine Learning model that is based on minimizing impurities in the internal nodes. The most common impurity measures are Shannon entropy and Gini impurity. These impurity measures are insensitive to the order of training data and hence the final tree obtained is invariant to any permutation of the data. This is a limitation in terms of modeling when there are temporal order dependencies between data instances. In this research, we propose the adoption of Effort-To-Compress (ETC) - a complexity measure, for the first time, as an alternative impurity measure. Unlike Shannon entropy and Gini impurity, structural impurity based on ETC is able to capture order dependencies in the data, thus obtaining potentially different decision trees for different permutations of the same data instances, a concept we term as Permutation Decision Trees (PDT). We then introduce the notion of Permutation Bagging achieved using permutation decision trees without the need for random feature selection and sub-sampling. We conduct a performance comparison between Permutation Decision Trees and classical decision trees across various real-world datasets, including Appendicitis, Breast Cancer Wisconsin, Diabetes Pima Indian, Ionosphere, Iris, Sonar, and Wine. Our findings reveal that PDT demonstrates comparable performance to classical decision trees across most datasets. Remarkably, in certain instances, PDT even slightly surpasses the performance of classical decision trees. In comparing Permutation Bagging with Random Forest, we attain comparable performance to Random Forest models consisting of 50 to 1000 trees, using merely 21 trees. This highlights the efficiency and effectiveness of Permutation Bagging in achieving comparable performance outcomes with significantly fewer trees.

Parallel Ordinal Decision Tree Algorithm and Its Implementation in Framework of MapReduce.

DTRadar: Accelerating Search Process of Decision Trees in Packet Classification.

DOT: Towards Fast Decision Tree Packet Classification by Optimizing Rule Partitions

A parallel fuzzy rule-base based decision tree in the framework of map-reduce

A Communication-Efficient Parallel Algorithm for Decision Tree

FROD: an Efficient Framework for Optimizing Decision Trees in Packet Classification

Ordinal Outlier Detection Based On Recursive Uniform Partitioning

Parallel incremental efficient attribute reduction algorithm based on attribute tree

A Parallel Attribute Reduction Method Based on Classification

Evaluating Large Graph Processing in MapReduce Based on Message Passing

Novel Design of Decision-Tree-Based Support Vector Machines Multi-Class Classifier

Fast Supervised Hashing with Decision Trees for High-Dimensional Data

An improved parallel association rules algorithm based on MapReduce framework for big data

Single MCMC Chain Parallelisation on Decision Trees

Efficient Tree Classifiers for Large Scale Datasets.

Heuristic Mode Research and Application of Decision Tree Algorithm

Permutation Decision Trees

Weighted Oblique Decision Trees

On Implementation of Parallel Map and Shuffle Phases for Coded Distributed Computing

Intermediate Value Size Aware Coded MapReduce.

Speeding Up Very Fast Decision Tree with Low Computational Cost