Abstract:Decision Trees (DTs) are a class of supervised learning models that are widely used for both classification and regression applications. They are well-known for their interpretability and robustness, which have led them to remain popular even 60 years after they were first proposed. However, because traditional tree algorithms use greedy methods that are prone to suboptimality, several works have explored the usage of evolutionary algorithms instead. Although these algorithms are often reported to outperform the traditional greedy approach, their computational cost is much higher, since the evolutionary component requires a large number (millions or billions) of function evaluations in order to produce a single tree. Aiming to reduce this computational cost, in this work we propose an encoding that allows the training and evaluation of DTs using only matrix operations. The proposed procedure is shown to be much faster than the traditional tree implementation for complete trees with depths ranging from 2 to 6, and for datasets ranging in size from 100 to 100,000 observations. In particular, the results show speedups of nearly up to 20 times, especially when the dataset is large and the desired tree is small enough to be interpretable. The proposed procedure also benefits from GPU parallelization, although it is still highly performing without it. Furthermore, we propose an evolutionary algorithm, called Coral Reef Optimization for Decision Trees (CRO-DT), that integrates this encoding with a pre-existing ensemble algorithm to evolve better univariate trees. The results obtained show that the proposed CRO-DT is competitive with traditional and modern tree algorithms, consistently producing models of good quality across 14 tested UCI Datasets. We conclude that for most relevant situations, the proposed matrix encoding provides significant speedups over the traditional implementation, and also may serve as a basis for high quality evolutionary DT algorithms.

Learning decision trees through Monte Carlo tree search: An empirical evaluation

Optimized Monte Carlo Tree Search for Enhanced Decision Making in the FrozenLake Environment

Decision Tree Learning for Uncertain Clinical Measurements

An Analysis on the Effects of Evolving the Monte Carlo Tree Search Upper Confidence for Trees Selection Policy on Unimodal, Multimodal and Deceptive Landscapes

Efficient evolution of decision trees via fully matrix-based fitness evaluation

Bayesian Decision Trees Inspired from Evolutionary Algorithms

Doing Better Than UCT: Rational Monte Carlo Sampling in Trees

A Survey of Monte Carlo Tree Search Methods

Generalized Mean Estimation in Monte-Carlo Tree Search

Online Learning of Decision Trees with Thompson Sampling

Monte Carlo Tree Search with Boltzmann Exploration

Better trees: an empirical study on hyperparameter tuning of classification decision tree induction algorithms

Recent advances in decision trees: an updated survey

Monte Carlo Tree Search: a review of recent modifications and applications

A New Method for Learning Decision Tree Classifier

Monte Carlo Search Algorithms Discovering Monte Carlo Tree Search Exploration Terms

Monte Carlo Tree Search in the Presence of Transition Uncertainty

An Optimal Computing Budget Allocation Tree Policy for Monte Carlo Tree Search

Towards Understanding the Effects of Evolving the MCTS UCT Selection Policy

An Efficient Dynamic Sampling Policy for Monte Carlo Tree Search.

RJHMC-Tree for Exploration of the Bayesian Decision Tree Posterior