Abstract:The rise of machine learning methods on heavily resource constrained devices requires not only the choice of a suitable model architecture for the target platform, but also the optimization of the chosen model with regard to execution time consumption for inference in order to optimally utilize the available resources. Random forests and decision trees are shown to be a suitable model for such a scenario, since they are not only heavily tunable towards the total model size, but also offer a high potential for optimizing their executions according to the underlying memory architecture. In addition to the straightforward strategy of enforcing shorter paths through decision trees and hence reducing the execution time for inference, hardware-aware implementations can optimize the execution time in an orthogonal manner. One particular hardware-aware optimization is to layout the memory of decision trees in such a way, that higher probably paths are less likely to be evicted from system caches. This works particularly well when splits within tree nodes are uneven and have a high probability to visit one of the child nodes. In this paper, we present a method to reduce path lengths by rewarding uneven probability distributions during the training of decision trees at the cost of a minimal accuracy degradation. Specifically, we regularize the impurity computation of the CART algorithm in order to favor not only low impurity, but also highly asymmetric distributions for the evaluation of split criteria and hence offer a high optimization potential for a memory architecture-aware implementation. We show that especially for binary classification data sets and data sets with many samples, this form of regularization can lead to an reduction of up to approximately four times in the execution time with a minimal accuracy degradation.

Hardware Implementation of Random Forest Algorithm Based on Classification and Regression Tree

FROD: an Efficient Framework for Optimizing Decision Trees in Packet Classification

DTRadar: Accelerating Search Process of Decision Trees in Packet Classification.

TREE: Tree Regularization for Efficient Execution

Towards Efficient and Scalable Acceleration of Online Decision Tree Learning on FPGA

Nanosecond hardware regression trees in FPGA at the LHC

A Flexible HLS Hoeffding Tree Implementation for Runtime Learning on FPGA

KickTree: A Recursive Algorithmic Scheme for Packet Classification with BoundedWorst-Case Performance

RF-RISA: A novel flexible random forest accelerator based on FPGA

Taiga: Performance Optimization of the C4.5 Decision Tree Construction Algorithm

A parallel fuzzy rule-base based decision tree in the framework of map-reduce

Breadth-first, Depth-next Training of Random Forests

Deep random forest with ferroelectric analog content addressable memory

A Fully-Pipelined Fpga Design for Tree-Reweighted Message Passing Algorithm

Pipelined training accelerator for portable devices

New machine learning algorithm: random forest

FPGA-Based Updatable Packet Classification Using TSS-Combined Bit-Selecting Tree

Fast and Energy-Efficient Oblique Decision Tree Implementation with Potential Error Detection

Financial Data Center Configuration Management System Based on Random Forest Algorithm and Few-Shot Learning

A Parallel and Updatable Architecture for FPGA-Based Packet Classification With Large-Scale Rule Sets

Updatable Packet Classification on FPGA with Bounded Worst-Case Performance