Abstract:Abstract We apply multiparty computation (MPC) techniques to show, given a database that is secret-shared among multiple mutually distrustful parties, how the parties may obliviously construct a decision tree based on the secret data. We consider data with continuous attributes (i.e., coming from a large domain), and develop a secure version of a learning algorithm similar to the C4.5 or CART algorithms. Previous MPC-based work only focused on decision tree learning with discrete attributes (De Hoogh et al. 2014). Our starting point is to apply an existing generic MPC protocol to a standard decision tree learning algorithm, which we then optimize in several ways. We exploit the fact that even if we allow the data to have continuous values, which a priori might require fixed or floating point representations, the output of the tree learning algorithm only depends on the relative ordering of the data. By obliviously sorting the data we reduce the number of comparisons needed per node to O ( N log 2 N ) from the naive O ( N 2 ), where N is the number of training records in the dataset, thus making the algorithm feasible for larger datasets. This does however introduce a problem when duplicate values occur in the dataset, but we manage to overcome this problem with a relatively cheap subprotocol. We show a procedure to convert a sorting network into a permutation network of smaller complexity, resulting in a round complexity of O (log N ) per layer in the tree. We implement our algorithm in the MP-SPDZ framework and benchmark our implementation for both passive and active three-party computation using arithmetic modulo 2 64 . We apply our implementation to a large scale medical dataset of ≈ 290 000 rows using random forests, and thus demonstrate practical feasibility of using MPC for privacy-preserving machine learning based on decision trees for large datasets.

Constructing decision tree with continuous attributes for binary classification

Rough set based decision tree

An improved decision tree algorithm based on boundary mixed attribute dependency

Method of Discretization of Continuous Attributes of Decision Table

A cost sensitive decision tree algorithm with two adaptive mechanisms

Novel Design of Decision-Tree-Based Support Vector Machines Multi-Class Classifier

A New Discretization Approach of Continuous Attributes

Discretization of Continuous-Valued Attributes and Instance-Based Learning

Attribute reduction and optimal decision rules acquisition for continuous valued information systems

The Max-Cut Decision Tree: Improving on the Accuracy and Running Time of Decision Trees

Quant-BnB: A Scalable Branch-and-Bound Method for Optimal Decision Trees with Continuous Features

Building decision trees for the multi-class imbalance problem

Learning with continuous piecewise linear decision trees

Secure training of decision trees with continuous attributes

Speeding Up Very Fast Decision Tree with Low Computational Cost

AH3: An adaptive hierarchical feature representation model for three-way decision boundary processing

Induction of Hybrid Decision Tree Based on Post-Discretization Strategy*

Efficient Tree Classifiers for Large Scale Datasets.

A Constructing Method of Decision Tree and Classification Rule Extraction for Incomplete Information System

A less-greedy two-term Tsallis Entropy Information Metric approach for decision tree classification

Belief Entropy Tree and Random Forest: Learning from Data with Continuous Attributes and Evidential Labels