Abstract:Neural Networks and Decision Trees: two popular techniques for supervised learning that are seemingly disconnected in their formulation and optimization method, have recently been combined in a single construct. The connection pivots on assembling an artificial Neural Network with nodes that allow for a gate-like function to mimic a tree split, optimized using the standard approach of recursively applying the chain rule to update its parameters. Yet two main challenges have impeded wide use of this hybrid approach: (a) the inability of global gradient ascent techniques to optimize hierarchical parameters (as introduced by the gate function); and (b) the construction of the tree structure, which has relied on standard decision tree algorithms to learn the network topology or incrementally (and heuristically) searching the space at random. Here we propose a probabilistic construct that exploits the idea of a node's unexplained potential (the total error channeled through the node) in order to decide where to expand further, mimicking the standard tree construction in a Neural Network setting, alongside a modified gradient ascent that first locally optimizes an expanded node before a global optimization. The probabilistic approach allows us to evaluate each new split as a ratio of likelihoods that balances the statistical improvement in explaining the evidence against the additional model complexity --- thus providing a natural stopping condition. The result is a novel classification and regression technique that leverages the strength of both: a tree-structure that grows naturally and is simple to interpret with the plasticity of Neural Networks that allow for soft margins and slanted boundaries.

Improving The Performance Of Decision Tree: A Hybrid Approach

Induction of Hybrid Decision Tree Based on Post-Discretization Strategy*

A Technique For Improving The Performance Of Naive Bayes Text Classification

Improving Naive Bayes Classifier by Dividing Its Decision Regions

Improving performance of decision tree algorithms with multi-edited nearest neighbor rule

bsnsing: A decision tree induction method based on recursive optimal boolean rule composition

Improving the Performance of Boosting for Naive Bayesian Classification

Learning hybrid Bayesian network based on divide and conquer strategy

Lazy Bayesian Rules: A Lazy Semi-Naive Bayesian Learning Technique Competitive to Boosting Decision Trees

for Further Innproving the Performance of Decision Tree Learning

Improving Performance of Decision Trees with Multi-Edit-nearest-neighbor Algorithm

Induction of a Novel Hybrid Decision Forest Model Based on Information Theory.

A New Method for Learning Decision Tree Classifier

Quant-BnB: A Scalable Branch-and-Bound Method for Optimal Decision Trees with Continuous Features

Adaptive Bayesian Reticulum

Structure Extension of Tree-Augmented Naive Bayes

A Method to Boost Naïve Bayesian Classifiers

Boosting-Based Sequential Meta-Tree Ensemble Construction for Improved Decision Trees

Taiga: Performance Optimization of the C4.5 Decision Tree Construction Algorithm

An Improved Decision Tree Algorithm Based on Variable Precision Neighborhood Similarity

Bayesian neural tree models for nonparametric regression