Abstract:Neural Networks and Decision Trees: two popular techniques for supervised learning that are seemingly disconnected in their formulation and optimization method, have recently been combined in a single construct. The connection pivots on assembling an artificial Neural Network with nodes that allow for a gate-like function to mimic a tree split, optimized using the standard approach of recursively applying the chain rule to update its parameters. Yet two main challenges have impeded wide use of this hybrid approach: (a) the inability of global gradient ascent techniques to optimize hierarchical parameters (as introduced by the gate function); and (b) the construction of the tree structure, which has relied on standard decision tree algorithms to learn the network topology or incrementally (and heuristically) searching the space at random. Here we propose a probabilistic construct that exploits the idea of a node's unexplained potential (the total error channeled through the node) in order to decide where to expand further, mimicking the standard tree construction in a Neural Network setting, alongside a modified gradient ascent that first locally optimizes an expanded node before a global optimization. The probabilistic approach allows us to evaluate each new split as a ratio of likelihoods that balances the statistical improvement in explaining the evidence against the additional model complexity --- thus providing a natural stopping condition. The result is a novel classification and regression technique that leverages the strength of both: a tree-structure that grows naturally and is simple to interpret with the plasticity of Neural Networks that allow for soft margins and slanted boundaries.

A Bayesian Classifier by Using the Adaptive Construct Algorithm of the RBF Networks

A Bayesian Classifier by Using the Merging RBF Networks

A Nonlinear Adaptive Controller Based on RBF Networks

Max-margin Based Bayesian Classifier.

Adaptive RBF Neural Networks for Pattern Classifications

An Adaptive Classifier Based on Artificial Immune Network

Learning Bayesian Networks Using the Constrained Maximum a Posteriori Probability Method

Function Approximation Based on Self-Adaptive RBF Neural Network with Combined Clustering Algorithm

Adaptive Bayesian Reticulum

Classification Algorithm Based on Constructive RBF Neuron Networks

A New RBF Neuron Network Classification Algorithm

An Adaptive Learning Algorithm Aimed at Improving RBF Network Generalization Ability

Stochastic Optimization for Bayesian Network Classifiers

A Novel Two Stage Algorithm for Construction of RBF Neural Models Based on A-optimality Criterion

A GA-based RBF classifier with class-dependent features

A Ga-Based Novel Rbf Classifier With Class-Dependent Features

Optimal Classifier Fusion in a Non-Bayesian Probabilistic Framework

Design and Realization of New C-RBF Neural Networks Classifier

RBF neural network classifier based on manifold analysis and AP algorithm

An Error-Driven Learning Algorithm for Bayesian Network Classifiers

Optimizing Radial Basis Function Neural Network Based on Rough Sets and Affinity Propagation Clustering Algorithm