Abstract:In (\cite{zhang2014nonlinear,zhang2014nonlinear2}), we have viewed machine learning as a coding and dimensionality reduction problem, and further proposed a simple unsupervised dimensionality reduction method, entitled deep distributed random samplings (DDRS). In this paper, we further extend it to supervised learning incrementally. The key idea here is to incorporate label information into the coding process by reformulating that each center in DDRS has multiple output units indicating which class the center belongs to. The supervised learning method seems somewhat similar with random forests (\cite{breiman2001random}), here we emphasize their differences as follows. (i) Each layer of our method considers the relationship between part of the data points in training data with all training data points, while random forests focus on building each decision tree on only part of training data points independently. (ii) Our method builds gradually-narrowed network by sampling less and less data points, while random forests builds gradually-narrowed network by merging subclasses. (iii) Our method is trained more straightforward from bottom layer to top layer, while random forests build each tree from top layer to bottom layer by splitting. (iv) Our method encodes output targets implicitly in sparse codes, while random forests encode output targets by remembering the class attributes of the activated nodes. Therefore, our method is a simpler, more straightforward, and maybe a better alternative choice, though both methods use two very basic elements---randomization and nearest neighbor optimization---as the core. This preprint is used to protect the incremental idea from (\cite{zhang2014nonlinear,zhang2014nonlinear2}). Full empirical evaluation will be announced carefully later.

Neural Forest Learning.

Neural Random Subspace

UDRN: Unified Dimensional Reduction Neural Network for Feature Selection and Feature Projection

Very High Resolution Remote Sensing Imagery Classification Using a Fusion of Random Forest and Deep Learning Technique—Subtropical Area for Example

Forward Thinking: Building Deep Random Forests

Random Bits Forest: a Strong Classifier/Regressor for Big Data

A Novel Feature Subspace Selection Method in Random Forests for High Dimensional Data.

A New Rotation Forest Ensemble Algorithm

Classification of Spatiotemporal Events Based on Random Forest

Random Survival Forests Incorporated by the Nadaraya-Watson Regression

Towards the effectiveness of Deep Convolutional Neural Network based Fast Random Forest Classifier

Understanding The Prediction Process Of Deep Networks By Forests

Randomnet: clustering time series using untrained deep neural networks

Nonparametric feature selection by random forests and deep neural networks

The random subspace method for constructing decision forests

GrCAN: Gradient Boost Convolutional Autoencoder with Neural Decision Forest

A Deep Neural Network Model using Random Forest to Extract Feature Representation for Gene Expression Data Classification

Deep Distributed Random Samplings for Supervised Learning: An Alternative to Random Forests?

A Deep Random Forest Model on Spark for Network Intrusion Detection

Exploiting random projections and sparsity with random forests and gradient boosting methods -- Application to multi-label and multi-output learning, random forest model compression and leveraging input sparsity