Abstract:Deep neural networks are a powerful tool for feature learning and extraction given their ability to model high-level abstractions in highly complex data. One area worth exploring in feature learning and extraction using deep neural networks is efficient neural connectivity formation for faster feature learning and extraction. Motivated by findings of stochastic synaptic connectivity formation in the brain as well as the brain's uncanny ability to efficiently represent information, we propose the efficient learning and extraction of features via StochasticNets, where sparsely-connected deep neural networks can be formed via stochastic connectivity between neurons. To evaluate the feasibility of such a deep neural network architecture for feature learning and extraction, we train deep convolutional StochasticNets to learn abstract features using the CIFAR-10 dataset, and extract the learned features from images to perform classification on the SVHN and STL-10 datasets. Experimental results show that features learned using deep convolutional StochasticNets, with fewer neural connections than conventional deep convolutional neural networks, can allow for better or comparable classification accuracy than conventional deep neural networks: relative test error decrease of ~4.5% for classification on the STL-10 dataset and ~1% for classification on the SVHN dataset. Furthermore, it was shown that the deep features extracted using deep convolutional StochasticNets can provide comparable classification accuracy even when only 10% of the training data is used for feature learning. Finally, it was also shown that significant gains in feature extraction speed can be achieved in embedded applications using StochasticNets. As such, StochasticNets allow for faster feature learning and extraction performance while facilitate for better or comparable accuracy performances.

Learning Sparse Feature Representations using Probabilistic Quadtrees and Deep Belief Nets

Sparse Representation for Three-Dimensional Number Ball Recognition

Pixel-level Reconstruction and Classification for Noisy Handwritten Bangla Characters

Handwritten Digit Recognition Via Active Belief Decision Trees

A Fast Learning Algorithm for Deep Belief Nets

A Unified Feature Representation and Learning Framework for 3D Shape

Sparse Concept Coded Tetrolet Transform for Unconstrained Odia Character Recognition

Attention-based hand semantic segmentation and gesture recognition using deep networks

Bayesian Self-Training for Semi-Supervised 3D Segmentation

Deep Neural Networks for Learning Graph Representations

Sparse Deep Stacking Network for Image Classification

Learning High-Level Feature by Deep Belief Networks for 3-D Model Retrieval and Recognition.

BDNet: Bengali Handwritten Numeral Digit Recognition based on Densely connected Convolutional Neural Networks

Multi-Object Classification and Unsupervised Scene Understanding Using Deep Learning Features and Latent Tree Probabilistic Models

Training Deep Neural Networks with Discrete State Transition

Efficient Deep Feature Learning and Extraction via StochasticNets

Deep Self-Taught Learning for Handwritten Character Recognition

Reinforcement Learning Based Handwritten Digit Recognition with Two-State Q-Learning

Spatiotemporal features representation with dynamic mode decomposition for hand gesture recognition using deep neural networks

Instance-level Sketch-based Retrieval by Deep Triplet Classification Siamese Network