Abstract:Existing deep convolutional neural networks (CNNs) have shown their great success on image classification. CNNs mainly consist of convolutional and pooling layers, both of which are performed on local image areas without considering the dependencies among different image regions. However, such dependencies are very important for generating explicit image representation. In contrast, recurrent neural networks (RNNs) are well known for their ability of encoding contextual information among sequential data, and they only require a limited number of network parameters. General RNNs can hardly be directly applied on non-sequential data. Thus, we proposed the hierarchical RNNs (HRNNs). In HRNNs, each RNN layer focuses on modeling spatial dependencies among image regions from the same scale but different locations. While the cross RNN scale connections target on modeling scale dependencies among regions from the same location but different scales. Specifically, we propose two recurrent neural network models: 1) hierarchical simple recurrent network (HSRN), which is fast and has low computational cost; and 2) hierarchical long-short term memory recurrent network (HLSTM), which performs better than HSRN with the price of more computational cost. In this manuscript, we integrate CNNs with HRNNs, and develop end-to-end convolutional hierarchical recurrent neural networks (C-HRNNs). C-HRNNs not only make use of the representation power of CNNs, but also efficiently encodes spatial and scale dependencies among different image regions. On four of the most challenging object/scene image classification benchmarks, our C-HRNNs achieve state-of-the-art results on Places 205, SUN 397, MIT indoor, and competitive results on ILSVRC 2012.

Convolutional neural network learning for generic data classification.

EDLT: Enabling Deep Learning for Generic Data Classification

Learning Sensor-Specific Spatial-Spectral Features of Hyperspectral Images Via Convolutional Neural Networks

Hierarchical Transfer Convolutional Neural Networks for Image Classification

Feature Learning For SAR Images Using Convolutional Neural Network.

Exploring Kernel Based Spatial Context for CNN Based Hyperspectral Image Classification.

HCFNN: High-order Coverage Function Neural Network for Image Classification

Transform-Invariant Convolutional Neural Networks for Image Classification and Search

On the Large-Scale Transferability of Convolutional Neural Networks.

Coarse-to-fine Trained Multi-Scale Convolutional Neural Networks for Image Classification.

Towards Learning Spatially Discriminative Feature Representations

Learning a Smart Convolutional Neural Network with High-Level Semantic Information

Comparable Study Of Convolutional Neural Networks In Classification And Feature Extraction Applications

Dense Attention Convolutional Network for Image Classification

Learning Geometric Invariance Features and Discrimination Representation for Image Classification via Spatial Transform Network and XGBoost Modeling

An Ensemble Learning Framework for Convolutional Neural Network Based on Multiple Classifiers

Study Of The Application Of Deep Convolutional Neural Networks (Cnns) In Processing Sensor Data And Biomedical Images

Twisted Convolutional Networks (TCNs): Enhancing Feature Interactions for Non-Spatial Data Classification

Learning Contextual Dependencies with Convolutional Hierarchical Recurrent Neural Networks

Generalizing Long Short-Term Memory Network for Deep Learning from Generic Data

Robust Neighborhood Preserving Low-Rank Sparse CNN Features for Classification.