Abstract:There are some traditional pooling methods in convolutional neural network, such as max-pooling, average pooling, stochastic pooling and so on, which determine the results of pooling based on the distribution of each activation in the pooling region. Zeiler and Fergus (Stochastic-pooling for regularization of deep convolutional neural networks, 2013 ) However, it is difficult for the feature mapping process to select a perfect activation representative of the pooling region, and can lead to the phenomenon of over-fitting. In this paper, the following theoretical basis comes out information theory (Shannon in Bell Syst. Tech. J. 27:379–423, 1948 ). First, we quantify the information entropy of each pooling region, and then propose an efficient pooling method by comparing the mutual information between activations and the pooling region which they are located in. Moreover, we assign different weights to different activations based on mutual information, and named it weighted-pooling. The main features of the weighted-pooling method are as follows: (1) The information quantity of the pooling region is quantified by information theory for the first time. (2) Also, each activation’s contribution was quantified for the first time and these contributions eliminate the uncertainty of the pooling region which it is located in. (3) For choosing a representative in this pooling region, the weight of each activation obviously superiors to the value of activation. In the experimental part, we respectively use MNIST and CIFAR-10 (Krizhevsky in Learning multiple layers of featurs from tiny images, University of Toronto, 2009 ; LeCun in The MNIST database, 2012 ) data sets to compare different pooling methods. The results show that the weighted-pooling method has higher recognition accuracy than other pooling methods and reaches a new state-of-the-art.

Deep Image Retrieval: Indicator and Gram Matrix Weighting for Aggregated Convolutional Features

Aggregating Hierarchical Binary Activations for Image Retrieval

Exploring Geometric Information in CNN for Image Retrieval.

Combining Self-Aggregation Vector and Indicator Matrix for Improved Object Retrieval

Image retrieval based on aggregated deep features weighted by regional significance and channel sensitivity

Learning Feature Embedding with Strong Neural Activations for Fine-Grained Retrieval

Gram Matrix Based Representation For Image Retrieval

Adaptive Co-weighting Deep Convolutional Features For Object Retrieval

Unsupervised Semantic-Based Convolutional Features Aggregation for Image Retrieval

Object-Based Aggregation of Deep Features for Image Retrieval.

Multiple Saliency and Channel Sensitivity Network for Aggregated Convolutional Feature

Aggregating Deep Convolutional Features for Image Retrieval Using Multi-regional Cross Weighting

Co-weighting semantic convolutional features for object retrieval.

A Novel Feature Representation: Aggregating Convolution Kernels for Image Retrieval

Deepindex For Accurate And Efficient Image Retrieval

Deep Aggregation of Regional Convolutional Activations for Content Based Image Retrieval

Multi-level Image Representation for Large-Scale Image-Based Instance Retrieval.

Weighted Pooling for Image Recognition of Deep Convolutional Neural Networks

Deep Learning Using Isotroping, Laplacing, Eigenvalues Interpolative Binding, and Convolved Determinants with Normed Mapping for Large-Scale Image Retrieval

Learning Token-Based Representation for Image Retrieval

Image retrieval by aggregating deep orientation structure features