Abstract:There are some traditional pooling methods in convolutional neural network, such as max-pooling, average pooling, stochastic pooling and so on, which determine the results of pooling based on the distribution of each activation in the pooling region. Zeiler and Fergus (Stochastic-pooling for regularization of deep convolutional neural networks, 2013 ) However, it is difficult for the feature mapping process to select a perfect activation representative of the pooling region, and can lead to the phenomenon of over-fitting. In this paper, the following theoretical basis comes out information theory (Shannon in Bell Syst. Tech. J. 27:379–423, 1948 ). First, we quantify the information entropy of each pooling region, and then propose an efficient pooling method by comparing the mutual information between activations and the pooling region which they are located in. Moreover, we assign different weights to different activations based on mutual information, and named it weighted-pooling. The main features of the weighted-pooling method are as follows: (1) The information quantity of the pooling region is quantified by information theory for the first time. (2) Also, each activation’s contribution was quantified for the first time and these contributions eliminate the uncertainty of the pooling region which it is located in. (3) For choosing a representative in this pooling region, the weight of each activation obviously superiors to the value of activation. In the experimental part, we respectively use MNIST and CIFAR-10 (Krizhevsky in Learning multiple layers of featurs from tiny images, University of Toronto, 2009 ; LeCun in The MNIST database, 2012 ) data sets to compare different pooling methods. The results show that the weighted-pooling method has higher recognition accuracy than other pooling methods and reaches a new state-of-the-art.

Analysing the Effects of Pooling Combinations on Invariance to Position and Deformation in Convolutional Neural Networks

Pooling is neither necessary nor sufficient for appropriate deformation stability in CNNs

OVPT: Optimal Viewset Pooling Transformer for 3D Object Recognition.

Mixed Pooling For Convolutional Neural Networks

Wasserstein Pooling for Image Classification

Enhanced Pooling Method for Convolutional Neural Networks Based on Optimal Search Theory

On the Shift Invariance of Max Pooling Feature Maps in Convolutional Neural Networks

PoolNet plus : Exploring the Potential of Pooling for Salient Object Detection

Progress and limitations of deep networks to recognize objects in unusual poses

Inductive Bias of Deep Convolutional Networks through Pooling Geometry

Weighted Pooling for Image Recognition of Deep Convolutional Neural Networks

Adaptive Salience Preserving Pooling for Deep Convolutional Neural Networks

Cross-convolutional-layer Pooling for Generic Visual Recognition.

Deep CNNs Meet Global Covariance Pooling: Better Representation and Generalization

Quantifying Translation-Invariance in Convolutional Neural Networks

Grassmann Pooling As Compact Homogeneous Bilinear Pooling For Fine-Grained Visual Classification

Generalizing Pooling Functions in Convolutional Neural Networks: Mixed, Gated, and Tree

Exploring Novel Pooling Strategies for Edge Preserved Feature Maps in Convolutional Neural Networks

General Deformable RoI Pooling and Semi-Decoupled Head for Object Detection

Gradient Corner Pooling for Keypoint-Based Object Detection

Equivariant vs. Invariant Layers: A Comparison of Backbone and Pooling for Point Cloud Classification