Abstract:Spatial downsampling layers are favored in convolutional neural networks (CNNs) to downscale feature maps for larger receptive fields and less memory consumption. However, for visual recognition tasks, these layers might lose discriminative details due to improper pooling strategies. In this paper, we present a unified framework (LAN) over the common downsampling layers (e.g., average pooling, max pooling, and strided convolution) from a view of local aggregation based on importance. In this LAN framework, we analyze the issues of these widely-used pooling layers and figure out the criteria of designing an effective downsampling layer. Based on this analysis, we propose a simple, general, and effective pooling operation based on local importance modeling, termed as Local Importance-based Pooling (LIP). LIP is able to enhance discriminative features during the downsampling procedure by learning adaptive importance weights based on inputs. To further modulate different pooling windows for more effective pooling, we present the improved version of LIP, termed LIP++, by introducing an explicit margin term and efficient logit modules. Our LIP++ can yield consistent accuracy improvement over the original LIP yet with a smaller computational cost. Extensive experiments show that our presented LIP method consistently yields notable gains with different CNN architectures on the image classification task. In the challenging MS COCO dataset, detectors with our LIP-ResNets as backbones obtain a consistent performance improvement over the vanilla ResNets on both bounding box detection and instance segmentation. Finally, we also verify the effectiveness of LIP on the tasks of pose estimation and semantic segmentation, demonstrating its generalization to the dense prediction task.

Learning to Pool High-Level Features for Face Representation.

Learning Individual-Specific Dictionaries With Fused Multiple Features For Face Recognition

Face Image Classification by Pooling Raw Features

Combination of Local Multiple Patterns and Exponential Discriminant Analysis for facial recognition

Face Identification with Second-Order Pooling

Combining Local and Global: Rich and Robust Feature Pooling for Visual Recognition.

R2FP: Rich and Robust Feature Pooling for Mining Visual Data

Face Feature Extraction and Recognition Via Local Binary Pattern and Two-Dimensional Locality Preserving Projection

The face image super-resolution algorithm based on combined representation learning

Bag of Shape Features with a Learned Pooling Function for Shape Recognition

Toward High Quality Facial Representation Learning

Adaptive random down-sampling data augmentation and area attention pooling for low resolution face recognition

LIP: Local Importance-Based Pooling

Face Anti-Spoofing Using Spatial Pyramid Pooling

Enhanced mechanisms of pooling and channel attention for deep learning feature maps

Joint Feature Learning for Face Recognition

Face recognition using part-based dense sampling local features

Multi-scale Patch Based Representation Feature Learning for Low-Resolution Face Recognition

Enhanced Local Gradient Order Features and Discriminant Analysis for Face Recognition

Adaptive Bilinear Pooling for Fine-grained Representation Learning.

A Fully Trainable Network with RNN-based Pooling