Abstract:Feature pooling is a key component in modern visual classification system. However, the conventional two prevailing pooling techniques, namely average and max poolings, are not theoretically optimal, due to the unrecoverable loss of the spatial information during the statistical summarization and the underlying over-simplified assumption about the feature distribution. Addressing these issues, this paper proposes to generalize previous pooling methods toward a weighted ℓp-norm spatial pooling function tailored for class-specific feature spatial distribution. Optimizing such a pooling function toward discriminative class separability that is subject to a spatial smoothness constraint yields a so-called geometric ℓp-norm pooling (GLP) method. Furthermore, to handle the variation of object scale/position, which would affect not only the learning of discriminative pooling weights but also the applicability of the learned weights, we propose a simple yet effective self-alignment step during both learning and testing to adaptively adjust the pooling weights for individual images. Image segmentation and visual saliency map are utilized to construct a directed pixel adjacency graph. The discriminative pooling weights are diffused using random walk on the constructed graph and therefore the discriminative pooling weights are propagated onto the salient and foreground region. This leads to a robust version of GLP (RGLP) which can cope with the misalignment of object position and scale in images. Comprehensive experiments validate the effectiveness of the proposed GLP feature pooling framework. The proposed random walk based self-alignment step can effectively alleviate the image misalignment issue and further boost classification accuracy. State-of-the-art image classification and action recognition performances are attained on several benchmarks.

Fine-grained Visual Classification Via Multilayer Bilinear Pooling with Object Localization

Fine-Grained Visual Categorization With Fine-Tuned Segmentation

Grouping Bilinear Pooling for Fine-Grained Image Classification

Adaptive Bilinear Pooling for Fine-grained Representation Learning.

Grassmann Pooling As Compact Homogeneous Bilinear Pooling For Fine-Grained Visual Classification

Weakly Supervised Fine-Grained Image Recognition Based on Multi-Channel Attention and Object Localization

Fine-grained Image Analysis Via Progressive Feature Learning

On Combining Multiple Instance Linear SVM and Bag Splitting for High Performance Visual Object Localization

Weakly Supervised Bilinear Attention Network for Fine-Grained Visual Classification

Cross-convolutional-layer Pooling for Generic Visual Recognition.

Robust Geometric ℓp-Norm Feature Pooling for Image Classification and Action Recognition

Relevance Pooling Guidance and Class-Balanced Feature Enhancement for Fine-Grained Oriented Object Detection in Remote Sensing Images

Fine-Grained Image Classification with Object-Part Model

Improved Bilinear CNN Model for Remote Sensing Scene Classification

Object-Part Attention Driven Discriminative Localization for Fine-grained Image Classification.

Coarse-to-Fine Description for Fine-Grained Visual Categorization

Multimodal Local Perception Bilinear Pooling for Visual Question Answering

Friend or Foe: Fine-Grained Categorization With Weak Supervision.

ELoPE: Fine-Grained Visual Classification with Efficient Localization, Pooling and Embedding

A Fine-Grained Image Classification and Detection Method Based on Convolutional Neural Network Fused with Attention Mechanism

Simple Techniques Make Sense: Feature Pooling and Normalization for Image Classification