Abstract:Efficient custom pooling techniques that can aggressively trim the dimensions of a feature map and thereby reduce inference compute and memory footprint for resource-constrained computer vision applications have recently gained significant traction. However, prior pooling works extract only the local context of the activation maps, limiting their effectiveness. In contrast, we propose a novel non-local self-attentive pooling method that can be used as a drop-in replacement to the standard pooling layers, such as max/average pooling or strided convolution. The proposed self-attention module uses patch embedding, multi-head self-attention, and spatial-channel restoration, followed by sigmoid activation and exponential soft-max. This self-attention mechanism efficiently aggregates dependencies between non-local activation patches during down-sampling. Extensive experiments on standard object classification and detection tasks with various convolutional neural network (CNN) architectures demonstrate the superiority of our proposed mechanism over the state-of-the-art (SOTA) pooling techniques. In particular, we surpass the test accuracy of existing pooling techniques on different variants of MobileNet-V2 on ImageNet by an average of 1.2%. With the aggressive down-sampling of the activation maps in the initial layers (providing up to 22x reduction in memory consumption), our approach achieves 1.43% higher test accuracy compared to SOTA techniques with iso-memory footprints. This enables the deployment of our models in memory-constrained devices, such as micro-controllers (without losing significant accuracy), because the initial activation maps consume a significant amount of on-chip memory for high-resolution images required for complex vision tasks. Our proposed pooling method also leverages the idea of channel pruning to further reduce memory footprints.

Matched Wavelet Pooling for Efficient Light-Weight CNN Network Architectures with Application to Image Classification

Less Is More: Matched Wavelet Pooling-Based Light-Weight CNNs With Application to Image Classification

Evaluating the Potential of Wavelet Pooling on Improving the Data Efficiency of Light-Weight CNNs

Wasserstein Pooling for Image Classification

WaveCNet: Wavelet Integrated CNNs to Suppress Aliasing Effect for Noise-Robust Image Classification

Wavelet Integrated CNNs for Noise-Robust Image Classification

Lightweight MIMO-WNet for Single Image Deblurring

Texture and Materials Image Classification Based on Wavelet Pooling Layer in CNN

Exploring Novel Pooling Strategies for Edge Preserved Feature Maps in Convolutional Neural Networks

Research on improved wavelet convolutional wavelet neural networks

Multi-level Wavelet Convolutional Neural Networks

Wavelet-Based Dual-Task Network

Efficient Deep Image Denoising Via Class Specific Convolution

Self-Attentive Pooling for Efficient Deep Learning

Design of Image Matched Non-Separable Wavelet using Convolutional Neural Network

generalized approach of convolutional and pooling layer in image processing using wavelet CNN

Cascade wavelet transform based convolutional neural networks with application to image classification

Efficient channel expansion and pyramid depthwise-pointwise-depthwise neural networks

Wavelet Convolutional Neural Networks

Progressive Meta-Pooling Learning for Lightweight Image Classification Model

WaveNets: Wavelet Channel Attention Networks