Abstract:Attention mechanisms have gradually become necessary to enhance the representational power of convolutional neural networks (CNNs). Despite recent progress in attention mechanism research, some open problems still exist. Most existing methods ignore modeling multi-scale feature representations, structural information, and long-range channel dependencies, which are essential for delivering more discriminative attention maps. This study proposes a novel, low-overhead, high-performance attention mechanism with strong generalization ability for various networks and datasets. This mechanism is called Multi-Scale Spatial Pyramid Attention (MSPA) and can be used to solve the limitations of other attention methods. For the critical components of MSPA, we not only develop the Hierarchical-Phantom Convolution (HPC) module, which can extract multi-scale spatial information at a more granular level utilizing hierarchical residual-like connections, but also design the Spatial Pyramid Recalibration (SPR) module, which can integrate structural regularization and structural information in an adaptive combination mechanism, while employing the Softmax operation to build long-range channel dependencies. The proposed MSPA is a powerful tool that can be conveniently embedded into various CNNs as a plug-and-play component. Correspondingly, using MSPA to replace the 3 × 3 convolution in the bottleneck residual blocks of ResNets, we created a series of simple and efficient backbones named MSPANet, which naturally inherit the advantages of MSPA. Without bells and whistles, our method substantially outperforms other state-of-the-art counterparts in all evaluation metrics based on extensive experimental results from CIFAR-100 and ImageNet-1K image recognition. When applying MSPA to ResNet-50, our model achieves top-1 classification accuracy of 81.74% and 78.40% on the CIFAR-100 and ImageNet-1K benchmarks, exceeding the corresponding baselines by 3.95% and 2.27%, respectively. We also obtained promising performance improvements of 1.15% and 0.91% compared to the competitive EPSANet-50. In addition, empirical research results in autonomous driving engineering applications also demonstrate that our method can significantly improve the accuracy and real-time performance of image recognition with cheaper overhead. Our code is publicly available at https://github.com/ndsclark/MSPANet .

Spanet: Spatial Pyramid Attention Network for Enhanced Image Recognition

Spatial Pyramid Attention for Deep Convolutional Neural Networks.

Multi-scale spatial pyramid attention mechanism for image recognition: An effective approach

Salient Positions based Attention Network for Image Classification

Deep Superpixel Convolutional Network for Image Recognition

Stacked Pyramid Attention Network for Object Detection

SPANet: Successive Pooling Attention Network for Semantic Segmentation of Remote Sensing Images

Balanced Spatial Feature Distillation and Pyramid Attention Network for Lightweight Image Super-resolution

PCSA: Enhancing CNN Performance With Pyramid Channel and Spatial Attention

Memory-Efficient Sparse Pyramid Attention Networks for Whole Slide Image Analysis

Learning Spatial Pyramid Attentive Pooling in Image Synthesis and Image-to-Image Translation

A Spatial–Channel–Temporal-Fused Attention for Spiking Neural Networks

Scale Channel Attention Network for Image Segmentation

Nonlocal Spatial Attention Module for Image Classification

A2SPPNet: Attentive Atrous Spatial Pyramid Pooling Network for Salient Object Detection.

A Symmetric Efficient Spatial and Channel Attention (ESCA) Module Based on Convolutional Neural Networks

Spatial--spectral FFPNet: Attention-Based Pyramid Network for Segmentation and Classification of Remote Sensing Images

Spatial Group-Wise Enhance: Enhancing Semantic Feature Learning in CNN

Attend and Guide (AG-Net): A Keypoints-driven Attention-based Deep Network for Image Recognition

A Deep Pyramid Attention Network for Single Image Super-resolution

MAM: A Multipath Attention Mechanism for Image Recognition