Abstract:Attention mechanisms have gradually become necessary to enhance the representational power of convolutional neural networks (CNNs). Despite recent progress in attention mechanism research, some open problems still exist. Most existing methods ignore modeling multi-scale feature representations, structural information, and long-range channel dependencies, which are essential for delivering more discriminative attention maps. This study proposes a novel, low-overhead, high-performance attention mechanism with strong generalization ability for various networks and datasets. This mechanism is called Multi-Scale Spatial Pyramid Attention (MSPA) and can be used to solve the limitations of other attention methods. For the critical components of MSPA, we not only develop the Hierarchical-Phantom Convolution (HPC) module, which can extract multi-scale spatial information at a more granular level utilizing hierarchical residual-like connections, but also design the Spatial Pyramid Recalibration (SPR) module, which can integrate structural regularization and structural information in an adaptive combination mechanism, while employing the Softmax operation to build long-range channel dependencies. The proposed MSPA is a powerful tool that can be conveniently embedded into various CNNs as a plug-and-play component. Correspondingly, using MSPA to replace the 3 × 3 convolution in the bottleneck residual blocks of ResNets, we created a series of simple and efficient backbones named MSPANet, which naturally inherit the advantages of MSPA. Without bells and whistles, our method substantially outperforms other state-of-the-art counterparts in all evaluation metrics based on extensive experimental results from CIFAR-100 and ImageNet-1K image recognition. When applying MSPA to ResNet-50, our model achieves top-1 classification accuracy of 81.74% and 78.40% on the CIFAR-100 and ImageNet-1K benchmarks, exceeding the corresponding baselines by 3.95% and 2.27%, respectively. We also obtained promising performance improvements of 1.15% and 0.91% compared to the competitive EPSANet-50. In addition, empirical research results in autonomous driving engineering applications also demonstrate that our method can significantly improve the accuracy and real-time performance of image recognition with cheaper overhead. Our code is publicly available at https://github.com/ndsclark/MSPANet .

Attention Pyramid Module for Scene Recognition

Dynamic Parallel Pyramid Networks for Scene Recognition

Multi-scale spatial pyramid attention mechanism for image recognition: An effective approach

Scene Recognition Based on an Improved DenseNet with Attention Mechanism

mAPm: multi-scale Attention Pyramid module for Enhanced scale-variation in RLD detection

PAM: Pyramid Attention Mechanism Based on Contextual Reasoning

Deep Neural Network with Attention Model for Scene Text Recognition.

MLA-Net: Feature Pyramid Network with Multi-Level Local Attention for Object Detection

Remote Sensing Image Scene Classification Based on an Enhanced Attention Module

PCANet: Pyramid convolutional attention network for semantic segmentation

EPRNet: Efficient Pyramid Representation Network for Real-Time Street Scene Segmentation

An End-to-End Trainable Multi-Column CNN for Scene Recognition in Extremely Changing Environment

An Attention Model Based on Spatial Transformers for Scene Recognition

Scene Recognition with Objectness, Attribute and Category Learning

Pyramid Feature Attention Network for Monocular Depth Prediction

DeepScene: Scene classification via convolutional neural network with spatial pyramid pooling

Attention-guided Context Feature Pyramid Network for Object Detection

Remote Sensing Image Instance Segmentation Based on Attention Balanced Feature Pyramid

MultiScene: A Large-scale Dataset and Benchmark for Multi-scene Recognition in Single Aerial Images

Multi-branch Input Structure for Pyramid Scene Parsing Network