Abstract:Attention mechanisms have gradually become necessary to enhance the representational power of convolutional neural networks (CNNs). Despite recent progress in attention mechanism research, some open problems still exist. Most existing methods ignore modeling multi-scale feature representations, structural information, and long-range channel dependencies, which are essential for delivering more discriminative attention maps. This study proposes a novel, low-overhead, high-performance attention mechanism with strong generalization ability for various networks and datasets. This mechanism is called Multi-Scale Spatial Pyramid Attention (MSPA) and can be used to solve the limitations of other attention methods. For the critical components of MSPA, we not only develop the Hierarchical-Phantom Convolution (HPC) module, which can extract multi-scale spatial information at a more granular level utilizing hierarchical residual-like connections, but also design the Spatial Pyramid Recalibration (SPR) module, which can integrate structural regularization and structural information in an adaptive combination mechanism, while employing the Softmax operation to build long-range channel dependencies. The proposed MSPA is a powerful tool that can be conveniently embedded into various CNNs as a plug-and-play component. Correspondingly, using MSPA to replace the 3 × 3 convolution in the bottleneck residual blocks of ResNets, we created a series of simple and efficient backbones named MSPANet, which naturally inherit the advantages of MSPA. Without bells and whistles, our method substantially outperforms other state-of-the-art counterparts in all evaluation metrics based on extensive experimental results from CIFAR-100 and ImageNet-1K image recognition. When applying MSPA to ResNet-50, our model achieves top-1 classification accuracy of 81.74% and 78.40% on the CIFAR-100 and ImageNet-1K benchmarks, exceeding the corresponding baselines by 3.95% and 2.27%, respectively. We also obtained promising performance improvements of 1.15% and 0.91% compared to the competitive EPSANet-50. In addition, empirical research results in autonomous driving engineering applications also demonstrate that our method can significantly improve the accuracy and real-time performance of image recognition with cheaper overhead. Our code is publicly available at https://github.com/ndsclark/MSPANet .

MCA: Multidimensional Collaborative Attention in Deep Convolutional Neural Networks for Image Recognition

A Multi-scale Contextual Attention Mechanism for Convolutional Neural Networks

Cascaded Multi-Scale Attention for Enhanced Multi-Scale Feature Extraction and Interaction with Low-Resolution Images

MCA: Moment Channel Attention Networks

MS3DAAM: Multi-scale 3-D Analytic Attention Module for Convolutional Neural Networks

HAM: Hybrid Attention Module in Deep Convolutional Neural Networks for Image Classification

Multiscale Hybrid Convolutional Deep Neural Networks with Channel Attention

CAT: Learning to Collaborate Channel and Spatial Attention from Multi-Information Fusion

MAM: A Multipath Attention Mechanism for Image Recognition

Multi-scale spatial pyramid attention mechanism for image recognition: An effective approach

MCANet: Multiscale Cross-Modality Attention Network for Multispectral Pedestrian Detection

Multi-Attention Multi-Class Constraint for Fine-grained Image Recognition

MCANet: Medical Image Segmentation with Multi-Scale Cross-Axis Attention

Multi-Scale Cross-Modal Spatial Attention Fusion for Multi-label Image Recognition.

Multibranch Attention Mechanism Based on Channel and Spatial Attention Fusion

LMA: Lightweight Mixed-Domain Attention for Efficient Network Design

Efficient Multi-Scale Attention Module with Cross-Spatial Learning

ECA-Net: Efficient Channel Attention for Deep Convolutional Neural Networks

CMAN: Leaning Global Structure Correlation for Monocular 3D Object Detection

Multi-Scale Spatial and Channel-wise Attention for Improving Object Detection in Remote Sensing Imagery