Abstract:Attention mechanisms have gradually become necessary to enhance the representational power of convolutional neural networks (CNNs). Despite recent progress in attention mechanism research, some open problems still exist. Most existing methods ignore modeling multi-scale feature representations, structural information, and long-range channel dependencies, which are essential for delivering more discriminative attention maps. This study proposes a novel, low-overhead, high-performance attention mechanism with strong generalization ability for various networks and datasets. This mechanism is called Multi-Scale Spatial Pyramid Attention (MSPA) and can be used to solve the limitations of other attention methods. For the critical components of MSPA, we not only develop the Hierarchical-Phantom Convolution (HPC) module, which can extract multi-scale spatial information at a more granular level utilizing hierarchical residual-like connections, but also design the Spatial Pyramid Recalibration (SPR) module, which can integrate structural regularization and structural information in an adaptive combination mechanism, while employing the Softmax operation to build long-range channel dependencies. The proposed MSPA is a powerful tool that can be conveniently embedded into various CNNs as a plug-and-play component. Correspondingly, using MSPA to replace the 3 × 3 convolution in the bottleneck residual blocks of ResNets, we created a series of simple and efficient backbones named MSPANet, which naturally inherit the advantages of MSPA. Without bells and whistles, our method substantially outperforms other state-of-the-art counterparts in all evaluation metrics based on extensive experimental results from CIFAR-100 and ImageNet-1K image recognition. When applying MSPA to ResNet-50, our model achieves top-1 classification accuracy of 81.74% and 78.40% on the CIFAR-100 and ImageNet-1K benchmarks, exceeding the corresponding baselines by 3.95% and 2.27%, respectively. We also obtained promising performance improvements of 1.15% and 0.91% compared to the competitive EPSANet-50. In addition, empirical research results in autonomous driving engineering applications also demonstrate that our method can significantly improve the accuracy and real-time performance of image recognition with cheaper overhead. Our code is publicly available at https://github.com/ndsclark/MSPANet .

MPANet: Multi-scale Pyramid Attention Network for Collaborative Modeling Spatio-Temporal Patterns of Default Mode Network.

Modeling Default Mode Network Patterns via a Universal Spatio-Temporal Brain Attention Skip Network

Multi-head attention-based masked sequence model for mapping functional brain networks

Modeling spatio-temporal patterns of holistic functional brain networks via multi-head guided attention graph neural networks (Multi-Head GAGNNs)

A Guided Attention 4D Convolutional Neural Network for Modeling Spatio-Temporal Patterns of Functional Brain Networks.

Multi-scale spatial pyramid attention mechanism for image recognition: An effective approach

Characterizing Functional Brain Networks Via Spatio-Temporal Attention 4D Convolutional Neural Networks (Sta-4Dcnns)

Multi-Scale Spatial-Temporal Attention Networks for Functional Connectome Classification

DSAM: A Deep Learning Framework for Analyzing Temporal and Spatial Dynamics in Brain Networks

A Multi-Scale Progressive Collaborative Attention Network for Remote Sensing Fusion Classification.

Modeling 4D fMRI Data via Spatio-Temporal Convolutional Neural Networks (ST-CNN)

MMASleepNet: A multimodal attention network based on electrophysiological signals for automatic sleep staging

Spatial Pyramid Attention for Deep Convolutional Neural Networks.

Multivariate Empirical Mode Decomposition Based Sub-Frequency Bands Analysis of the Default Mode Network: a Resting-State Fmri Data Study

Hierarchical Spatio-Temporal State-Space Modeling for fMRI Analysis

3d Convolutional Long-Short Term Memory Network For Spatiotemporal Modeling Of Fmri Data

MHNet: Multi-view High-order Network for Diagnosing Neurodevelopmental Disorders Using Resting-state fMRI

mTBI-DSANet: A Deep Self-Attention model for diagnosing mild traumatic brain injury using multi-level functional connectivity networks

Multi-Task Collaborative Pre-Training and Adaptive Token Selection: A Unified Framework for Brain Representation Learning

Spatio-temporal Modeling of Connectome-Scale Brain Network Interactions Via Time-Evolving Graphs

MAPANet: A Multi-Scale Attention-Guided Progressive Aggregation Network for Multi-Contrast MRI Super-Resolution