Abstract:Channel and spatial attentions have respectively brought significant improvements in extracting feature dependencies and spatial structure relations for various downstream vision tasks. While their combination is more beneficial for leveraging their individual strengths, the synergy between channel and spatial attentions has not been fully explored, lacking in fully harness the synergistic potential of multi-semantic information for feature guidance and mitigation of semantic disparities. Our study attempts to reveal the synergistic relationship between spatial and channel attention at multiple semantic levels, proposing a novel Spatial and Channel Synergistic Attention module (SCSA). Our SCSA consists of two parts: the Shareable Multi-Semantic Spatial Attention (SMSA) and the Progressive Channel-wise Self-Attention (PCSA). SMSA integrates multi-semantic information and utilizes a progressive compression strategy to inject discriminative spatial priors into PCSA's channel self-attention, effectively guiding channel recalibration. Additionally, the robust feature interactions based on the self-attention mechanism in PCSA further mitigate the disparities in multi-semantic information among different sub-features within SMSA. We conduct extensive experiments on seven benchmark datasets, including classification on ImageNet-1K, object detection on MSCOCO 2017, segmentation on ADE20K, and four other complex scene detection datasets. Our results demonstrate that our proposed SCSA not only surpasses the current state-of-the-art attention but also exhibits enhanced generalization capabilities across various task scenarios. The code and models are available at: <a class="link-external link-https" href="https://github.com/HZAI-ZJNU/SCSA" rel="external noopener nofollow">this https URL</a>.

PCSA: Enhancing CNN Performance With Pyramid Channel and Spatial Attention

Parallel-Connected Residual Channel Attention Network for Remote Sensing Image Super-Resolution

EPSANet: an Efficient Pyramid Squeeze Attention Block on Convolutional Neural Network

PCANet: Pyramid convolutional attention network for semantic segmentation

PAM: Pyramid Attention Mechanism Based on Contextual Reasoning

CSA-Net: Channel-wise Spatially Autocorrelated Attention Networks

SCSA: Exploring the Synergistic Effects Between Spatial and Channel Attention

ECA-Net: Efficient Channel Attention for Deep Convolutional Neural Networks

A Symmetric Efficient Spatial and Channel Attention (ESCA) Module Based on Convolutional Neural Networks

CSA: A Channel-Separated Attention Module for Enhancing MRI Reconstruction

Pyramid Feature Attention Network for Monocular Depth Prediction

Pyramid Pixel Context Adaption Network for Medical Image Classification with Supervised Contrastive Learning

A Deep Pyramid Attention Network for Single Image Super-resolution

Improved Channel Attention Methods Via Hierarchical Pooling and Reducing Information Loss

Channel-Attention-Based DenseNet Network for Remote Sensing Image Scene Classification

Salient Object Detection with Pyramid Attention and Salient Edges

HSCA-Net: A Hybrid Spatial-Channel Attention Network in Multi-Scale Feature Pyramid for Document Layout Analysis

CAT: Learning to Collaborate Channel and Spatial Attention from Multi-Information Fusion

Convolutional Neural Network optimization via Channel Reassessment Attention module

Semantic segmentation based on double pyramid network with improved global attention mechanism