Abstract:Attention mechanisms have demonstrated great potential in improving the performance of deep convolutional neural networks (CNNs). However, many existing methods dedicate to developing channel or spatial attention modules for CNNs with lots of parameters, and complex attention modules inevitably affect the performance of CNNs. During our experiments of embedding Convolutional Block Attention Module (CBAM) in light-weight model YOLOv5s, CBAM does influence the speed and increase model complexity while reduce the average precision, but Squeeze-and-Excitation (SE) has a positive impact in the model as part of CBAM. To replace the spatial attention module in CBAM and offer a suitable scheme of channel and spatial attention modules, this paper proposes one Spatio-temporal Sharpening Attention Mechanism (SSAM), which sequentially infers intermediate maps along channel attention module and Sharpening Spatial Attention (SSA) module. By introducing sharpening filter in spatial attention module, we propose SSA module with low complexity. We try to find a scheme to combine our SSA module with SE module or Efficient Channel Attention (ECA) module and show best improvement in models such as YOLOv5s and YOLOv3-tiny. Therefore, we perform various replacement experiments and offer one best scheme that is to embed channel attention modules in backbone and neck of the model and integrate SSAM into YOLO head. We verify the positive effect of our SSAM on two general object detection datasets VOC2012 and MS COCO2017. One for obtaining a suitable scheme and the other for proving the versatility of our method in complex scenes. Experimental results on the two datasets show obvious promotion in terms of average precision and detection performance, which demonstrates the usefulness of our SSAM in light-weight YOLO models. Furthermore, visualization results also show the advantage of enhancing positioning ability with our SSAM.

Visual Attention Model with Cross-Layer Saliency Optimization

Visual Saliency Detection Based on Topographic Independent Component Analysis

A visual attention model for news video

CSA-Net: Deep Cross-Complementary Self Attention and Modality-Specific Preservation for Saliency Detection

MSGC: A New Bottom-Up Model for Salient Object Detection

Video Saliency Detection via Dynamic Consistent Spatio-Temporal Attention Modelling.

Visual Attention Model Based on Fussion of Spatiotemporal Features

Semantic and Contrast-Aware Saliency

Deep supervised visual saliency model addressing low-level features

Opponent and Feedback: Visual Attention Captured

An Integrated Model for Effective Saliency Prediction

A Saliency Model Based on Wavelet Transform and Visual Attention

Hierarchical identification of visually salient image regions

What Do Deep Saliency Models Learn about Visual Attention?

Saliency Detection Via Absorbing Markov Chain with Learnt Transition Probability.

A multi-stage area saliency detection model

A Novel Target-Objected Visual Saliency Detection Model in Optical Satellite Images.

One Spatio-Temporal Sharpening Attention Mechanism for Light-Weight YOLO Models Based on Sharpening Spatial Attention

A Novel Method to Study Bottom-up Visual Saliency and its Neural Mechanism

A VISUAL ATTENTION MODEL FOR NATURAL SCENES BASED ON DYNAMIC FEATURE COMBINATION

Research on Visual Saliency Model Based on CovSal Algorithm and Histogram Contrast