Abstract:Attention mechanisms have demonstrated great potential in improving the performance of deep convolutional neural networks (CNNs). However, many existing methods dedicate to developing channel or spatial attention modules for CNNs with lots of parameters, and complex attention modules inevitably affect the performance of CNNs. During our experiments of embedding Convolutional Block Attention Module (CBAM) in light-weight model YOLOv5s, CBAM does influence the speed and increase model complexity while reduce the average precision, but Squeeze-and-Excitation (SE) has a positive impact in the model as part of CBAM. To replace the spatial attention module in CBAM and offer a suitable scheme of channel and spatial attention modules, this paper proposes one Spatio-temporal Sharpening Attention Mechanism (SSAM), which sequentially infers intermediate maps along channel attention module and Sharpening Spatial Attention (SSA) module. By introducing sharpening filter in spatial attention module, we propose SSA module with low complexity. We try to find a scheme to combine our SSA module with SE module or Efficient Channel Attention (ECA) module and show best improvement in models such as YOLOv5s and YOLOv3-tiny. Therefore, we perform various replacement experiments and offer one best scheme that is to embed channel attention modules in backbone and neck of the model and integrate SSAM into YOLO head. We verify the positive effect of our SSAM on two general object detection datasets VOC2012 and MS COCO2017. One for obtaining a suitable scheme and the other for proving the versatility of our method in complex scenes. Experimental results on the two datasets show obvious promotion in terms of average precision and detection performance, which demonstrates the usefulness of our SSAM in light-weight YOLO models. Furthermore, visualization results also show the advantage of enhancing positioning ability with our SSAM.

LMA: Lightweight Mixed-Domain Attention for Efficient Network Design

LAM: Lightweight Attention Module.

HAM: Hybrid Attention Module in Deep Convolutional Neural Networks for Image Classification

Lightweight Attention Module for Deep Learning on Classification and Segmentation of 3-D Point Clouds

Adaptive Attention Module for Image Recognition Systems in Autonomous Driving

Multiscale Hybrid Convolutional Deep Neural Networks with Channel Attention

MCA: Multidimensional Collaborative Attention in Deep Convolutional Neural Networks for Image Recognition

ELA: Efficient Local Attention for Deep Convolutional Neural Networks

Efficient Multi-Scale Attention Module with Cross-Spatial Learning

ECA-Net: Efficient Channel Attention for Deep Convolutional Neural Networks

MS3DAAM: Multi-scale 3-D Analytic Attention Module for Convolutional Neural Networks

Nonlocal Spatial Attention Module for Image Classification

Lightweight DenseNet Model with A Hybrid Attention Mechanism and A Cross-Module Channel Reuse Strategy for Edge Computing

Improved deep CNNs based on Nonlinear Hybrid Attention Module for image classification

One Spatio-Temporal Sharpening Attention Mechanism for Light-Weight YOLO Models Based on Sharpening Spatial Attention

Lightweight Attention Network for Very High-Resolution Image Semantic Segmentation

MAM: A Multipath Attention Mechanism for Image Recognition

A Lightweight Attention-Based Network for Micro-Expression Recognition

LDA-Mono: A Lightweight Dual Aggregation Network for Self-Supervised Monocular Depth Estimation

Multibranch Attention Mechanism Based on Channel and Spatial Attention Fusion