Abstract:Recently, it is generally acknowledged that the receptive field size of visual cortical neurons are regulated by the stimulus in the neuroscience community. Thus, once the global receptive field is obtained, the network performance can be greatly improved. Unfortunately, the larger receptive field method has been rarely considered in constructing CNNs. Recent studies on network design have demonstrated that the key to improving model performance is channel attention. However, they usually neglect the location information. Hence, it is difficult to capture the long-term dependency of location information. In particular, the location information is important for generating spatially selective attention blocks. Therefore, in this paper, we propose a novel attention neural network termed "GEA" by embedding global context information into channel attention. Firstly, instead of channel attention, which is transformed from feature tensor to single feature vector via 2D pooling, our method decomposes the channel attention into two 1D feature encoding processes that aggregate features along two spatial directions. In particular, the long-term dependency can be captured by using one spatial direction as well as preserving accurate location information in the opposite direction. Then, we concatenate the results of the two directions, while carry out batch normalization and use relu activation function. In addition, the cross channel soft attention is used to adaptively select different spatial scales of the information. Finally, our method is simple and can be flexibly inserted into classic networks, such as ResNet and EfficientNet, with limited computational overhead. A large number of experiments show that our method is not only conducive to the classification of COCO and ImageNet, but also demonstrates promising performance for 3D features, such as Magnetic Resonance Imaging(MRI).

GRA-Net: Group Response Attention for Deep Learning

GRA-Net: Global Receptive Attention Network for Surface Defect Detection

GAttANet: Global attention agreement for convolutional neural networks

GRA: Detecting Oriented Objects through Group-wise Rotating and Attention

Research on image classification based on residual group multi-scale enhanced attention network

Rega-Net: Retina Gabor Attention for Deep Convolutional Neural Networks

GEA-net - Global Embedded Attention Neural Network for Image Classification.

Spatial Pyramid Attention for Deep Convolutional Neural Networks.

GRA: Graph Representation Alignment for Semi-Supervised Action Recognition

Global Reference Attention Network for Vehicle Re-Identification.

Attend and Guide (AG-Net): A Keypoints-driven Attention-based Deep Network for Image Recognition

SA-Net: Shuffle Attention for Deep Convolutional Neural Networks

The Costs and Benefits of Goal-Directed Attention in Deep Convolutional Neural Networks

RA-Net: reverse attention for generalizing residual learning

GAU-Net: U-Net Based on Global Attention Mechanism for Brain Tumor Segmentation

ResGANet: Residual Group Attention Network for Medical Image Classification and Segmentation

Second-order Attention Guided Convolutional Activations for Visual Recognition.

ECA-Net: Efficient Channel Attention for Deep Convolutional Neural Networks

An Attention Module for Convolutional Neural Networks

ELA: Efficient Local Attention for Deep Convolutional Neural Networks