Abstract:Recently, it is generally acknowledged that the receptive field size of visual cortical neurons are regulated by the stimulus in the neuroscience community. Thus, once the global receptive field is obtained, the network performance can be greatly improved. Unfortunately, the larger receptive field method has been rarely considered in constructing CNNs. Recent studies on network design have demonstrated that the key to improving model performance is channel attention. However, they usually neglect the location information. Hence, it is difficult to capture the long-term dependency of location information. In particular, the location information is important for generating spatially selective attention blocks. Therefore, in this paper, we propose a novel attention neural network termed "GEA" by embedding global context information into channel attention. Firstly, instead of channel attention, which is transformed from feature tensor to single feature vector via 2D pooling, our method decomposes the channel attention into two 1D feature encoding processes that aggregate features along two spatial directions. In particular, the long-term dependency can be captured by using one spatial direction as well as preserving accurate location information in the opposite direction. Then, we concatenate the results of the two directions, while carry out batch normalization and use relu activation function. In addition, the cross channel soft attention is used to adaptively select different spatial scales of the information. Finally, our method is simple and can be flexibly inserted into classic networks, such as ResNet and EfficientNet, with limited computational overhead. A large number of experiments show that our method is not only conducive to the classification of COCO and ImageNet, but also demonstrates promising performance for 3D features, such as Magnetic Resonance Imaging(MRI).

GPCA: A Probabilistic Framework for Gaussian Process Embedded Channel Attention

Gaussian Context Transformer.

Re-parameterized Global Channel Interaction Attention Module for Convolutional Neural Networks

Adaptive Channel Encoding for Point Cloud Analysis

Improved Channel Attention Methods Via Hierarchical Pooling and Reducing Information Loss

GEA-net - Global Embedded Attention Neural Network for Image Classification.

CAT: Learning to Collaborate Channel and Spatial Attention from Multi-Information Fusion

Attention in Attention: Modeling Context Correlation for Efficient Video Classification

Adaptive gaze estimation based on channel attention mechanism

Attention in Attention: Modeling Context Correlation for Efficient Video Classification

Partial Channel Pooling Attention Beats Convolutional Attention.

Probabilistic Graph Attention Network with Conditional Kernels for Pixel-Wise Prediction

ECA-Net: Efficient Channel Attention for Deep Convolutional Neural Networks

FcaNet: Frequency Channel Attention Networks

CAAGP: Rethinking Channel Attention with Adaptive Global Pooling for Liver Tumor Segmentation

Attention Graph: Learning Effective Visual Features for Large-Scale Image Classification

Scalable Gaussian Process Classification with Additive Noise for Non-Gaussian Likelihoods

A Globally Oriented Weight Integration Attention Module with Different Scales

Gaze estimation via bilinear pooling-based attention networks

PCSA: Enhancing CNN Performance With Pyramid Channel and Spatial Attention