Abstract:In this paper, the authors propose a refined global attention module (RGAM) to address known shortcomings of existing attention mechanisms. To validate the effectiveness and generality of the proposed method, the authors conducted extensive experiments on four distinct medical image segmentation datasets. The results demonstrate the effectiveness of RGAM in achieving state‐of‐the‐art performance compared to existing methods. Attention mechanisms are popular techniques in computer vision that mimic the ability of the human visual system to analyse complex scenes, enhancing the performance of convolutional neural networks (CNN). In this paper, the authors propose a refined global attention module (RGAM) to address known shortcomings of existing attention mechanisms: (1) Traditional channel attention mechanisms are not refined enough when concentrating features, which may lead to overlooking important information. (2) The 1‐dimensional attention map generated by traditional spatial attention mechanisms make it difficult to accurately summarise the weights of all channels in the original feature map at the same position. The RGAM is composed of two parts: refined channel attention and refined spatial attention. In the channel attention part, the authors used multiple weight‐shared dilated convolutions with varying dilation rates to perceive features with different receptive fields at the feature compression stage. The authors also combined dilated convolutions with depth‐wise convolution to reduce the number of parameters. In the spatial attention part, the authors grouped the feature maps and calculated the attention for each group independently, allowing for a more accurate assessment of each spatial position's importance. Specifically, the authors calculated the attention weights separately for the width and height directions, similar to SENet, to obtain more refined attention weights. To validate the effectiveness and generality of the proposed method, the authors conducted extensive experiments on four distinct medical image segmentation datasets. The results demonstrate the effectiveness of RGAM in achieving state‐of‐the‐art performance compared to existing methods.

A Generalized Attention Mechanism to Enhance the Accuracy Performance of Neural Networks

HAM: Hybrid Attention Module in Deep Convolutional Neural Networks for Image Classification

Understanding More about Human and Machine Attention in Deep Neural Networks

Adaptive Attention Module for Image Recognition Systems in Autonomous Driving

A Generic Shared Attention Mechanism for Various Backbone Neural Networks

Human Vs Machine Attention in Neural Networks: A Comparative Study.

The Costs and Benefits of Goal-Directed Attention in Deep Convolutional Neural Networks

Efficient Attention Network: Accelerate Attention by Searching Where to Plug

GAttANet: Global attention agreement for convolutional neural networks

A Regularized Framework for Sparse and Structured Neural Attention

RGAM: A refined global attention mechanism for medical image segmentation

Enhancing spiking neural networks with hybrid top-down attention

An Attention Module for Convolutional Neural Networks

A General Survey on Attention Mechanisms in Deep Learning

Attention mechanism models for precision medicine

Attention mechanism in neural networks: where it comes and where it goes

Global Attention Mechanism: Retain Information to Enhance Channel-Spatial Interactions

How biological attention mechanisms improve task performance in a large-scale visual system model

Multiscale Hybrid Convolutional Deep Neural Networks with Channel Attention

A Neural Network Model of Visual Attention Integrating Biased Competition and Reinforcement Learning