Abstract:In this article, we propose a Dual Relation-aware Attention Network (DRANet) to handle the task of scene segmentation. How to efficiently exploit context is essential for pixel-level recognition. To address the issue, we adaptively capture contextual information based on the relation-aware attention mechanism. Especially, we append two types of attention modules on the top of the dilated fully convolutional network (FCN), which model the contextual dependencies in spatial and channel dimensions, respectively. In the attention modules, we adopt a self-attention mechanism to model semantic associations between any two pixels or channels. Each pixel or channel can adaptively aggregate context from all pixels or channels according to their correlations. To reduce the high cost of computation and memory caused by the abovementioned pairwise association computation, we further design two types of compact attention modules. In the compact attention modules, each pixel or channel is built into association only with a few numbers of gathering centers and obtains corresponding context aggregation over these gathering centers. Meanwhile, we add a cross-level gating decoder to selectively enhance spatial details that boost the performance of the network. We conduct extensive experiments to validate the effectiveness of our network and achieve new state-of-the-art segmentation performance on four challenging scene segmentation data sets, i.e., Cityscapes, ADE20K, PASCAL Context, and COCO Stuff data sets. In particular, a Mean IoU score of 82.9% on the Cityscapes test set is achieved without using extra coarse annotated data.

End-to-End Instance Segmentation with Recurrent Attention

EHANet: Efficient Hybrid Attention Network Towards Real-time Semantic Segmentation

Scene Classification with Recurrent Attention of VHR Remote Sensing Images

AttentionRNN: A Structured Spatial Attention Mechanism

Supervised Edge Attention Network for Accurate Image Instance Segmentation

Group-wise Deep Object Co-Segmentation with Co-Attention Recurrent Neural Network

Semantic Attention and Scale Complementary Network for Instance Segmentation in Remote Sensing Images

Semantic Segmentation With Attention Mechanism for Remote Sensing Images

Realtime Global Attention Network for Semantic Segmentation

Embedded Attention Network for Semantic Segmentation

Progressive Scene Segmentation Based on Self-Attention Mechanism.

An Empirical Study of Attention Networks for Semantic Segmentation

Look Closer to See Better: Recurrent Attention Convolutional Neural Network for Fine-Grained Image Recognition

Consistent Video Instance Segmentation with Inter-Frame Recurrent Attention

Scene Segmentation With Dual Relation-Aware Attention Network

Where to Look: A Unified Attention Model for Visual Recognition with Reinforcement Learning

End-to-end Semantic-Aware Object Retrieval Based on Region-Wise Attention

SegNeXt: Rethinking Convolutional Attention Design for Semantic Segmentation

Salient instance segmentation via subitizing and clustering