Abstract:Global context information is vital in visual understanding problems, especially in pixel-level semantic segmentation. The mainstream methods adopt the self-attention mechanism to model global context information. However, pixels belonging to different classes usually have weak feature correlation. Modeling the global pixel-level correlation matrix indiscriminately is extremely redundant in the self-attention mechanism. In order to solve the above problem, we propose a hierarchical context network to differentially model homogeneous pixels with strong correlations and heterogeneous pixels with weak correlations. Specifically, we first propose a multi-scale guided pre-segmentation module to divide the entire feature map into different classed-based homogeneous regions. Within each homogeneous region, we design the pixel context module to capture pixel-level correlations. Subsequently, different from the self-attention mechanism that still models weak heterogeneous correlations in a dense pixel-level manner, the region context module is proposed to model sparse region-level dependencies using a unified representation of each region. Through aggregating fine-grained pixel context features and coarse-grained region context features, our proposed network can not only hierarchically model global context information but also harvest multi-granularity representations to more robustly identify multi-scale objects. We evaluate our approach on Cityscapes and the ISPRS Vaihingen dataset. Without Bells or Whistles, our approach realizes a mean IoU of 82.8% and overall accuracy of 91.4% on Cityscapes and ISPRS Vaihingen test set, achieving state-of-the-art results.

An Attention Enhanced Graph Convolutional Network for Semantic Segmentation

Dual Graph Convolutional Network for Semantic Segmentation.

Class-related Graph Convolution for Weakly Supervised Semantic Segmentation

Optimizing Spatial Relationships in GCN to Improve the Classification Accuracy of Remote Sensing Images

Edge Enhancement Oriented Graph Convolutional Networks for Point Cloud Segmentation

Class-Wise Dynamic Graph Convolution for Semantic Segmentation

Self-constructing graph neural networks to model long-range pixel dependencies for semantic segmentation of remote sensing images

Edge-Enhanced GCIFFNet: A Multiclass Semantic Segmentation Network Based on Edge Enhancement and Multiscale Attention Mechanism

SGA-Net: Self-Constructing Graph Attention Neural Network for Semantic Segmentation of Remote Sensing Images

Class Semantic Enhancement Network for Semantic Segmentation

DeepGCNs-Att: Point Cloud Semantic Segmentation with Contextual Point Representations

Improving CNN-based Semantic Segmentation on Structurally Similar Data Using Contrastive Graph Convolutional Networks

Attention Guided Global Enhancement and Local Refinement Network for Semantic Segmentation

Densely connected graph convolutional network for joint semantic and instance segmentation of indoor point clouds

Boundary-aware Graph Convolution for Semantic Segmentation.

HCNet: Hierarchical Context Network for Semantic Segmentation

Improving Graph Representation for Point Cloud Segmentation Via Attentive Filtering

Superpixel-Based Attention Graph Neural Network for Semantic Segmentation in Aerial Images

A Relation-Augmented Fully Convolutional Network for Semantic Segmentation in Aerial Scenes

2 Related Work Following the success of deep neural networks for image classification

Cross Attention Network For Semantic Segmentation