Abstract:Global context information is vital in visual understanding problems, especially in pixel-level semantic segmentation. The mainstream methods adopt the self-attention mechanism to model global context information. However, pixels belonging to different classes usually have weak feature correlation. Modeling the global pixel-level correlation matrix indiscriminately is extremely redundant in the self-attention mechanism. In order to solve the above problem, we propose a hierarchical context network to differentially model homogeneous pixels with strong correlations and heterogeneous pixels with weak correlations. Specifically, we first propose a multi-scale guided pre-segmentation module to divide the entire feature map into different classed-based homogeneous regions. Within each homogeneous region, we design the pixel context module to capture pixel-level correlations. Subsequently, different from the self-attention mechanism that still models weak heterogeneous correlations in a dense pixel-level manner, the region context module is proposed to model sparse region-level dependencies using a unified representation of each region. Through aggregating fine-grained pixel context features and coarse-grained region context features, our proposed network can not only hierarchically model global context information but also harvest multi-granularity representations to more robustly identify multi-scale objects. We evaluate our approach on Cityscapes and the ISPRS Vaihingen dataset. Without Bells or Whistles, our approach realizes a mean IoU of 82.8% and overall accuracy of 91.4% on Cityscapes and ISPRS Vaihingen test set, achieving state-of-the-art results.

Context propagation embedding network for weakly supervised semantic segmentation

Spatially-Aware Context Neural Networks.

Weakly supervised segmentation via instance-aware propagation

Context Prior for Scene Segmentation.

Context-Reinforced Semantic Segmentation.

Class Semantic Enhancement Network for Semantic Segmentation

Attention Guided Global Enhancement and Local Refinement Network for Semantic Segmentation

Context Encoding for Semantic Segmentation

Semantic boundary enhancement and position attention network with long-range dependency for semantic segmentation

Context Union Edge Network for Semantic Segmentation of Small-Scale Objects in Very High Resolution Remote Sensing Images

CI-Net: a joint depth estimation and semantic segmentation network using contextual information

Weakly Supervised Semantic Segmentation via Progressive Patch Learning

Boosting Semantic Segmentation from the Perspective of Explicit Class Embeddings

Long and short-range relevance context network for semantic segmentation

Lightweight semantic segmentation network with configurable context and small object attention

Weakly-Supervised Spatial Context Networks.

Global Context Dependencies Aware Network for Efficient Semantic Segmentation of Fine-Resolution Remoted Sensing Images

HCNet: Hierarchical Context Network for Semantic Segmentation

Learning to Predict Context-adaptive Convolution for Semantic Segmentation

Semantic Context Encoding for Accurate 3D Point Cloud Segmentation

Encouraging the Mutual Interact Between Dataset-Level and Image-Level Context for Semantic Segmentation of Remote Sensing Image