Abstract:The robustness of scene segmentation can be enhanced with the aid of other modality information, e.g. , thermal or/and depth, under poor environmental conditions. In this context, RGB-X semantic segmentation is becoming prevalent. Most existing RGB-X semantic segmentation models focus on the fusion strategy between different modalities or between multiple stages, but ignore feature recovery at the decoder side. This makes it difficult to recover the information loss due to downsampling, and also overlooks the pixel connections between segmented objects. To solve these problems, we propose a Shape and Semantic Enhancements Module (SASEM) in this paper, which is characterized by innovations on the decoder side. More specifically, we divide the decoder into a shape supervision branch and a semantic supervision branch. The former reinforces the shape information of the category by using a signed distance map. A multi-stage enhancement structure is designed to further strengthen the shape information of features. The latter directly enhances the semantic extraction capability of the decoder by employing a channel-level semantic enhancement module, which reduces the interference of the semantic information by the shape supervision branch. The two branches work together to enhance the inter-pixel relationship, thus making the decoder more capable of recovering the fused encoded features. Our proposed SASEM serves as an excellent plug-and-play module for different networks, as is evident by the experiments on various RGB-Thermal and RGB-Depth datasets, where our module can be easily integrated and help to improve the performance consistently. The code of our method will be released at: https://github.com/HenonBamboo/SASEM .

Region-based High-Level Semantics Extraction with CEDD

Image Semantic Segmentation Based on Region and Deep Residual Network

Decompose Image into Meaningful Regions Based on Contour Detector and Watershed Algorithm

Image Region Annotation Based on Segmentation and Semantic Correlation Analysis

A New Region-Edge Based Level Set Model with Applications to Image Segmentation

A semantic segmentation method for remote sensing images based on multiple contextual feature extraction

Image Region Labeling by Exploring Contextual Information of Visual Spatial and Semantic Concepts

Hierarchical Image Segmentation Using Semantic Edge Constraint

Cascading Context Enhancement Network for RGB-D Semantic Segmentation

Automatic Image Region Annotation Through Segmentation Based Visual Semantic Analysis and Discriminative Classification

Semantic segmentation using regions in natural scenes

Efficient Perceptual Region Detector Based on Object Boundary.

Object-Level Image Segmentation Using Low Level Cues.

CFIDNet: cascaded feature interaction decoder for RGB-D salient object detection

Joining Geometric and RGB Features for RGB-D Semantic Segmentation

Edge-Enhanced GCIFFNet: A Multiclass Semantic Segmentation Network Based on Edge Enhancement and Multiscale Attention Mechanism

On Exploring Shape and Semantic Enhancements for RGB-X Semantic Segmentation

A deep learning-based and adaptive region proposal algorithm for semantic segmentation

FatRegion: A Fast Adaptive Tree-Structured Region Extraction Approach

Class Semantic Enhancement Network for Semantic Segmentation

DDNet: Depth Dominant Network for Semantic Segmentation of RGB-D Images