Abstract:Semantic segmentation is a crucial task in vision measurement systems that involves understanding and segmenting different objects and regions within an image. Over the years, numerous RGB-D semantic segmentation methods have been developed, leveraging the encoder-decoder architecture to achieve outstanding performance. However, existing methods have two main problems that constrain further performance improvement. Firstly, in the encoding stage, existing methods have a weak ability to fuse cross-modal information, and low-quality depth maps can easily lead to poor feature representation. Secondly, in the decoding stage, the upsampling of high-level semantic information may cause the loss of contextual information, and low-level features from the encoder may bring noises to the decoder through skip connections. To solve these issues, we propose a novel Encoding Fusion and Decoding Correction Network (EFDCNet) for RGB-D indoor semantic segmentation. First, in the encoding stage of EFDCNet, we focus on extracting valuable information from low-quality depth maps, and employ a channel-wise filter to select informative depth features. Additionally, we establish the global dependencies between RGB and depth features via the self-attention mechanism to enhance the cross-modal feature interactions, extracting discriminant and powerful features. Then, in the decoding stage of EFDCNet, we use the highest-level information as semantic guidance to compensate for the upsampling information and filter out noise from the low-level encoder features propagated through the skip connections to the decoder. Extensive experiments conducted on two widely-used RGB-D indoor semantic segmentation datasets demonstrate that the proposed EFDCNet surpasses the performance of relevant state-of-the-art methods. The code is available at https://github.com/ Mark9010/EFDCNet

Image Enhancement and Translation for RGB-D Indoor Scene Recognition

ACNET: Attention Based Network to Exploit Complementary Features for RGBD Semantic Segmentation.

Translate-to-Recognize Networks for RGB-D Scene Recognition

Depth-assisted RefineNet for Indoor Semantic Segmentation

Cross-Modal Pyramid Translation for RGB-D Scene Recognition

TCANet: three-stream coordinate attention network for RGB-D indoor semantic segmentation

RGB-D Scene Recognition Via Spatial-Related Multi-Modal Feature Learning

MSN: Modality Separation Networks for RGB-D Scene Recognition

Dual Attention Based Multi-scale Feature Fusion Network for Indoor RGBD Semantic Segmentation.

INDOOR SEMANTIC SEGMENTATION FROM RGB-D IMAGES BY INTEGRATING FULLY CONVOLUTIONAL NETWORK WITH HIGHER-ORDER MARKOV RANDOM FIELD

EFDCNet: Encoding Fusion and Decoding Correction Network for RGB-D Indoor Semantic Segmentation

Interactive Efficient Multi-Task Network for RGB-D Semantic Segmentation

Attention-Based Deep Neural Network Combined Local and Global Features for Indoor Scene Recognition

RGB×D: Learning Depth-Weighted RGB Patches for RGB-D Indoor Semantic Segmentation

Depth Images Could Tell Us More: Enhancing Depth Discriminability for RGB-D Scene Recognition

Cross-Modal Matching and Adaptive Graph Attention Network for RGB-D Scene Recognition

Image attention transformer network for indoor 3D object detection

SRNet: A 3D Scene Recognition Network Using Static Graph and Dense Semantic Fusion.

AMCFNet: Asymmetric multiscale and crossmodal fusion network for RGB-D semantic segmentation in indoor service robots

The Network of Attention-Aware Multimodal Fusion for RGB-D Indoor Semantic Segmentation Method

Indoor Scene Recognition from RGB-D Images by Learning Scene Bases