Abstract:Semantic segmentation is a crucial task in vision measurement systems that involves understanding and segmenting different objects and regions within an image. Over the years, numerous RGB-D semantic segmentation methods have been developed, leveraging the encoder-decoder architecture to achieve outstanding performance. However, existing methods have two main problems that constrain further performance improvement. Firstly, in the encoding stage, existing methods have a weak ability to fuse cross-modal information, and low-quality depth maps can easily lead to poor feature representation. Secondly, in the decoding stage, the upsampling of high-level semantic information may cause the loss of contextual information, and low-level features from the encoder may bring noises to the decoder through skip connections. To solve these issues, we propose a novel Encoding Fusion and Decoding Correction Network (EFDCNet) for RGB-D indoor semantic segmentation. First, in the encoding stage of EFDCNet, we focus on extracting valuable information from low-quality depth maps, and employ a channel-wise filter to select informative depth features. Additionally, we establish the global dependencies between RGB and depth features via the self-attention mechanism to enhance the cross-modal feature interactions, extracting discriminant and powerful features. Then, in the decoding stage of EFDCNet, we use the highest-level information as semantic guidance to compensate for the upsampling information and filter out noise from the low-level encoder features propagated through the skip connections to the decoder. Extensive experiments conducted on two widely-used RGB-D indoor semantic segmentation datasets demonstrate that the proposed EFDCNet surpasses the performance of relevant state-of-the-art methods. The code is available at https://github.com/ Mark9010/EFDCNet

DHFNet: Decoupled Hierarchical Fusion Network for RGB-T dense prediction tasks

NLFNet: Non-Local Fusion Towards Generalized Multimodal Semantic Segmentation Across RGB-Depth, Polarization, and Thermal Images

Heterogeneous Fusion and Integrity Learning Network for RGB-D Salient Object Detection

A Feature Divide-and-Conquer Network for RGB-T Semantic Segmentation

HFMDNet: Hierarchical Fusion and Multilevel Decoder Network for RGB-D Salient Object Detection

DCFusion: Dual-Headed Fusion Strategy and Contextual Information Awareness for Infrared and Visible Remote Sensing Image

Compensated Attention Feature Fusion and Hierarchical Multiplication Decoder Network for RGB-D Salient Object Detection

JL-DCF: Joint Learning and Densely-Cooperative Fusion Framework for RGB-D Salient Object Detection

DAF-Net: A Dual-Branch Feature Decomposition Fusion Network with Domain Adaptive for Infrared and Visible Image Fusion

MFUR-Net

AMDFNet: Adaptive multi-level deformable fusion network for RGB-D saliency detection

Unified Information Fusion Network for Multi-Modal RGB-D and RGB-T Salient Object Detection

Hierarchical Dynamic Filtering Network for RGB-D Salient Object Detection

FAFNet: Fully aligned fusion network for RGBD semantic segmentation based on hierarchical semantic flows

An adaptive guidance fusion network for RGB-D salient object detection

CMDFusion: Bidirectional Fusion Network with Cross-modality Knowledge Distillation for LIDAR Semantic Segmentation

EFDCNet: Encoding Fusion and Decoding Correction Network for RGB-D Indoor Semantic Segmentation

Deep Feature Selection-And-Fusion for RGB-D Semantic Segmentation

HAFFseg: RGB-Thermal semantic segmentation network with hybrid adaptive feature fusion strategy

FDGSNet: A Multi-modal Gated Segmentation Network for Remote Sensing Image Based on Frequency Decomposition

Bidirectional Feature Learning Network for RGB-D Salient Object Detection