Abstract:Red-green-blue and depth (RGB-D) semantic segmentation is essential for indoor service robots to achieve accurate artificial intelligence. Various RGB-D indoor semantic segmentation methods have been proposed since the widespread adoption of depth maps. These methods have focused mainly on integrating the multiscale and crossmodal features extracted from RGB images and depth maps in the encoder and used unified strategies to recover the local details at the decoder progressively. However, these methods emphasized crossmodal fusion at the encoder, neglecting the distinguishability between RGB and depth features during decoding, thereby undermining the segmentation performance . To adequately exploit the features, we propose an efficient encoder-decoder architecture called asymmetric multiscale and crossmodal fusion network (AMCFNet) endowed with a differential feature integration strategy. Unlike existing methods, we use simple crossmodal fusion at the encoder and design an elaborate decoder to improve the semantic segmentation performance. Specifically, considering high- and low-level features, we propose a semantic aggregation module (SAM) to process the multiscale and crossmodal features in the last three network layers to aggregate high-level semantic information through a cascaded pyramid structure. Moreover, we design a spatial detail supplement module using low-level spatial details from depth maps to adaptively fuse these details and the information obtained from the SAM. Extensive experiments are conducted to demonstrate that the proposed AMCFNet outperforms state-of-the-art approaches.

THCANet: Two-layer hop cascaded asymptotic network for robot-driving road-scene semantic segmentation in RGB-D images

NLFNet: Non-Local Fusion Towards Generalized Multimodal Semantic Segmentation Across RGB-Depth, Polarization, and Thermal Images

TCANet: three-stream coordinate attention network for RGB-D indoor semantic segmentation

Context-Aware Interaction Network for RGB-T Semantic Segmentation

CCANet: Cross-Modality Comprehensive Feature Aggregation Network for Indoor Scene Semantic Segmentation

Context Aggregation Network for Remote Sensing Image Semantic Segmentation

CANet: Co-attention Network for RGB-D Semantic Segmentation

D-CANet: Diverse Class-Aware Coding and Decoding Structure Network for Semantic Segmentation of High-Resolution Remote Sensing Images

MTANet: Multitask-Aware Network with Hierarchical Multimodal Fusion for RGB-T Urban Scene Understanding

DCANet: Differential Convolution Attention Network for RGB-D Semantic Segmentation

MFCANet: A Road Scene Segmentation Network Based on Multi-Scale Feature Fusion and Context Information Aggregation

Attention-guided chained context aggregation for semantic segmentation

Adaptive multi-scale dual attention network for semantic segmentation

Interactive Context-Aware Network for RGB-T Salient Object Detection

Hybridizing Cross-Level Contextual and Attentive Representations for Remote Sensing Imagery Semantic Segmentation

AMCFNet: Asymmetric multiscale and crossmodal fusion network for RGB-D semantic segmentation in indoor service robots

Spatial-information Guided Adaptive Context-aware Network for Efficient RGB-D Semantic Segmentation

DCANet: Dense Context-Aware Network for Semantic Segmentation

A Fusion Network for Semantic Segmentation Using RGB-D Data

CI-Net: a joint depth estimation and semantic segmentation network using contextual information