Abstract:Real-time semantic segmentation, which aims to achieve high segmentation accuracy at real-time inference speed, has received substantial attention over the past few years. However, many state-of-the-art real-time semantic segmentation methods tend to sacrifice some spatial details or contextual information for fast inference, thus leading to degradation in segmentation quality. In this paper, we propose a novel Deep Multi-branch Aggregation Network (called DMA-Net) based on the encoder-decoder structure to perform real-time semantic segmentation in street scenes. Specifically, we first adopt ResNet-18 as the encoder to efficiently generate various levels of feature maps from different stages of convolutions. Then, we develop a Multi-branch Aggregation Network (MAN) as the decoder to effectively aggregate different levels of feature maps and capture the multi-scale information. In MAN, a lattice enhanced residual block is designed to enhance feature representations of the network by taking advantage of the lattice structure. Meanwhile, a feature transformation block is introduced to explicitly transform the feature map from the neighboring branch before feature aggregation. Moreover, a global context block is used to exploit the global contextual information. These key components are tightly combined and jointly optimized in a unified network. Extensive experimental results on the challenging Cityscapes and CamVid datasets demonstrate that our proposed DMA-Net respectively obtains 77.0% and 73.6% mean Intersection over Union (mIoU) at the inference speed of 46.7 FPS and 119.8 FPS by only using a single NVIDIA GTX 1080Ti GPU. This shows that DMA-Net provides a good tradeoff between segmentation quality and speed for semantic segmentation in street scenes.

Non-Local Aggregation for RGB-D Semantic Segmentation

ACNET: Attention Based Network to Exploit Complementary Features for RGBD Semantic Segmentation.

NLFNet: Non-Local Fusion Towards Generalized Multimodal Semantic Segmentation Across RGB-Depth, Polarization, and Thermal Images

DCANet: Differential Convolution Attention Network for RGB-D Semantic Segmentation

An RGB-D Fusion Based Semantic Segmentation Algorithm Based on Neighborhood Metric Relations

TCANet: three-stream coordinate attention network for RGB-D indoor semantic segmentation

Spatial-information Guided Adaptive Context-aware Network for Efficient RGB-D Semantic Segmentation

Global-Local Propagation Network for RGB-D Semantic Segmentation

Cascading Context Enhancement Network for RGB-D Semantic Segmentation

Attention-based Dual Context Aggregation for Image Semantic Segmentation

MIPANet: optimizing RGB-D semantic segmentation through multi-modal interaction and pooling attention

Optimizing rgb-d semantic segmentation through multi-modal interaction and pooling attention

A Fusion Network for Semantic Segmentation Using RGB-D Data

DDNet: Depth Dominant Network for Semantic Segmentation of RGB-D Images

Asymmetric Non-Local Neural Networks for Semantic Segmentation.

Deep Multi-Branch Aggregation Network for Real-Time Semantic Segmentation in Street Scenes.

NeiEA-NET: Semantic segmentation of large-scale point cloud scene via neighbor enhancement and aggregation

Interactive Efficient Multi-Task Network for RGB-D Semantic Segmentation

Global Aggregation then Local Distribution for Scene Parsing

Rethinking feature aggregation for deep RGB-D salient object detection

CCANet: Cross-Modality Comprehensive Feature Aggregation Network for Indoor Scene Semantic Segmentation