Abstract:Deep Convolutional Neural Networks (DCNNs) have recently shown outstanding performance in semantic image segmentation. However, state-of-the-art DCNN-based semantic segmentation methods usually suffer from high computational complexity due to the use of complex network architectures. This greatly limits their applications in the real-world scenarios that require real-time processing. In this paper, we propose a real-time high-performance DCNN-based method for robust semantic segmentation of urban street scenes, which achieves a good trade-off between accuracy and speed. Specifically, a Lightweight Baseline Network with Atrous convolution and Attention (LBN-AA) is firstly used as our baseline network to efficiently obtain dense feature maps. Then, the Distinctive Atrous Spatial Pyramid Pooling (DASPP), which exploits the different sizes of pooling operations to encode the rich and distinctive semantic information, is developed to detect objects at multiple scales. Meanwhile, a Spatial detail-Preserving Network (SPN) with shallow convolutional layers is designed to generate high-resolution feature maps preserving the detailed spatial information. Finally, a simple but practical Feature Fusion Network (FFN) is used to effectively combine both shallow and deep features from the semantic branch (DASPP) and the spatial branch (SPN), respectively. Extensive experimental results show that the proposed method respectively achieves the accuracy of 73.6% and 68.0% mean Intersection over Union (mIoU) with the inference speed of 51.0 fps and 39.3 fps on the challenging Cityscapes and CamVid test datasets (by only using a single NVIDIA TITAN X card). This demonstrates that the proposed method offers excellent performance at the real-time speed for semantic segmentation of urban street scenes.

AtICNet: Semantic Segmentation with Atrous Spatial Pyramid Pooling in Image Cascade Network

Cascaded Multiscale Structure with Self-Smoothing Atrous Convolution for Semantic Segmentation

Atrous Convolutional Neural Network (ACNN) for Semantic Image Segmentation with Full-Scale Feature Maps

DSNet: A Novel Way to Use Atrous Convolutions in Semantic Segmentation

DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs

Real-Time High-Performance Semantic Image Segmentation of Urban Street Scenes

Partial Atrous Cascade R-CNN

ISDNet: Integrating Shallow and Deep Networks for Efficient Ultra-high Resolution Segmentation

DPNet: Dual-Pyramid Semantic Segmentation Network Based on Improved Deeplabv3 Plus

LiDAR Point Clouds Semantic Segmentation in Autonomous Driving Based on Asymmetrical Convolution

APDC-Net: Attention Pooling-Based Convolutional Network for Aerial Scene Classification

An Aerial Image Segmentation Approach Based on Enhanced Multi-Scale Convolutional Neural Network

Semantic Segmentation of Aerial Imagery Via Split-Attention Networks with Disentangled Nonlocal and Edge Supervision

DPANET:Dual Pooling Attention Network for Semantic Segmentation

Aggregation Architecture and All-to-one Network for Real-Time Semantic Segmentation

PCANet: Pyramid convolutional attention network for semantic segmentation

Semantic Segmentation Network Based on Adaptive Attention and Deep Fusion Utilizing a Multi-Scale Dilated Convolutional Pyramid

Aerial-BiSeNet: A real-time semantic segmentation network for high resolution aerial imagery

Rethinking Atrous Convolution for Semantic Image Segmentation

FCPFNet: Feature Complementation Network with Pyramid Fusion for Semantic Segmentation

Densely Connected Feature Pyramid Network for Image Segmentation