Abstract:Low-level details and high-level semantics are both essential to the semantic segmentation task. However, to speed up the model inference, current approaches almost always sacrifice the low-level details, leading to a considerable decrease in accuracy. We propose to treat these spatial details and categorical semantics separately to achieve high accuracy and high efficiency for real-time semantic segmentation. For this purpose, we propose an efficient and effective architecture with a good trade-off between speed and accuracy, termed Bilateral Segmentation Network (BiSeNet V2). This architecture involves the following: (i) A detail branch, with wide channels and shallow layers to capture low-level details and generate high-resolution feature representation; (ii) A semantics branch, with narrow channels and deep layers to obtain high-level semantic context. The detail branch has wide channel dimensions and shallow layers, while the semantics branch has narrow channel dimensions and deep layers. Due to the reduction in the channel capacity and the use of a fast-downsampling strategy, the semantics branch is lightweight and can be implemented by any efficient model. We design a guided aggregation layer to enhance mutual connections and fuse both types of feature representation. Moreover, a booster training strategy is designed to improve the segmentation performance without any extra inference cost. Extensive quantitative and qualitative evaluations demonstrate that the proposed architecture shows favorable performance compared to several state-of-the-art real-time semantic segmentation approaches. Specifically, for a input, we achieve 72.6% Mean IoU on the Cityscapes test set with a speed of 156 FPS on one NVIDIA GeForce GTX 1080 Ti card, which is significantly faster than existing methods, yet we achieve better segmentation accuracy. The code and trained models are available online at https://git.io/BiSeNet.

BSSNet:A Real-Time Semantic Segmentation Network for Road Scenes Inspired from AutoEncoder

Real-time Semantic Segmentation in Traffic Scene Using Cross Stage Partial-based Encoder–decoder Network

FBSNet: A Fast Bilateral Symmetrical Network for Real-Time Semantic Segmentation

Spatial-Assistant Encoder-Decoder Network for Real Time Semantic Segmentation

BiSeNet: Bilateral Segmentation Network for Real-time Semantic Segmentation

DSANet: Dilated Spatial Attention for Real-Time Semantic Segmentation in Urban Street Scenes.

Fast Semantic Segmentation for Scene Perception

LCFNets: Compensation Strategy for Real-time Semantic Segmentation of Autonomous Driving

Asymmetric-Convolution-Guided Multipath Fusion for Real-Time Semantic Segmentation Networks

Semantics Recalibration and Detail Enhancement Network for Real‐time Semantic Segmentation

A Real-Time Semantic Segmentation Approach for Autonomous Driving Scenes

BiSeNet V2: Bilateral Network with Guided Aggregation for Real-Time Semantic Segmentation

Three-branch Semantic Segmentation Network

IMM-Net: Integrated Multi-stream Memory Network for Real-Time Semantic Segmentation

Rethinking BiSeNet for Real-time Semantic Segmentation

BiSeNet V3: Bilateral Segmentation Network with Coordinate Attention for Real-time Semantic Segmentation

Faster BiSeNet: A Faster Bilateral Segmentation Network for Real-time Semantic Segmentation.

SPNet: Dual-Branch Network with Spatial Supplementary Information for Building and Water Segmentation of Remote Sensing Images

FBRNet: a feature fusion and border refinement network for real-time semantic segmentation

Multi-feature Fusion Network for Road Scene Semantic Segmentation.