Abstract:Low-level details and high-level semantics are both essential to the semantic segmentation task. However, to speed up the model inference, current approaches almost always sacrifice the low-level details, leading to a considerable decrease in accuracy. We propose to treat these spatial details and categorical semantics separately to achieve high accuracy and high efficiency for real-time semantic segmentation. For this purpose, we propose an efficient and effective architecture with a good trade-off between speed and accuracy, termed Bilateral Segmentation Network (BiSeNet V2). This architecture involves the following: (i) A detail branch, with wide channels and shallow layers to capture low-level details and generate high-resolution feature representation; (ii) A semantics branch, with narrow channels and deep layers to obtain high-level semantic context. The detail branch has wide channel dimensions and shallow layers, while the semantics branch has narrow channel dimensions and deep layers. Due to the reduction in the channel capacity and the use of a fast-downsampling strategy, the semantics branch is lightweight and can be implemented by any efficient model. We design a guided aggregation layer to enhance mutual connections and fuse both types of feature representation. Moreover, a booster training strategy is designed to improve the segmentation performance without any extra inference cost. Extensive quantitative and qualitative evaluations demonstrate that the proposed architecture shows favorable performance compared to several state-of-the-art real-time semantic segmentation approaches. Specifically, for a input, we achieve 72.6% Mean IoU on the Cityscapes test set with a speed of 156 FPS on one NVIDIA GeForce GTX 1080 Ti card, which is significantly faster than existing methods, yet we achieve better segmentation accuracy. The code and trained models are available online at https://git.io/BiSeNet.

Asymmetric-Convolution-Guided Multipath Fusion for Real-Time Semantic Segmentation Networks

BFMNet: Bilateral Feature Fusion Network with Multi-Scale Context Aggregation for Real-Time Semantic Segmentation.

FBSNet: A Fast Bilateral Symmetrical Network for Real-Time Semantic Segmentation

Faster BiSeNet: A Faster Bilateral Segmentation Network for Real-time Semantic Segmentation.

Real-time Semantic Segmentation in Traffic Scene Using Cross Stage Partial-based Encoder–decoder Network

BiSeNet: Bilateral Segmentation Network for Real-time Semantic Segmentation

ASFNet: Adaptive Multiscale Segmentation Fusion Network for Real‐time Semantic Segmentation

A lightweight multi-dimension dynamic convolutional network for real-time semantic segmentation

BiSeNet V2: Bilateral Network with Guided Aggregation for Real-Time Semantic Segmentation

ZMNet: feature fusion and semantic boundary supervision for real-time semantic segmentation

FBRNet: a feature fusion and border refinement network for real-time semantic segmentation

Real-Time Semantic Segmentation via Multiply Spatial Fusion Network

DARSegNet: A Real-Time Semantic Segmentation Method Based on Dual Attention Fusion Module and Encoder-Decoder Network

Edge-guided Nonlinear Dynamic Convolution Network for Lightweight Semantic Segmentation

AM‐MulFSNet: A Fast Semantic Segmentation Network Combining Attention Mechanism and Multi‐branch

ELANet: an efficiently lightweight asymmetrical network for real-time semantic segmentation

MAFNet: dual-branch fusion network with multiscale atrous pyramid pooling aggregate contextual features for real-time semantic segmentation

MFAFNet: A Lightweight and Efficient Network with Multi-Level Feature Adaptive Fusion for Real-Time Semantic Segmentation

A Real-Time Semantic Segmentation Approach for Autonomous Driving Scenes

BSSNet:A Real-Time Semantic Segmentation Network for Road Scenes Inspired from AutoEncoder