Abstract:In this paper, we focus on exploring effective methods for faster and accurate semantic segmentation. A common practice to improve the performance is to attain high-resolution feature maps with strong semantic representation. Two strategies are widely used: atrous convolutions and feature pyramid fusion, while both are either computationally intensive or ineffective. Inspired by the Optical Flow for motion alignment between adjacent video frames, we propose a Flow Alignment Module (FAM) to learn \textit{Semantic Flow} between feature maps of adjacent levels and broadcast high-level features to high-resolution features effectively and efficiently. Furthermore, integrating our FAM to a standard feature pyramid structure exhibits superior performance over other real-time methods, even on lightweight backbone networks, such as ResNet-18 and DFNet. Then to further speed up the inference procedure, we also present a novel Gated Dual Flow Alignment Module to directly align high-resolution feature maps and low-resolution feature maps where we term the improved version network as SFNet-Lite. Extensive experiments are conducted on several challenging datasets, where results show the effectiveness of both SFNet and SFNet-Lite. In particular, when using Cityscapes test set, the SFNet-Lite series achieve 80.1 mIoU while running at 60 FPS using ResNet-18 backbone and 78.8 mIoU while running at 120 FPS using STDC backbone on RTX-3090. Moreover, we unify four challenging driving datasets into one large dataset, which we named Unified Driving Segmentation (UDS) dataset. It contains diverse domain and style information. We benchmark several representative works on UDS. Both SFNet and SFNet-Lite still achieve the best speed and accuracy trade-off on UDS, which serves as a strong baseline in such a challenging setting. The code and models are publicly available at <a class="link-external link-https" href="https://github.com/lxtGH/SFSegNets" rel="external noopener nofollow">this https URL</a>.

SGNet: a Fast and Accurate Semantic Segmentation Network Based on Semantic Guidance.

SFNet: Faster and Accurate Semantic Segmentation via Semantic Flow

ISDNet: Integrating Shallow and Deep Networks for Efficient Ultra-high Resolution Segmentation

Fast-SegNet: fast semantic segmentation network for small objects

Fast Semantic Segmentation of Remote Sensing Images Using a Network That Integrates Global and Local Information

FasterSeg: Searching for Faster Real-time Semantic Segmentation.

FSegNet: A Semantic Segmentation Network for High-Resolution Remote Sensing Images That Balances Efficiency and Performance

FBSNet: A Fast Bilateral Symmetrical Network for Real-Time Semantic Segmentation

HSNet: an Intelligent Hierarchical Semantic-Aware Network System for Real-Time Semantic Segmentation

AM‐MulFSNet: A Fast Semantic Segmentation Network Combining Attention Mechanism and Multi‐branch

SGFNet: Segmentation Guided Fusion Network for 3D Object Detection.

EFRNet: A Lightweight Network with Efficient Feature Fusion and Refinement for Real-Time Semantic Segmentation

Semantic Segmentation Network Based on Adaptive Attention and Deep Fusion Utilizing a Multi-Scale Dilated Convolutional Pyramid

FBRNet: a feature fusion and border refinement network for real-time semantic segmentation

M-FasterSeg: An efficient semantic segmentation network based on neural architecture search

Attention Guided Global Enhancement and Local Refinement Network for Semantic Segmentation

APPFNet: Adaptive point-pixel fusion network for 3D semantic segmentation with neighbor feature aggregation

ASFNet: Adaptive Multiscale Segmentation Fusion Network for Real‐time Semantic Segmentation

SPMNet: A Light-Weighted Network with Separable Pyramid Module for Real-Time Semantic Segmentation

Lightweight and Progressively-Scalable Networks for Semantic Segmentation

CGGLNet: Semantic Segmentation Network for Remote Sensing Images Based on Category-Guided Global–Local Feature Interaction