Abstract:In this paper, we focus on exploring effective methods for faster and accurate semantic segmentation. A common practice to improve the performance is to attain high-resolution feature maps with strong semantic representation. Two strategies are widely used: atrous convolutions and feature pyramid fusion, while both are either computationally intensive or ineffective. Inspired by the Optical Flow for motion alignment between adjacent video frames, we propose a Flow Alignment Module (FAM) to learn \textit{Semantic Flow} between feature maps of adjacent levels and broadcast high-level features to high-resolution features effectively and efficiently. Furthermore, integrating our FAM to a standard feature pyramid structure exhibits superior performance over other real-time methods, even on lightweight backbone networks, such as ResNet-18 and DFNet. Then to further speed up the inference procedure, we also present a novel Gated Dual Flow Alignment Module to directly align high-resolution feature maps and low-resolution feature maps where we term the improved version network as SFNet-Lite. Extensive experiments are conducted on several challenging datasets, where results show the effectiveness of both SFNet and SFNet-Lite. In particular, when using Cityscapes test set, the SFNet-Lite series achieve 80.1 mIoU while running at 60 FPS using ResNet-18 backbone and 78.8 mIoU while running at 120 FPS using STDC backbone on RTX-3090. Moreover, we unify four challenging driving datasets into one large dataset, which we named Unified Driving Segmentation (UDS) dataset. It contains diverse domain and style information. We benchmark several representative works on UDS. Both SFNet and SFNet-Lite still achieve the best speed and accuracy trade-off on UDS, which serves as a strong baseline in such a challenging setting. The code and models are publicly available at <a class="link-external link-https" href="https://github.com/lxtGH/SFSegNets" rel="external noopener nofollow">this https URL</a>.

SAN: Side Adapter Network for Open-vocabulary Semantic Segmentation

Side Adapter Network for Open-Vocabulary Semantic Segmentation

Integrating Spatial Prior Adapter for Enhancing SAM Performance in Medical Image Segmentation

When Masked Image Modeling Meets Source-free Unsupervised Domain Adaptation: Dual-Level Masked Network for Semantic Segmentation

Spatial-Assistant Encoder-Decoder Network for Real Time Semantic Segmentation

SAM-Adapter: Adapting Segment Anything in Underperformed Scenes

A Simple Baseline for Open-Vocabulary Semantic Segmentation with Pre-trained Vision-language Model

SAM Fails to Segment Anything? – SAM-Adapter: Adapting SAM in Underperformed Scenes: Camouflage, Shadow, Medical Image Segmentation, and More

Effective SAM Combination for Open-Vocabulary Semantic Segmentation

Open-Vocabulary Segmentation with Semantic-Assisted Calibration

Open-Vocabulary Semantic Segmentation with Image Embedding Balancing

RG-SAN: Rule-Guided Spatial Awareness Network for End-to-End 3D Referring Expression Segmentation

Harnessing Vision Foundation Models for High-Performance, Training-Free Open Vocabulary Segmentation

OpenSD: Unified Open-Vocabulary Segmentation and Detection

Open-Vocabulary Semantic Segmentation with Decoupled One-Pass Network

MANet: Fine-Tuning Segment Anything Model for Multimodal Remote Sensing Semantic Segmentation

PosSAM: Panoptic Open-vocabulary Segment Anything

RSAM-Seg: A SAM-based Approach with Prior Knowledge Integration for Remote Sensing Image Semantic Segmentation

Mask-Adapter: The Devil is in the Masks for Open-Vocabulary Segmentation

SFNet: Faster and Accurate Semantic Segmentation via Semantic Flow

GoodSAM++: Bridging Domain and Capacity Gaps via Segment Anything Model for Panoramic Semantic Segmentation