Abstract:There are still various challenges in remote sensing semantic segmentation due to objects diversity and complexity. Transformer-based models have significant advantages in capturing global feature dependencies for segmentation. However, it unfortunately ignores local feature details. On the other hand, convolutional neural network (CNN), with a different interaction mechanism from transformer-based models, captures more small-scale local features instead of global features. In this article, a new semantic segmentation net framework named SSNet is proposed, which incorporates an encoder–decoder structure, optimizing the advantages of both local and global features. In addition, we build feature fuse module and feature inject module to largely fuse these two-style features. The former module captures the dependencies between different positions and channels to extract multiscale features, which promotes the segmentation precision on similar objects. The latter module condenses the global information in transformer and injects it into CNN to obtain a broad global field of view, in which the depthwise strip convolution improves the segmentation accuracy on tiny objects. A CNN-based decoder progressively recovers the feature map size, and a block called atrous spatial pyramid pooling is adopted in decoder to obtain a multiscale context. The skip connection is established between the decoder and the encoder, which retains important feature information of the shallow layer network and is conducive to achieving flow of multiscale features. To evaluate our model, we compare it with current state-of-the-art models on WHDLD and Potsdam datasets. The experimental results indicate that our proposed model achieves more precise semantic segmentation.

A dual-branch hybrid network of CNN and transformer with adaptive keyframe scheduling for video semantic segmentation

EHANet: Efficient Hybrid Attention Network Towards Real-time Semantic Segmentation

SSNet: A Novel Transformer and CNN Hybrid Network for Remote Sensing Semantic Segmentation

Transformer and CNN Hybrid Deep Neural Network for Semantic Segmentation of Very-High-Resolution Remote Sensing Imagery

Dual-resolution Transformer Combined with Multi-Layer Separable Convolution Fusion Network for Real-Time Semantic Segmentation

Hybrid Dilated Convolution Network Using Attentive Kernels for Real-Time Semantic Segmentation

SCTNet: Single-Branch CNN with Transformer Semantic Information for Real-Time Segmentation

Dual-Augmented Transformer Network for Weakly Supervised Semantic Segmentation

Fast Real-time Semantic Segmentation Network with an Asymmetric Encoder-Decoder Structure

LACTNet: A Lightweight Real-Time Semantic Segmentation Network Based on an Aggregated Convolutional Neural Network and Transformer

Capturing the Spatio-Temporal Continuity for Video Semantic Segmentation.

Semantic Segmentation Network with Multi-Path Structure, Attention Reweighting and Multi-Scale Encoding

Deep Dual-resolution Networks for Real-time and Accurate Semantic Segmentation of Road Scenes

DARSegNet: A Real-Time Semantic Segmentation Method Based on Dual Attention Fusion Module and Encoder-Decoder Network

Asymmetric-Convolution-Guided Multipath Fusion for Real-Time Semantic Segmentation Networks

SegTransConv: Transformer and CNN Hybrid Method for Real-Time Semantic Segmentation of Autonomous Vehicles

LACTNet: A Lightweight Real-time Semantic Segmentation Network Based on Aggregation CNN and Transformer

Video Semantic Segmentation Via Sparse Temporal Transformer.

Lightweight Real-Time Image Semantic Segmentation Network Based on Multi-Resolution Hybrid Attention Mechanism

Bilateral Network with Residual U-blocks and Dual-Guided Attention for Real-time Semantic Segmentation

TCNet: Multiscale Fusion of Transformer and CNN for Semantic Segmentation of Remote Sensing Images