Abstract:The encoder-decoder structure is the basic structure of most semantic segmentation models and is adopted by a large number of segmentation models. How to effectively extract image features and achieve high-precision mapping through the optimal design of encoder and decoder is the key issue of current research. SegFormer designs an encoder with excellent performance, which fully extracts the feature information of different semantic granularity in the image with a large receptive field. Even if a simple fully connected layer decoder is used, excellent segmentation results can also be achieved. However, this simplified decoder does not make full use of the advantages of the SegFormer encoder. Therefore, a decoder structure with dual-path multi-scale feature fusion is designed in this paper, and the decoder is redesigned according to the characteristics of the SegFormer encoder. The decoder adopts a dual-path structure, one path passes the abstract global information layer by layer to the local detail information through the layer-by-layer upsampling fusion module (LFM), and gradually upsamples the feature maps obtained from the encoder, and then use the channel fusion module to learn the importance of different channels in the deep abstract semantic feature map and the shallow local detail feature map, and perform dynamic fusion to obtain a feature map containing both abstract semantic information and local details. The other path takes advantage of the large receptive field of the feature map output by the SegFormer encoder, and uses the weighted hybrid multi-scale feature extraction module (WMF) to extract multi-scale features containing global semantics from the deep semantic feature map finally output by the encoder. Finally, the Deep Feature Fusion Module (DFM) is used to fuse the outputs of the first two modules, fully mining the multi-scale global information in the encoder, and obtaine the feature maps with rich semantic information, which effectively improves the algorithm model performance.

CFPFormer: Feature-pyramid like Transformer Decoder for Segmentation and Detection

A Transformer-Based Object Detector with Coarse-Fine Crossing Representations

Feature Shrinkage Pyramid for Camouflaged Object Detection with Transformers.

FeedFormer: Revisiting Transformer Decoder for Efficient Semantic Segmentation

A Transformer-Based Decoder for Semantic Segmentation with Multi-level Context Mining

ConvFormer: Plug-and-Play CNN-Style Transformers for Improving Medical Image Segmentation

SDPT: Semantic-Aware Dimension-Pooling Transformer for Image Segmentation

A Holistically-Guided Decoder for Deep Representation Learning with Applications to Semantic Segmentation and Object Detection.

Transformer-Based Context Condensation for Boosting Feature Pyramids in Object Detection

CPFTransformer: transformer fusion context pyramid medical image segmentation network

ELiFormer: A Hierarchical Transformer Based Model with Efficient Encoder and Lightweight Decoder for Semantic Segmentation.

FPNFormer: Rethink the Method of Processing the Rotation-Invariance and Rotation-Equivariance on Arbitrary-Oriented Object Detection

Pyramid Fusion Transformer for Semantic Segmentation

Content-Augmented Feature Pyramid Network with Light Linear Transformers

A Dual-Path Multi-Scale Feature Fusion Decoder for SegFormer.

CardiacSegFormer: Transformer for Semantic Segmentation of Cardiac Images.

HD-Former: A hierarchical dependency Transformer for medical image segmentation

Cats: Complementary CNN and Transformer Encoders for Segmentation

FDR-TransUNet: A novel encoder-decoder architecture with vision transformer for improved medical image segmentation

SegFormer: Simple and Efficient Design for Semantic Segmentation with Transformers

Enhanced Feature Pyramid Network for Semantic Segmentation.