Abstract:Background and Objectives: Transformers using self-attention mechanisms have recently advanced medical imaging by modeling long-range semantic dependencies, though they lack CNNs’ ability to capture local spatial details. This study introduced a novel segmentation network derived from a mixed CNN-Transformer (MixFormer) feature extraction backbone to enhance medical image segmentation. Method: The MixFormer network seamlessly integrates global and local information from Transformer and CNN architectures during the downsampling process. To comprehensively capture the inter-scale perspective, we introduced a Multi-scale Spatial-aware Fusion (MSAF) module, enabling effective interaction between coarse and fine feature representations. Additionally, we proposed a Mixed Multi-branch Dilated Attention (MMDA) module to bridge the semantic gap between encoding and decoding stages while emphasizing specific regions. Lastly, we implemented a CNN-based upsampling approach to recover low-level features, substantially improving segmentation accuracy. Results: Experimental validations on prevalent medical image datasets demonstrated the superior performance of MixFormer. On the Synapse dataset, our approach achieved a mean Dice Similarity Coefficient (DSC) of 82.64% and a mean Hausdorff Distance (HD) of 12.67 mm. On the ACDC dataset, the DSC was 91.01%. On the ISIC 2018 dataset, the model achieved a mean Intersection over Union (mIOU) of 0.841, Accuracy of 0.958, Precision of 0.910, Recall of 0.934, and an F1 score of 0.913. For the Kvasir-SEG dataset, we recorded a mean Dice of 0.9247, mIOU of 0.8615, Precision of 0.9181, and Recall of 0.9463. On the CVC-ClinicDB dataset, the results were a mean Dice of 0.9441, mIOU of 0.8922, Precision of 0.9437, and Recall of 0.9458. Conclusion: These findings underscore the superior segmentation performance of MixFormer compared to most mainstream segmentation networks such as CNNs and other Transformerbased structures.

MetaFormer and CNN Hybrid Model for Polyp Image Segmentation

MixFormer: a Mixed CNN-Transformer Backbone for Medical Image Segmentation

Meta-Polyp: a baseline for efficient Polyp segmentation

Multi‐scale nested UNet with transformer for colorectal polyp segmentation

Probabilistic Modeling Ensemble Vision Transformer Improves Complex Polyp Segmentation

NA-segformer: A multi-level transformer model based on neighborhood attention for colonoscopic polyp segmentation

RetSeg: Retention-based Colorectal Polyps Segmentation Network

ACU-TransNet: Attention and convolution-augmented UNet-transformer network for polyp segmentation

Colorectal Polyp Segmentation Based on Group Convolution and Transformer

MugenNet: A Novel Combined Convolution Neural Network and Transformer Network with its Application for Colonic Polyp Image Segmentation

SeUNet-Trans: A Simple yet Effective UNet-Transformer Model for Medical Image Segmentation

CTNet: Contrastive Transformer Network for Polyp Segmentation

Polyp Segmentation With the FCB-SwinV2 Transformer

TransNetR: Transformer-based Residual Network for Polyp Segmentation with Multi-Center Out-of-Distribution Testing

TMPSformer: An Efficient Hybrid Transformer-MLP Network for Polyp Segmentation

SegT: A Novel Separated Edge-guidance Transformer Network for Polyp Segmentation

Multi parallel U-net encoder network for effective polyp image segmentation

PolySegNet: improving polyp segmentation through swin transformer and vision transformer fusion

Three-stage polyp segmentation network based on reverse attention feature purification with Pyramid Vision Transformer

Using DUCK-Net for Polyp Image Segmentation