Abstract:The hybrid architecture of convolution neural networks (CNN) and Transformer has been the most popular method for medical image segmentation. However, the existing networks based on the hybrid architecture suffer from two problems. First, although the CNN branch can capture image local features by using convolution operation, the vanilla convolution is unable to achieve adaptive extraction of image features. Second, although the Transformer branch can model the global information of images, the conventional self-attention only focuses on the spatial self-attention of images and ignores the channel and cross-dimensional self-attention leading to low segmentation accuracy for medical images with complex backgrounds. To solve these problems, we propose vision Transformer embrace convolutional neural networks for medical image segmentation (TEC-Net). Our network has two advantages. First, dynamic deformable convolution (DDConv) is designed in the CNN branch, which not only overcomes the difficulty of adaptive feature extraction using fixed-size convolution kernels, but also solves the defect that different inputs share the same convolution kernel parameters, effectively improving the feature expression ability of CNN branch. Second, in the Transformer branch, a (shifted)-window adaptive complementary attention module ((S)W-ACAM) and compact convolutional projection are designed to enable the network to fully learn the cross-dimensional long-range dependency of medical images with few parameters and calculations. Experimental results show that the proposed TEC-Net provides better medical image segmentation results than SOTA methods including CNN and Transformer networks. In addition, our TEC-Net requires fewer parameters and computational costs and does not rely on pre-training. The code is publicly available at <a class="link-external link-https" href="https://github.com/SR0920/TEC-Net" rel="external noopener nofollow">this https URL</a>.

BMCS-Net: A Bi-directional multi-scale cascaded segmentation network based on transformer-guided feature Aggregation for medical images

MS-TCNet: An effective Transformer–CNN combined network using multi-scale feature learning for 3D medical image segmentation

MCA-Net: Multi-scale Context-aware Aggregation Network for Medical Image Segmentation.

Sub-pixel multi-scale fusion network for medical image segmentation

HmsU-Net: A hybrid multi-scale U-net based on a CNN and transformer for medical image segmentation

CTCNet: A Bi-directional Cascaded Segmentation Network Combining Transformers with CNNs for Skin Lesions.

HTC-Net: A hybrid CNN-transformer framework for medical image segmentation

2-D general network based on channel-space attention for medical image segmentation

Multi-Scale Convolution-Transformer Fusion Network for Endoscopic Image Segmentation.

MpMsCFMA-Net: Multi-path Multi-scale Context Feature Mixup and Aggregation Network for medical image segmentation

MSA$^2$Net: Multi-scale Adaptive Attention-guided Network for Medical Image Segmentation

A Lightweight Multi-Scale Multi-Angle Dynamic Interactive Transformer-CNN Fusion Model for 3D Medical Image Segmentation

Multiscale Transunet + + : Dense Hybrid U-Net with Transformer for Medical Image Segmentation

CascadeMedSeg: integrating pyramid vision transformer with multi-scale fusion for precise medical image segmentation

MambaClinix: Hierarchical Gated Convolution and Mamba-Based U-Net for Enhanced 3D Medical Image Segmentation

TEC-Net: Vision Transformer Embrace Convolutional Neural Networks for Medical Image Segmentation

BRAU-Net++: U-Shaped Hybrid CNN-Transformer Network for Medical Image Segmentation

MSTCNet: Parallel Multi-Scale Network For Medical Image Segmentation.

TransSea: Hybrid CNN–Transformer With Semantic Awareness for 3-D Brain Tumor Segmentation

TransSea: Hybrid CNN-Transformer with Semantic Awareness for 3D Brain Tumor Segmentation

MSCT-UNET: multi-scale contrastive transformer within U-shaped network for medical image segmentation