Abstract:The hybrid architecture of convolution neural networks (CNN) and Transformer has been the most popular method for medical image segmentation. However, the existing networks based on the hybrid architecture suffer from two problems. First, although the CNN branch can capture image local features by using convolution operation, the vanilla convolution is unable to achieve adaptive extraction of image features. Second, although the Transformer branch can model the global information of images, the conventional self-attention only focuses on the spatial self-attention of images and ignores the channel and cross-dimensional self-attention leading to low segmentation accuracy for medical images with complex backgrounds. To solve these problems, we propose vision Transformer embrace convolutional neural networks for medical image segmentation (TEC-Net). Our network has two advantages. First, dynamic deformable convolution (DDConv) is designed in the CNN branch, which not only overcomes the difficulty of adaptive feature extraction using fixed-size convolution kernels, but also solves the defect that different inputs share the same convolution kernel parameters, effectively improving the feature expression ability of CNN branch. Second, in the Transformer branch, a (shifted)-window adaptive complementary attention module ((S)W-ACAM) and compact convolutional projection are designed to enable the network to fully learn the cross-dimensional long-range dependency of medical images with few parameters and calculations. Experimental results show that the proposed TEC-Net provides better medical image segmentation results than SOTA methods including CNN and Transformer networks. In addition, our TEC-Net requires fewer parameters and computational costs and does not rely on pre-training. The code is publicly available at <a class="link-external link-https" href="https://github.com/SR0920/TEC-Net" rel="external noopener nofollow">this https URL</a>.

TSCA-Net: Transformer based spatial-channel attention segmentation network for medical images

A Improved Spatial-Reduction Attention Transformer for Medical Image Segmentation

TransAttUnet: Multi-level Attention-guided U-Net with Transformer for Medical Image Segmentation.

TEC-Net: Vision Transformer Embrace Convolutional Neural Networks for Medical Image Segmentation

BRAU-Net++: U-Shaped Hybrid CNN-Transformer Network for Medical Image Segmentation

DW-SCA Unet: medical image segmentation based on depth-wise separable convolutional attention U-shaped network

Dual-attention transformer-based hybrid network for multi-modal medical image segmentation

Dual Channel-Spatial Self-Attention Transformer and CNN Synergy Network for 3D Medical Image Segmentation

DA-TransUNet: Integrating Spatial and Channel Dual Attention with Transformer U-Net for Medical Image Segmentation

Sfe-Transunet: A Transformer-Based U-Net With Skipped Features Enhancer For Medical Image Segmentation

SCA-Former: transformer-like network based on stream-cross attention for medical image segmentation

UCTNet: Uncertainty-guided CNN-Transformer hybrid networks for medical image segmentation

A Multi-Scale Cross-Fusion Medical Image Segmentation Network Based on Dual-Attention Mechanism Transformer

CiT-Net: Convolutional Neural Networks Hand in Hand with Vision Transformers for Medical Image Segmentation

SCTransNet: 3D Medical Image Segmentation Model Based on the Fusion of CNN and Transformer.

Multi-scale Neighborhood Attention Transformer on U-Net for Medical Image Segmentation.

[Multi-scale medical image segmentation based on pixel encoding and spatial attention mechanism]

MSCT-UNET: multi-scale contrastive transformer within U-shaped network for medical image segmentation

DECTNet: Dual Encoder Network combined convolution and Transformer architecture for medical image segmentation

TransUNet: Transformers Make Strong Encoders for Medical Image Segmentation

A Hybrid Enhanced Attention Transformer Network for Medical Ultrasound Image Segmentation