Abstract:The hybrid architecture of convolution neural networks (CNN) and Transformer has been the most popular method for medical image segmentation. However, the existing networks based on the hybrid architecture suffer from two problems. First, although the CNN branch can capture image local features by using convolution operation, the vanilla convolution is unable to achieve adaptive extraction of image features. Second, although the Transformer branch can model the global information of images, the conventional self-attention only focuses on the spatial self-attention of images and ignores the channel and cross-dimensional self-attention leading to low segmentation accuracy for medical images with complex backgrounds. To solve these problems, we propose vision Transformer embrace convolutional neural networks for medical image segmentation (TEC-Net). Our network has two advantages. First, dynamic deformable convolution (DDConv) is designed in the CNN branch, which not only overcomes the difficulty of adaptive feature extraction using fixed-size convolution kernels, but also solves the defect that different inputs share the same convolution kernel parameters, effectively improving the feature expression ability of CNN branch. Second, in the Transformer branch, a (shifted)-window adaptive complementary attention module ((S)W-ACAM) and compact convolutional projection are designed to enable the network to fully learn the cross-dimensional long-range dependency of medical images with few parameters and calculations. Experimental results show that the proposed TEC-Net provides better medical image segmentation results than SOTA methods including CNN and Transformer networks. In addition, our TEC-Net requires fewer parameters and computational costs and does not rely on pre-training. The code is publicly available at <a class="link-external link-https" href="https://github.com/SR0920/TEC-Net" rel="external noopener nofollow">this https URL</a>.

TESN: Transformers Enhanced Segmentation Network for Accurate Nanoparticle Size Measurement of TEM Images

TSO-DETR: A Network for Small Object Detection of Cervical Cells in TCT Smear

Size Measurement of Nanoparticle Assembly Using Multilevel Segmented TEM Images

TEC-Net: Vision Transformer Embrace Convolutional Neural Networks for Medical Image Segmentation

Automatic Center Identification of Electron Diffraction with Multi-scale Transformer Networks

Deep learning for automated size and shape analysis of nanoparticles in scanning electron microscopy

LET-Net: locally enhanced transformer network for medical image segmentation

Deep learning-based size prediction for optical trapped nanoparticles and extracellular vesicles from limited bandwidth camera detection

Spherical Nanoparticle Parameter Measurement Method Based on Mask R-CNN Segmentation and Edge Fitting

TT-Net: Tensorized Transformer Network for 3D medical image segmentation

Adaptive Patching for High-resolution Image Segmentation with Transformers

MT-SCnet: multi-scale token divided and spatial-channel fusion transformer network for microscopic hyperspectral image segmentation

TransUNet: Transformers Make Strong Encoders for Medical Image Segmentation

Automated Classification of Nanoparticles with Various Ultrastructures and Sizes

TESL-Net: A Transformer-Enhanced CNN for Accurate Skin Lesion Segmentation

EFCNet: Every Feature Counts for Small Medical Object Segmentation

MT_Net: A Multi-Scale Framework Using the Transformer Block for Retina Layer Segmentation

TSCA-Net: Transformer based spatial-channel attention segmentation network for medical images

TFCNs: A CNN-Transformer Hybrid Network for Medical Image Segmentation

EPT-Net: Edge Perception Transformer for 3D Medical Image Segmentation

TranSegNet: Hybrid CNN-Vision Transformers Encoder for Retina Segmentation of Optical Coherence Tomography