Abstract:Vision Transformer (ViT) has emerged as a potential alternative to convolutional neural networks for large datasets. However, applying ViT directly to medical image segmentation is challenging due to its lack of induction bias, which requires a large number of high-quality annotated medical images for effective model training. Recent studies have discovered that, in addition to the increased model capacity and generalization resulting from the lack of induction bias, the excellent performance of Transformer can also be attributed to its large receptive field. In this paper, we propose a U-shaped medical image segmentation model that combines large kernel convolutions with Transformers. Specifically, we construct a basic Transformer unit using pyramidal convolution modules with multi-scale kernels and multi-layer perceptron. In the pyramid convolution module, we employ grouped convolution to reduce parameter and computational complexity while utilizing multi-scale large kernel attention as a foundation for more efficient feature extraction. For different types of grouping, different sizes of convolutions are used to enhance the extraction of features with multiple receptive fields. To optimize the extracted features from the encoder, the U-shaped model integrates a variant of the pyramidal convolutional module into the skip connections. This variant utilizes multi-scale large kernel convolutional attention based on channel splitting. The incorporation of this variant enables efficient refinement of the feature representations within the skip connections. Through extensive comparisons on multi-modal medical image datasets, our model outperforms state-of-the-art methods across various evaluation metrics, with notable superiority observed on small-scale medical datasets. Our research findings suggest that the combination of large kernel convolutions and Transformer models introduces an advantageous inductive bias, resulting in enhanced performance specifically for small-scale medical image datasets. To facilitate accessibility, we have made our code openly accessible on our GitHub repository, which can be found at https://github.com/medical-images-process/CNN-Transformer .

Advantages of transformer and its application for medical image segmentation: a survey

Transformers in medical image segmentation: a narrative review

Transformers in medical image segmentation: A review

Recent Progress in Transformer-based Medical Image Analysis

Transformers in medical imaging: A survey

Transforming medical imaging with Transformers? A comparative review of key properties, current progresses, and future perspectives

MBUTransNet: multi-branch U-shaped network fusion transformer architecture for medical image segmentation

From CNN to Transformer: A Review of Medical Image Segmentation Models

Transformers in Medical Image Analysis

Pyramid Medical Transformer for Medical Image Segmentation

TransUNet: Transformers Make Strong Encoders for Medical Image Segmentation

CTC-Net: A Novel Coupled Feature-Enhanced Transformer and Inverted Convolution Network for Medical Image Segmentation

Hybrid CNN-Transformer model for medical image segmentation with pyramid convolution and multi-layer perceptron

Optimization of U-shaped pure transformer medical image segmentation network

Transformer-Based Visual Segmentation: A Survey

TSCA-Net: Transformer based spatial-channel attention segmentation network for medical images

Context-aware and local-aware fusion with transformer for medical image segmentation

Slimmable transformer with hybrid axial-attention for medical image segmentation

X-Net: a dual encoding–decoding method in medical image segmentation

ConvFormer: Plug-and-Play CNN-Style Transformers for Improving Medical Image Segmentation

Review of Research on Application of Vision Transformer in Medical Image Analysis