Abstract:Gastrointestinal (GI) cancer is a malignancy affecting the digestive organs. During radiation therapy, the radiation oncologist must precisely aim the X-ray beam at the tumor while avoiding unaffected areas of the stomach and intestines. Consequently, accurate, automated GI image segmentation is urgently needed in clinical practice. While the fully convolutional network (FCN) and U-Net framework have shown impressive results in medical image segmentation, their ability to model long-range dependencies is constrained by the convolutional kernel's restricted receptive field. The transformer has a robust capacity for global modeling owing to its inherent global self-attention mechanism. The TransUnet model leverages the strengths of both the convolutional neural network (CNN) and transformer models through a hybrid CNN-transformer encoder. However, the concatenation of high- and low-level features in the decoder is ineffective in fusing global and local information. To overcome this limitation, we propose an innovative transformer-based medical image segmentation architecture called BiFTransNet, which introduces a BiFusion module into the decoder stage, enabling effective global and local feature fusion by enabling feature integration from various modules. Further, a multilevel loss (ML) strategy is introduced to oversee the learning process of each decoder layer and optimize the use of globally and locally fused contextual features at different scales. Our method achieved a Dice score of 89.51% and an intersection-over-union (IoU) score of 86.54% on the UW-Madison Gastrointestinal Segmentation dataset. Moreover, our method attained a Dice score of 78.77% and a Hausdorff distance (HD) of 27.94% on the Synapse Multi-organ Segmentation dataset. Compared with the state-of-the-art methods, our proposed method achieves superior segmentation performance in gastrointestinal segmentation tasks. More significantly, our method can be easily extended to medical segmentation in different modalities such as CT and MRI. Our method achieves clinical multimodal medical segmentation and provides decision supports for clinical radiotherapy plans.

FCTrans UNet: A Hybrid CNN and Transformer Model for Medical Image Segmentations

Mixed Transformer U-Net for Medical Image Segmentation

TransUNet: Transformers Make Strong Encoders for Medical Image Segmentation

MTC-TransUNet: A Multi-Scale Mixed Convolution TransUNet for Medical Image Segmentation

Sfe-Transunet: A Transformer-Based U-Net With Skipped Features Enhancer For Medical Image Segmentation

A novel full-convolution UNet-transformer for medical image segmentation

CFATransUnet: Channel-wise cross fusion attention and transformer for 2D medical image segmentation

FTUNet: A Feature-Enhanced Network for Medical Image Segmentation Based on the Combination of U-Shaped Network and Vision Transformer

UCTNet: Uncertainty-guided CNN-Transformer hybrid networks for medical image segmentation

BiFTransNet: A unified and simultaneous segmentation network for gastrointestinal images of CT & MRI

Context-aware and local-aware fusion with transformer for medical image segmentation

HCT-Unet: multi-target medical image segmentation via a hybrid CNN-transformer Unet incorporating multi-axis gated multi-layer perceptron

FCSU-Net: A novel full-scale Cross-dimension Self-attention U-Net with collaborative fusion of multi-scale feature for medical image segmentation

TransUNet: Rethinking the U-Net architecture design for medical image segmentation through the lens of transformers

3D TransUNet: Advancing Medical Image Segmentation through Vision Transformers

MSCT-UNET: multi-scale contrastive transformer within U-shaped network for medical image segmentation

TransFuse: Fusing Transformers and CNNs for Medical Image Segmentation

HmsU-Net: A hybrid multi-scale U-net based on a CNN and transformer for medical image segmentation

SeUNet-Trans: A Simple yet Effective UNet-Transformer Model for Medical Image Segmentation

MFH‐Net: A Hybrid CNN‐Transformer Network Based Multi‐Scale Fusion for Medical Image Segmentation

P-TransUNet: an improved parallel network for medical image segmentation