Abstract:Gastrointestinal (GI) cancer is a malignancy affecting the digestive organs. During radiation therapy, the radiation oncologist must precisely aim the X-ray beam at the tumor while avoiding unaffected areas of the stomach and intestines. Consequently, accurate, automated GI image segmentation is urgently needed in clinical practice. While the fully convolutional network (FCN) and U-Net framework have shown impressive results in medical image segmentation, their ability to model long-range dependencies is constrained by the convolutional kernel's restricted receptive field. The transformer has a robust capacity for global modeling owing to its inherent global self-attention mechanism. The TransUnet model leverages the strengths of both the convolutional neural network (CNN) and transformer models through a hybrid CNN-transformer encoder. However, the concatenation of high- and low-level features in the decoder is ineffective in fusing global and local information. To overcome this limitation, we propose an innovative transformer-based medical image segmentation architecture called BiFTransNet, which introduces a BiFusion module into the decoder stage, enabling effective global and local feature fusion by enabling feature integration from various modules. Further, a multilevel loss (ML) strategy is introduced to oversee the learning process of each decoder layer and optimize the use of globally and locally fused contextual features at different scales. Our method achieved a Dice score of 89.51% and an intersection-over-union (IoU) score of 86.54% on the UW-Madison Gastrointestinal Segmentation dataset. Moreover, our method attained a Dice score of 78.77% and a Hausdorff distance (HD) of 27.94% on the Synapse Multi-organ Segmentation dataset. Compared with the state-of-the-art methods, our proposed method achieves superior segmentation performance in gastrointestinal segmentation tasks. More significantly, our method can be easily extended to medical segmentation in different modalities such as CT and MRI. Our method achieves clinical multimodal medical segmentation and provides decision supports for clinical radiotherapy plans.

Twist-Net: A multi-modality transfer learning network with the hybrid bilateral encoder for hypopharyngeal cancer segmentation

A multi-branched semantic segmentation network based on twisted information sharing pattern for medical images

HTC-Net: A hybrid CNN-transformer framework for medical image segmentation

A multi-path adaptive fusion network for multimodal brain tumor segmentation

H-DenseFormer: An Efficient Hybrid Densely Connected Transformer for Multimodal Tumor Segmentation

HCT-Unet: multi-target medical image segmentation via a hybrid CNN-transformer Unet incorporating multi-axis gated multi-layer perceptron

A 2.5D multi-path fusion network framework with focusing on z-axis 3D joint for medical image segmentation

A hybrid network for automatic hepatocellular carcinoma segmentation in H&E-stained whole slide images

HCTNet: A hybrid CNN-transformer network for breast ultrasound image segmentation

Dual-branch hybrid network for lesion segmentation in gastric cancer images

HTTU-Net: Hybrid Two Track U-Net for Automatic Brain Tumor Segmentation

3D Reconstruction-Oriented Fully Automatic Multi-Modal Tumor Segmentation by Dual Attention-Guided VNet

Head and neck tumor segmentation convolutional neural network robust to missing PET/CT modalities using channel dropout

HBNet: Hybrid Blocks Network for Segmentation of Gastric Tumor from Ordinary CT Images

MultiTrans: Multi-scale Feature Fusion Transformer with Transfer Learning Strategy for Multiple Organs Segmentation of Head and Neck CT Images

BiFTransNet: A unified and simultaneous segmentation network for gastrointestinal images of CT & MRI

Robust Multimodal Brain Tumor Segmentation via Feature Disentanglement and Gated Fusion

S2DA-Net: Spatial and Spectral-Learning Double-Branch Aggregation Network for Liver Tumor Segmentation in CT Images

MFMSNet: A Multi-frequency and Multi-scale Interactive CNN-Transformer Hybrid Network for breast ultrasound image segmentation

MFHARFNet: Multi-branch feature hybrid and adaptive receptive field network for image segmentation

A 3D boundary-guided hybrid network with convolutions and Transformers for lung tumor segmentation in CT images