Abstract:Purpose: Deep learning-based networks have become increasingly popular in the field of medical image segmentation. The purpose of this research was to develop and optimize a new architecture for automatic segmentation of the prostate gland and normal organs in the pelvic, thoracic, and upper gastro-intestinal (GI) regions. Methods: We developed an architecture which combines a shifted-window (Swin) transformer with a convolutional U-Net. The network includes a parallel encoder, a cross-fusion block, and a CNN-based decoder to extract local and global information and merge related features on the same scale. A skip connection is applied between the cross-fusion block and decoder to integrate low-level semantic features. Attention gates (AGs) are integrated within the CNN to suppress features in image background regions. Our network is termed "SwinAttUNet." We optimized the architecture for automatic image segmentation. Training datasets consisted of planning-CT datasets from 300 prostate cancer patients from an institutional database and 100 CT datasets from a publicly available dataset (CT-ORG). Images were linearly interpolated and resampled to a spatial resolution of (1.0 × 1.0× 1.5) mm3 . A volume patch (192 × 192 × 96) was used for training and inference, and the dataset was split into training (75%), validation (10%), and test (15%) cohorts. Data augmentation transforms were applied consisting of random flip, rotation, and intensity scaling. The loss function comprised Dice and cross-entropy equally weighted and summed. We evaluated Dice coefficients (DSC), 95th percentile Hausdorff Distances (HD95), and Average Surface Distances (ASD) between results of our network and ground truth data. Results: SwinAttUNet, DSC values were 86.54 ± 1.21, 94.15 ± 1.17, and 87.15 ± 1.68% and HD95 values were 5.06 ± 1.42, 3.16 ± 0.93, and 5.54 ± 1.63 mm for the prostate, bladder, and rectum, respectively. Respective ASD values were 1.45 ± 0.57, 0.82 ± 0.12, and 1.42 ± 0.38 mm. For the lung, liver, kidneys and pelvic bones, respective DSC values were: 97.90 ± 0.80, 96.16 ± 0.76, 93.74 ± 2.25, and 89.31 ± 3.87%. Respective HD95 values were: 5.13 ± 4.11, 2.73 ± 1.19, 2.29 ± 1.47, and 5.31 ± 1.25 mm. Respective ASD values were: 1.88 ± 1.45, 1.78 ± 1.21, 0.71 ± 0.43, and 1.21 ± 1.11 mm. Our network outperformed several existing deep learning approaches using only attention-based convolutional or Transformer-based feature strategies, as detailed in the results section. Conclusions: We have demonstrated that our new architecture combining Transformer- and convolution-based features is able to better learn the local and global context for automatic segmentation of multi-organ, CT-based anatomy.

Cross-convolutional transformer for automated multi-organs segmentation in a variety of medical images

Dual encoder network with transformer-CNN for multi-organ segmentation

CCTrans: Improving Medical Image Segmentation with Contoured Convolutional Transformer Network

Cross Attention Multi Scale CNN-Transformer Hybrid Encoder is General Medical Image Learner.

Abdominal multi-organ segmentation in CT using Swinunter

MSCT-UNET: multi-scale contrastive transformer within U-shaped network for medical image segmentation

3D Medical image segmentation using parallel transformers

A new architecture combining convolutional and transformer-based networks for automatic 3D multi-organ segmentation on CT images

HCT-Unet: multi-target medical image segmentation via a hybrid CNN-transformer Unet incorporating multi-axis gated multi-layer perceptron

HCA-former: Hybrid Convolution Attention Transformer for 3D Medical Image Segmentation

MultiTrans: Multi-branch transformer network for medical image segmentation

A Parallelly Contextual Convolutional Transformer for Medical Image Segmentation

A Global-Local Features Exchange and Fusion Network for Multi-Organ Segmentation.

Male Pelvic Multi-Organ Segmentation Using Token-Based Transformer Vnet.

Male Pelvic Multi-Organ Segmentation Using V-transformer Network.

Multi-Scale Convolution-Transformer Fusion Network for Endoscopic Image Segmentation.

MultiTrans: Multi-scale Feature Fusion Transformer with Transfer Learning Strategy for Multiple Organs Segmentation of Head and Neck CT Images

CTC-Net: A Novel Coupled Feature-Enhanced Transformer and Inverted Convolution Network for Medical Image Segmentation

Multi-organ segmentation network for abdominal CT images based on spatial attention and deformable convolution

The auto segmentation for cardiac structures using a dual-input deep learning network based on vision saliency and transformer

HybridCTrm: Bridging CNN and Transformer for Multimodal Brain Image Segmentation