Abstract:Purpose: Deep learning-based networks have become increasingly popular in the field of medical image segmentation. The purpose of this research was to develop and optimize a new architecture for automatic segmentation of the prostate gland and normal organs in the pelvic, thoracic, and upper gastro-intestinal (GI) regions. Methods: We developed an architecture which combines a shifted-window (Swin) transformer with a convolutional U-Net. The network includes a parallel encoder, a cross-fusion block, and a CNN-based decoder to extract local and global information and merge related features on the same scale. A skip connection is applied between the cross-fusion block and decoder to integrate low-level semantic features. Attention gates (AGs) are integrated within the CNN to suppress features in image background regions. Our network is termed "SwinAttUNet." We optimized the architecture for automatic image segmentation. Training datasets consisted of planning-CT datasets from 300 prostate cancer patients from an institutional database and 100 CT datasets from a publicly available dataset (CT-ORG). Images were linearly interpolated and resampled to a spatial resolution of (1.0 × 1.0× 1.5) mm3 . A volume patch (192 × 192 × 96) was used for training and inference, and the dataset was split into training (75%), validation (10%), and test (15%) cohorts. Data augmentation transforms were applied consisting of random flip, rotation, and intensity scaling. The loss function comprised Dice and cross-entropy equally weighted and summed. We evaluated Dice coefficients (DSC), 95th percentile Hausdorff Distances (HD95), and Average Surface Distances (ASD) between results of our network and ground truth data. Results: SwinAttUNet, DSC values were 86.54 ± 1.21, 94.15 ± 1.17, and 87.15 ± 1.68% and HD95 values were 5.06 ± 1.42, 3.16 ± 0.93, and 5.54 ± 1.63 mm for the prostate, bladder, and rectum, respectively. Respective ASD values were 1.45 ± 0.57, 0.82 ± 0.12, and 1.42 ± 0.38 mm. For the lung, liver, kidneys and pelvic bones, respective DSC values were: 97.90 ± 0.80, 96.16 ± 0.76, 93.74 ± 2.25, and 89.31 ± 3.87%. Respective HD95 values were: 5.13 ± 4.11, 2.73 ± 1.19, 2.29 ± 1.47, and 5.31 ± 1.25 mm. Respective ASD values were: 1.88 ± 1.45, 1.78 ± 1.21, 0.71 ± 0.43, and 1.21 ± 1.11 mm. Our network outperformed several existing deep learning approaches using only attention-based convolutional or Transformer-based feature strategies, as detailed in the results section. Conclusions: We have demonstrated that our new architecture combining Transformer- and convolution-based features is able to better learn the local and global context for automatic segmentation of multi-organ, CT-based anatomy.

Technical note: The effect of image annotation with minimal manual interaction for semiautomatic prostate segmentation in CT images using fully convolutional neural networks

Automated Prostate Segmentation of Volumetric CT Images Using 3D Deeply Supervised Dilated FCN.

Segmentation of the prostate and organs at risk in male pelvic CT images using deep learning

A multi-object deep neural network architecture to detect prostate anatomy in T2-weighted MRI: Performance evaluation

Effect of dataset size, image quality, and image type on deep learning-based automatic prostate segmentation in 3D ultrasound

Iterative Label Denoising Network: Segmenting Male Pelvic Organs in CT From 3D Bounding Box Annotations

Deeply Supervised 3D Fully Convolutional Networks with Group Dilated Convolution for Automatic MRI Prostate Segmentation

Automatic Segmentation of the Prostate on CT Images Using Deep Neural Networks (DNN).

Automatic CT Segmentation from Bounding Box Annotations using Convolutional Neural Networks

CDA-Net: A Contrastive Deep Adversarial Model for Prostate Cancer Segmentation in MRI Images.

Fully automatic segmentation on prostate MR images based on cascaded fully convolution network.

A Novel Registration-Based Algorithm for Prostate Segmentation Via the Combination of SSM and CNN.

Collaborative Learning for Annotation-Efficient Volumetric MR Image Segmentation

USE-Net: incorporating Squeeze-and-Excitation blocks into U-Net for prostate zonal segmentation of multi-institutional MRI datasets

A new architecture combining convolutional and transformer-based networks for automatic 3D multi-organ segmentation on CT images

Attention-guided multi-scale learning network for automatic prostate and tumor segmentation on MRI

Automatic MRI Prostate Segmentation Using 3D Deeply Supervised FCN with Concatenated Atrous Convolution

Automatic Segmentation of the Prostate on MR Images Based on Anatomy and Deep Learning.

Fast interactive medical image segmentation with weakly supervised deep learning method

PSNet: prostate segmentation on MRI based on a convolutional neural network.

A dual attention-guided 3D convolution network for automatic segmentation of prostate and tumor