Abstract:Purpose: Deep learning-based networks have become increasingly popular in the field of medical image segmentation. The purpose of this research was to develop and optimize a new architecture for automatic segmentation of the prostate gland and normal organs in the pelvic, thoracic, and upper gastro-intestinal (GI) regions. Methods: We developed an architecture which combines a shifted-window (Swin) transformer with a convolutional U-Net. The network includes a parallel encoder, a cross-fusion block, and a CNN-based decoder to extract local and global information and merge related features on the same scale. A skip connection is applied between the cross-fusion block and decoder to integrate low-level semantic features. Attention gates (AGs) are integrated within the CNN to suppress features in image background regions. Our network is termed "SwinAttUNet." We optimized the architecture for automatic image segmentation. Training datasets consisted of planning-CT datasets from 300 prostate cancer patients from an institutional database and 100 CT datasets from a publicly available dataset (CT-ORG). Images were linearly interpolated and resampled to a spatial resolution of (1.0 × 1.0× 1.5) mm3 . A volume patch (192 × 192 × 96) was used for training and inference, and the dataset was split into training (75%), validation (10%), and test (15%) cohorts. Data augmentation transforms were applied consisting of random flip, rotation, and intensity scaling. The loss function comprised Dice and cross-entropy equally weighted and summed. We evaluated Dice coefficients (DSC), 95th percentile Hausdorff Distances (HD95), and Average Surface Distances (ASD) between results of our network and ground truth data. Results: SwinAttUNet, DSC values were 86.54 ± 1.21, 94.15 ± 1.17, and 87.15 ± 1.68% and HD95 values were 5.06 ± 1.42, 3.16 ± 0.93, and 5.54 ± 1.63 mm for the prostate, bladder, and rectum, respectively. Respective ASD values were 1.45 ± 0.57, 0.82 ± 0.12, and 1.42 ± 0.38 mm. For the lung, liver, kidneys and pelvic bones, respective DSC values were: 97.90 ± 0.80, 96.16 ± 0.76, 93.74 ± 2.25, and 89.31 ± 3.87%. Respective HD95 values were: 5.13 ± 4.11, 2.73 ± 1.19, 2.29 ± 1.47, and 5.31 ± 1.25 mm. Respective ASD values were: 1.88 ± 1.45, 1.78 ± 1.21, 0.71 ± 0.43, and 1.21 ± 1.11 mm. Our network outperformed several existing deep learning approaches using only attention-based convolutional or Transformer-based feature strategies, as detailed in the results section. Conclusions: We have demonstrated that our new architecture combining Transformer- and convolution-based features is able to better learn the local and global context for automatic segmentation of multi-organ, CT-based anatomy.

A Multi-Center Study of Ultrasound Images using a Fully Automated Segmentation Architecture

Multi-Organ Foundation Model for Universal Ultrasound Image Segmentation with Task Prompt and Anatomical Prior

Intelligent contour extraction approach for accurate segmentation of medical ultrasound images

A Robust and Explainable Structure-Based Algorithm for Detecting the Organ Boundary from Ultrasound Multi-Datasets

Precise segmentation of multiple organs in CT volumes using learning-based approach and information theory.

Organ Boundary Delineation for Automated Diagnosis from Multi-Center Using Ultrasound Images

Hybrid Intelligent-Annotation Organ Segmentation on Medical Datasets

Multi-organ Segmentation in Pelvic CT Images with CT-based Synthetic MRI

Ultrasound Prostate Segmentation Using Adaptive Selection Principal Curve and Smooth Mathematical Model

A new architecture combining convolutional and transformer-based networks for automatic 3D multi-organ segmentation on CT images

Coarse-to-fine Tuning Knowledgeable System for Boundary Delineation in Medical Images

Segmentation of the prostate and organs at risk in male pelvic CT images using deep learning

Automatic Segmentation of the Prostate on CT Images Using Deep Learning and Multi-Atlas Fusion

Multi-organ segmentation network for abdominal CT images based on spatial attention and deformable convolution

Automatic Abdominal Multi-Organ Segmentation Using Deep Convolutional Neural Network and Time-Implicit Level Sets

Multi-stage Fully Convolutional Network for Precise Prostate Segmentation in Ultrasound Images

Simultaneous Segmentation of Fetal Hearts and Lungs for Medical Ultrasound Images Via an Efficient Multi-scale Model Integrated with Attention Mechanism

Robust and Efficient Abdominal CT Segmentation Using Shape Constrained Multi-Scale Attention Network

Dual Knowledge‐guided Two‐stage Model for Precise Small Organ Segmentation in Abdominal CT Images

Male Pelvic Multi-Organ Segmentation on Transrectal Ultrasound Using Anchor-free Mask CNN.

Daily Cone-Beam CT Multi-Organ Segmentation for Prostate Adaptive Radiotherapy.