Abstract:Background Bi‐parametric magnetic resonance imaging (bpMRI) has demonstrated promising results in prostate cancer (PCa) detection. Vision transformers have achieved competitive performance compared to convolutional neural network (CNN) in deep learning, but they need abundant annotated data for training. Self‐supervised learning can effectively leverage unlabeled data to extract useful semantic representations without annotation and its associated costs. Purpose This study proposes a novel self‐supervised learning framework and a transformer model to enhance PCa detection using prostate bpMRI. Methods and materials We introduce a novel end‐to‐end Cross‐Shaped windows (CSwin) transformer UNet model, CSwin UNet, to detect clinically significant prostate cancer (csPCa) in prostate bpMRI. We also propose a multitask self‐supervised learning framework to leverage unlabeled data and improve network generalizability. Using a large prostate bpMRI dataset (PI‐CAI) with 1476 patients, we first pretrain CSwin transformer using multitask self‐supervised learning to improve data‐efficiency and network generalizability. We then finetune using lesion annotations to perform csPCa detection. We also test the network generalization using a separate bpMRI dataset with 158 patients (Prostate158). Results Five‐fold cross validation shows that self‐supervised CSwin UNet achieves 0.888 ± 0.010 aread under receiver operating characterstics curve (AUC) and 0.545 ± 0.060 Average Precision (AP) on PI‐CAI dataset, significantly outperforming four comparable models (nnFormer, Swin UNETR, DynUNet, Attention UNet, UNet). On model generalizability, self‐supervised CSwin UNet achieves 0.79 AUC and 0.45 AP, still outperforming all other comparable methods and demonstrating good generalization to external data. Conclusions This study proposes CSwin UNet, a new transformer‐based model for end‐to‐end detection of csPCa, enhanced by self‐supervised pretraining to enhance network generalizability. We employ an automatic weighted loss (AWL) to unify pretext tasks, improving representation learning. Evaluated on two multi‐institutional public datasets, our method surpasses existing methods in detection metrics and demonstrates good generalization to external data.

Shifted window-based Transformer with multimodal representation for the systematic staging of rectal cancer

A Transformer-Based Model for Preoperative Early Recurrence Prediction of Hepatocellular Carcinoma with Muti-modality MRI

Mmformer: Multimodal Medical Transformer for Incomplete Multimodal Learning of Brain Tumor Segmentation

FCTformer: Fusing Convolutional Operations and Transformer for 3D Rectal Tumor Segmentation in MR Images

Multi-scale Window Transformer for Cervical Cytopathology Image Recognition

Multi-dimension unified Swin Transformer for 3D Lesion Segmentation in Multiple Anatomical Locations

Cross‐shaped windows transformer with self‐supervised pretraining for clinically significant prostate cancer detection in bi‐parametric MRI

Transformer based multiple instance learning for WSI breast cancer classification

SwinHR: Hemodynamic-powered hierarchical vision transformer for breast tumor segmentation

Hierarchical Transformer for Survival Prediction Using Multimodality Whole Slide Images and Genomics

Swin transformers are robust to distribution and concept drift in endoscopy-based longitudinal rectal cancer assessment

Axial Attention Transformer Networks: A New Frontier in Breast Cancer Detection

EP093/#664 Automatic multimodal classification using transformer for cervical cancer

Automatic segmentation of rectal tumors from MRI using multiscale densely connected convolutional neural network based on attention mechanism

Transformers for colorectal cancer segmentation in CT imaging

Colorectal cancer lymph node metastasis prediction with weakly supervised transformer-based multi-instance learning

A Multi-Task Transformer with Local-Global Feature Interaction and Multiple Tumoral Region Guidance for Breast Cancer Diagnosis

iPCa-Former: A Multi-Task Transformer Framework for Perceiving Incidental Prostate Cancer

Efficient Lung Cancer Image Classification and Segmentation Algorithm Based on Improved Swin Transformer

Deep learning models for preoperative T-stage assessment in rectal cancer using MRI: exploring the impact of rectal filling

A LLM-Based Hybrid-Transformer Diagnosis System in Healthcare