Abstract:Accurate segmentation of brain tumors from 3D multimodal MRI is vital for diagnosis and treatment planning across diverse brain tumors. This paper addresses the challenges posed by the BraTS 2023, presenting a unified transfer learning approach that applies to a broader spectrum of brain tumors. We introduce HT-CNNs, an ensemble of Hybrid Transformers and Convolutional Neural Networks optimized through transfer learning for varied brain tumor segmentation. This method captures spatial and contextual details from MRI data, fine-tuned on diverse datasets representing common tumor types. Through transfer learning, HT-CNNs utilize the learned representations from one task to improve generalization in another, harnessing the power of pre-trained models on large datasets and fine-tuning them on specific tumor types. We preprocess diverse datasets from multiple international distributions, ensuring representativeness for the most common brain tumors. Our rigorous evaluation employs standardized quantitative metrics across all tumor types, ensuring robustness and generalizability. The proposed ensemble model achieves superior segmentation results across the BraTS validation datasets over the previous winning methods. Comprehensive quantitative evaluations using the DSC and HD95 demonstrate the effectiveness of our approach. Qualitative segmentation predictions further validate the high-quality outputs produced by our model. Our findings underscore the potential of transfer learning and ensemble approaches in medical image segmentation, indicating a substantial enhancement in clinical decision-making and patient care. Despite facing challenges related to post-processing and domain gaps, our study sets a new precedent for future research for brain tumor segmentation. The docker image for the code and models has been made publicly available, <a class="link-external link-https" href="https://hub.docker.com/r/razeineldin/ht-cnns" rel="external noopener nofollow">this https URL</a>.

MRI Brain Tumor Segmentation and Uncertainty Estimation Using 3D-UNet Architectures

Brain Tumor Segmentation using 3D-CNNs with Uncertainty Estimation

Asymmetric Ensemble of Asymmetric U-Net Models for Brain Tumor Segmentation With Uncertainty Estimation

Multimodal MRI brain tumor segmentation using 3D attention UNet with dense encoder blocks and residual decoder blocks

Automatic Brain Tumor Segmentation Based on Cascaded Convolutional Neural Networks with Uncertainty Estimation

Brain Tumor Segmentation and survival prediction using 3D U-net

A Volumetric Convolutional Neural Network for Brain Tumor Segmentation

S3d-Unet: Separable 3d U-Net For Brain Tumor Segmentation

3D Dilated Multi-Fiber Network for Real-time Brain Tumor Segmentation in MRI

Brain Tumor Segmentation Using an Ensemble of 3D U-Nets and Overall Survival Prediction Using Radiomic Features

Glioma Segmentation with Cascaded Unet

TSEUnet: A 3D Neural Network with Fused Transformer and SE-Attention for Brain Tumor Segmentation

Automatic Brain Tumor Segmentation Using 3D Architecture Based on ROI Extraction

MRI tumor segmentation with densely connected 3D CNN

A 3D Spatially Weighted Network for Segmentation of Brain Tissue from MRI

Segmentation and Classification of Brain Tumor using 3D-UNet Deep Neural Networks

Brain tumour segmentation using cascaded 3D densely-connected U-net

Robust Automated Tumour Segmentation Network Using 3D Direction-Wise Convolution and Transformer

3D Deep Neural Network-Based Brain Tumor Segmentation Using Multimodality Magnetic Resonance Sequences.

Unified HT-CNNs Architecture: Transfer Learning for Segmenting Diverse Brain Tumors in MRI from Gliomas to Pediatric Tumors

Combined 3D CNN for Brain Tumor Segmentation