Abstract:Accurate segmentation of brain tumors from 3D multimodal MRI is vital for diagnosis and treatment planning across diverse brain tumors. This paper addresses the challenges posed by the BraTS 2023, presenting a unified transfer learning approach that applies to a broader spectrum of brain tumors. We introduce HT-CNNs, an ensemble of Hybrid Transformers and Convolutional Neural Networks optimized through transfer learning for varied brain tumor segmentation. This method captures spatial and contextual details from MRI data, fine-tuned on diverse datasets representing common tumor types. Through transfer learning, HT-CNNs utilize the learned representations from one task to improve generalization in another, harnessing the power of pre-trained models on large datasets and fine-tuning them on specific tumor types. We preprocess diverse datasets from multiple international distributions, ensuring representativeness for the most common brain tumors. Our rigorous evaluation employs standardized quantitative metrics across all tumor types, ensuring robustness and generalizability. The proposed ensemble model achieves superior segmentation results across the BraTS validation datasets over the previous winning methods. Comprehensive quantitative evaluations using the DSC and HD95 demonstrate the effectiveness of our approach. Qualitative segmentation predictions further validate the high-quality outputs produced by our model. Our findings underscore the potential of transfer learning and ensemble approaches in medical image segmentation, indicating a substantial enhancement in clinical decision-making and patient care. Despite facing challenges related to post-processing and domain gaps, our study sets a new precedent for future research for brain tumor segmentation. The docker image for the code and models has been made publicly available, <a class="link-external link-https" href="https://hub.docker.com/r/razeineldin/ht-cnns" rel="external noopener nofollow">this https URL</a>.

CKD-TransBTS: Clinical Knowledge-Driven Hybrid Transformer with Modality-Correlated Cross-Attention for Brain Tumor Segmentation

Mmformer: Multimodal Medical Transformer for Incomplete Multimodal Learning of Brain Tumor Segmentation

TransBTS: Multimodal Brain Tumor Segmentation Using Transformer

multiPI-TransBTS: A Multi-Path Learning Framework for Brain Tumor Image Segmentation Based on Multi-Physical Information

TransBTSV2: Towards Better and More Efficient Volumetric Segmentation of Medical Images

TransBTSV2: Wider Instead of Deeper Transformer for Medical Image Segmentation

BTSC-TNAS: A Neural Architecture Search-Based Transformer for Brain Tumor Segmentation and Classification.

TransSea: Hybrid CNN-Transformer with Semantic Awareness for 3D Brain Tumor Segmentation

Clinical Knowledge-Based Hybrid Swin Transformer for Brain Tumor Segmentation

TransSea: Hybrid CNN–Transformer With Semantic Awareness for 3-D Brain Tumor Segmentation

Unified HT-CNNs Architecture: Transfer Learning for Segmenting Diverse Brain Tumors in MRI from Gliomas to Pediatric Tumors

Automatic Brain Tumor Segmentation By Exploring The Multi-Modality Complementary Information And Cascaded 3d Lightweight Cnns

CSU-Net: A CNN-Transformer Parallel Network for Multimodal Brain Tumour Segmentation

Cascade Residual Multiscale Convolution and Mamba-Structured UNet for Advanced Brain Tumor Image Segmentation

A Multimodal Feature Distillation with CNN-Transformer Network for Brain Tumor Segmentation with Incomplete Modalities

Cross-modality deep feature learning for brain tumor segmentation

MMMViT: Multiscale multimodal vision transformer for brain tumor segmentation with missing modalities

OdCFU-Net for Brain Tumor Segmentation Based on MRI

Combining CNNs With Transformer for Multimodal 3D MRI Brain Tumor Segmentation With Self-Supervised Pretraining

Augmented Transformer network for MRI brain tumor segmentation

Adaptive Cascaded Transformer U-Net for MRI Brain Tumor Segmentation