Abstract:Background: Deep learning methods have shown great potential in processing multi-modal Magnetic Resonance Imaging (MRI) data, enabling improved accuracy in brain tumor segmentation. However, the performance of these methods can suffer when dealing with incomplete modalities, which is a common issue in clinical practice. Existing solutions, such as missing modality synthesis, knowledge distillation, and architecture-based methods, suffer from drawbacks such as long training times, high model complexity, and poor scalability. Method: This paper proposes IMS 2 Trans, a novel lightweight scalable Swin Transformer network by utilizing a single encoder to extract latent feature maps from all available modalities. This unified feature extraction process enables efficient information sharing and fusion among the modalities, resulting in efficiency without compromising segmentation performance even in the presence of missing modalities. Results: Two datasets, BraTS 2018 and BraTS 2020, containing incomplete modalities for brain tumor segmentation are evaluated against popular benchmarks. On the BraTS 2018 dataset, our model achieved higher average Dice similarity coefficient (DSC) scores for the whole tumor, tumor core, and enhancing tumor regions (86.57, 75.67, and 58.28, respectively), in comparison with a state-of-the-art model, i.e. mmFormer (86.45, 75.51, and 57.79, respectively). Similarly, on the BraTS 2020 dataset, our model scored higher DSC scores in these three brain tumor regions (87.33, 79.09, and 62.11, respectively) compared to mmFormer (86.17, 78.34, and 60.36, respectively). We also conducted a Wilcoxon test on the experimental results, and the generated p -value confirmed that our model's performance was statistically significant. Moreover, our model exhibits significantly reduced complexity with only 4.47 M parameters, 121.89G FLOPs, and a model size of 77.13 MB, whereas mmFormer comprises 34.96 M parameters, 265.79 G FLOPs, and a model size of 559.74 MB. These indicate our model, being light-weighted with significantly reduced parameters, is still able to achieve better performance than a state-of-the-art model. Conclusion: By leveraging a single encoder for processing the available modalities, IMS 2 Trans offers notable scalability advantages over methods that rely on multiple encoders. This streamlined approach eliminates the need for maintaining separate encoders for each modality, resulting in a lightweight and scalable network architecture. The source code of IMS 2 Trans and the associated weights are both publicly available at https://github.com/hudscomdz/IMS2Trans .

Advancing brain tumor detection: harnessing the Swin Transformer’s power for accurate classification and performance analysis

A novel Swin transformer approach utilizing residual multi-layer perceptron for diagnosing brain tumors in MRI images

Brain Tumor Detection using Swin Transformers

Creating a deep learning model using a Swin Transformer and tree growth optimisation to classify brain tumour

Clinical Knowledge-Based Hybrid Swin Transformer for Brain Tumor Segmentation

SwinBTS: A Method for 3D Multimodal Brain Tumor Segmentation Using Swin Transformer

Swin UNETR: Swin Transformers for Semantic Segmentation of Brain Tumors in MRI Images

Tumor Localization and Classification from MRI of Brain using Deep Convolution Neural Network and Salp Swarm Algorithm

Exploring the Power of Deep Learning: Fine-Tuned Vision Transformer for Accurate and Efficient Brain Tumor Detection in MRI Scans

A fine-tuned transformer model for brain tumor detection and classification

Combining the Transformer and Convolution for Effective Brain Tumor Classification Using MRI Images

SwinCNN: An Integrated Swin Transformer and CNN for Improved Breast Cancer Grade Classification

SwinHR: Hemodynamic-powered hierarchical vision transformer for breast tumor segmentation

Scalable Swin Transformer network for brain tumor segmentation from incomplete MRI modalities

Enhancing brain tumor MRI classification with an ensemble of deep learning models and transformer integration

Efficient Lung Cancer Image Classification and Segmentation Algorithm Based on an Improved Swin Transformer

Enhancing Skin Cancer Diagnosis Using Swin Transformer with Hybrid Shifted Window-Based Multi-head Self-attention and SwiGLU-Based MLP

Detection and Classification of Lung Cancer Cells Using Swin Transformer

Refining neural network algorithms for accurate brain tumor classification in MRI imagery

Efficient Lung Cancer Image Classification and Segmentation Algorithm Based on Improved Swin Transformer

Multi-Level Swin Transformer Enabled Automatic Segmentation and Classification of Breast Metastases