Abstract:Automatic brain tumor segmentation from multi-modality Magnetic Resonance Images (MRI) using deep learning methods plays an important role in assisting the diagnosis and treatment of brain tumor. However, previous methods mostly ignore the latent relationship among different modalities. In this work, we propose a novel end-to-end Modality-Pairing learning method for brain tumor segmentation. Paralleled branches are designed to exploit different modality features and a series of layer connections are utilized to capture complex relationships and abundant information among modalities. We also use a consistency loss to minimize the prediction variance between two branches. Besides, learning rate warmup strategy is adopted to solve the problem of the training instability and early over-fitting. Lastly, we use average ensemble of multiple models and some post-processing techniques to get final results. Our method is tested on the BraTS 2020 online testing dataset, obtaining promising segmentation performance, with average dice scores of 0.891, 0.842, 0.816 for the whole tumor, tumor core and enhancing tumor, respectively. We won the second place of the BraTS 2020 Challenge for the tumor segmentation task.

What problem does this paper attempt to address?

The paper attempts to address the problem of automatic brain tumor segmentation, specifically using deep learning methods for brain tumor segmentation from multi-modal magnetic resonance imaging (MRI). Specifically, the authors point out that existing methods mostly ignore the potential relationships between different modalities, which leads to insufficient segmentation performance. Therefore, they propose a new end-to-end modality pairing learning method aimed at improving the accuracy of brain tumor segmentation by better integrating information from different modalities. ### Main Issues: 1. **Limitations of existing methods**: Most existing brain tumor segmentation methods mainly focus on a single modality or simply merge multiple modalities, ignoring the complex relationships between different modalities. 2. **Effective utilization of multi-modal information**: How to effectively utilize the complementary information in multi-modal MRI data to improve segmentation accuracy and robustness. 3. **Training instability and overfitting**: How to address the issues of training instability and early overfitting during the training process. ### Solutions: - **Modality pairing network**: Designed parallel branches to extract features from different modalities and capture the complex relationships between modalities through layer connections. - **Consistency loss**: Introduced consistency loss to minimize the prediction differences between the two branches. - **Learning rate warm-up strategy**: Adopted a learning rate warm-up strategy to address training instability and early overfitting issues. - **Model ensemble and post-processing**: Used the average ensemble of multiple models and some post-processing techniques to obtain the final segmentation results. ### Experimental Results: - On the BraTS 2020 online test dataset, the method achieved satisfactory segmentation performance with average Dice scores of 0.891 for the whole tumor, 0.842 for the tumor core, and 0.816 for the enhancing tumor. - The method won second place in the tumor segmentation task of the BraTS 2020 challenge. ### Summary: The paper effectively addresses key issues in multi-modal MRI brain tumor segmentation by proposing a new modality pairing learning method, significantly improving segmentation accuracy and robustness.

Modality-Pairing Learning for Brain Tumor Segmentation

Mmformer: Multimodal Medical Transformer for Incomplete Multimodal Learning of Brain Tumor Segmentation

Learning multi-modal brain tumor segmentation from privileged semi-paired MRI images with curriculum disentanglement learning

Brain Tumor Segmentation via Multi-Modalities Interactive Feature Learning

DIGEST: Deeply supervIsed knowledGE tranSfer neTwork learning for brain tumor segmentation with incomplete multi-modal MRI scans

Cross-modality deep feature learning for brain tumor segmentation

A Deep Multi-Task Learning Framework for Brain Tumor Segmentation

Multi-modal Brain Tumor Segmentation via Missing Modality Synthesis and Modality-level Attention Fusion

Robust Multimodal Brain Tumor Segmentation via Feature Disentanglement and Gated Fusion

Multi -Modality Brain Tumor Segmentation Network Based on Collaborative Feature Fusion

Multimodality-Assisted Semi-Supervised Brain Tumor Segmentation in Nondominant Modality Based on Consistency Learning

Brain tumor segmentation based on the fusion of deep semantics and edge information in multimodal MRI

Enhancing Incomplete Multi-modal Brain Tumor Segmentation with Intra-modal Asymmetry and Inter-modal Dependency

Automatic Segmentation of MRI of Brain Tumor Using Deep Convolutional Network

Multi-modal brain tumor segmentation via disentangled representation learning and region-aware contrastive learning

Automatic Brain Tumor Segmentation By Exploring The Multi-Modality Complementary Information And Cascaded 3d Lightweight Cnns

Hierarchical and Global Modality Interaction for Brain Tumor Segmentation

Latent Correlation Representation Learning for Brain Tumor Segmentation with Missing MRI Modalities

Brain Tumor Segmentation Network Using Attention-based Fusion and Spatial Relationship Constraint

Feature-enhanced generation and multi-modality fusion based deep neural network for brain tumor segmentation with missing MR modalities

Pre-Post Interaction Learning for Brain Tumor Segmentation with Missing MRI Modalities.