Abstract:Accurate medical image segmentation of brain tumors is necessary for the diagnosing, monitoring, and treating disease. In recent years, with the gradual emergence of multi-sequence magnetic resonance imaging (MRI), multi-modal MRI diagnosis has played an increasingly important role in the early diagnosis of brain tumors by providing complementary information for a given lesion. Different MRI modalities vary significantly in context, as well as in coarse and fine information. As the manual identification of brain tumors is very complicated, it usually requires the lengthy consultation of multiple experts. The automatic segmentation of brain tumors from MRI images can thus greatly reduce the workload of doctors and buy more time for treating patients. In this paper, we propose a multi-modal brain tumor segmentation framework that adopts the hybrid fusion of modality-specific features using a self-supervised learning strategy. The algorithm is based on a fully convolutional neural network. Firstly, we propose a multi-input architecture that learns independent features from multi-modal data, and can be adapted to different numbers of multi-modal inputs. Compared with single-modal multi-channel networks, our model provides a better feature extractor for segmentation tasks, which learns cross-modal information from multi-modal data. Secondly, we propose a new feature fusion scheme, named hybrid attentional fusion. This scheme enables the network to learn the hybrid representation of multiple features and capture the correlation information between them through an attention mechanism. Unlike popular methods, such as feature map concatenation, this scheme focuses on the complementarity between multi-modal data, which can significantly improve the segmentation results of specific regions. Thirdly, we propose a self-supervised learning strategy for brain tumor segmentation tasks. Our experimental results demonstrate the effectiveness of the proposed model against other state-of-the-art multi-modal medical segmentation methods.

A 3D Framework for Brain Tumor Segmentation from Multi-modal MR Images.

A Deep Multi-Task Learning Framework for Brain Tumor Segmentation

A Framework of Fuzzy Information Fusion for the Segmentation of Brain Tumor Tissues on MR Images

A Framework of Automatic Brain Tumor Segmentation Method Based on Information Fusion of Structural and Functional MRI Signals

Brain Tumor Segmentation via Multi-Modalities Interactive Feature Learning

Brain Tumor Segmentation in Multimodal MRI Via Pixel-Level and Feature-Level Image Fusion.

A Multi-View Dynamic Fusion Framework: How to Improve the Multimodal Brain Tumor Segmentation from Multi-Views?

3D Deep Neural Network-Based Brain Tumor Segmentation Using Multimodality Magnetic Resonance Sequences.

Prototype-Driven and Multi-Expert Integrated Multi-Modal MR Brain Tumor Image Segmentation

Brain tumor segmentation based on the fusion of deep semantics and edge information in multimodal MRI

Brain Tumor Segmentation Network Using Attention-based Fusion and Spatial Relationship Constraint

Multi-modal Brain Tumor Segmentation via Missing Modality Synthesis and Modality-level Attention Fusion

Brain Multi-Parametric MRI Tumor Subregion Segmentation Via Hierarchical Substructural Activation Network.

A Memory-efficient Deep Framework for Multi-Modal MRI-based Brain Tumor Segmentation

A Two-Stage Coarse-to-Fine Brain Tumor Segmentation Framework

Multi -Modality Brain Tumor Segmentation Network Based on Collaborative Feature Fusion

Brain Tumor Synthetic Segmentation in 3D Multimodal MRI Scans

A Novel Brain Tumor Segmentation from Multi-Modality MRI via A Level-Set-Based Model

Self-Supervised Multi-Modal Hybrid Fusion Network for Brain Tumor Segmentation.

Brain Tumor Segmentation through Data Fusion of T2-Weighted Image and MR Spectroscopy

An Improved Multi-Modal Brain Tumor Segmentation Hybrid Algorithm