Abstract:Accurate medical image segmentation of brain tumors is necessary for the diagnosing, monitoring, and treating disease. In recent years, with the gradual emergence of multi-sequence magnetic resonance imaging (MRI), multi-modal MRI diagnosis has played an increasingly important role in the early diagnosis of brain tumors by providing complementary information for a given lesion. Different MRI modalities vary significantly in context, as well as in coarse and fine information. As the manual identification of brain tumors is very complicated, it usually requires the lengthy consultation of multiple experts. The automatic segmentation of brain tumors from MRI images can thus greatly reduce the workload of doctors and buy more time for treating patients. In this paper, we propose a multi-modal brain tumor segmentation framework that adopts the hybrid fusion of modality-specific features using a self-supervised learning strategy. The algorithm is based on a fully convolutional neural network. Firstly, we propose a multi-input architecture that learns independent features from multi-modal data, and can be adapted to different numbers of multi-modal inputs. Compared with single-modal multi-channel networks, our model provides a better feature extractor for segmentation tasks, which learns cross-modal information from multi-modal data. Secondly, we propose a new feature fusion scheme, named hybrid attentional fusion. This scheme enables the network to learn the hybrid representation of multiple features and capture the correlation information between them through an attention mechanism. Unlike popular methods, such as feature map concatenation, this scheme focuses on the complementarity between multi-modal data, which can significantly improve the segmentation results of specific regions. Thirdly, we propose a self-supervised learning strategy for brain tumor segmentation tasks. Our experimental results demonstrate the effectiveness of the proposed model against other state-of-the-art multi-modal medical segmentation methods.

Soft Tissue Sarcoma Segmentation Network Based on Self-supervised Learning.

Soft Tissue Sarcoma Segmentation Network Based on Ensemble Learning.

Soft Tissue Sarcoma Co-Segmentation in Combined MRI and PET/CT Data

Segmentation and detection of soft tissue sarcomas based on mask regional convolutional neural network

Deep Learning-Based Image Segmentation on Multimodal Medical Imaging

Self-Supervised Multi-Modal Hybrid Fusion Network for Brain Tumor Segmentation.

Intelligent Segmentation Medical Assistance System for MRI Images of Osteosarcoma in Developing Countries

SSGNet: Selective Multi-Scale Receptive Field and Kernel Self-Attention Based on Group-Wise Modality for Brain Tumor Segmentation

Deep Multi-Modality Collaborative Learning for Distant Metastases Predication in PET-CT Soft-Tissue Sarcoma Studies.

Deep Learning-based Artificial Intelligence for Assisting Diagnosis, Assessment and Treatment in Soft Tissue Sarcomas

MSTT-199: MRI Dataset for Musculoskeletal Soft Tissue Tumor Segmentation

Self-supervised Multi-scale Multi-modal Graph Pool Transformer for Sellar Region Tumor Diagnosis

Multimodal Data-Driven Segmentation of Bone Metastasis Lesions in SPECT Bone Scans Using Deep Learning

Attention-guided multi-scale learning network for automatic prostate and tumor segmentation on MRI

Weakly supervised brain tumor segmentation via semantic affinity deep neural network

Medical image segmentation based on self-supervised hybrid fusion network

LSCS-Net: A lightweight skin cancer segmentation network with densely connected multi-rate atrous convolution

Predicting distant metastases in soft-tissue sarcomas from PET-CT scans using constrained hierarchical multi-modality feature learning

Medical Image Segmentation Based on Multi-Modal Convolutional Neural Network: Study on Image Fusion Schemes

MFSNet: A Multi Focus Segmentation Network for Skin Lesion Segmentation

Skin Lesion Segmentation Based on Multi-Scale Attention Convolutional Neural Network.