Abstract:Multimodal magnetic resonance imaging (MRI) provides complementary information about targets, and the segmentation of multimodal MRI is widely used as an essential preprocessing step for initial diagnosis, stage differentiation, and post-treatment efficacy evaluation in clinical situations. For the main modality or each of the modalities, it is important to enhance the visual information by modeling the connection and effectively fusing the features among them. However, the existing methods for multimodal segmentation have a drawback; they coincidentally drop information of individual modality during the fusion process. Recently, graph learning-based methods have been applied in segmentation, and these methods have achieved considerable improvements by modeling the relationships across feature regions and reasoning using global information. In this paper, we propose a graph learning-based approach to efficiently extract modality-specific features and establish regional correspondence effectively among all modalities. In detail, after projecting features into a graph domain and employing graph convolution to propagate information across all regions for learning global modality-specific features, we propose a mutual information-based graph co-attention module to learn the weight coefficients of one bipartite graph constructed by the fully connected graphs having different modalities in the graph domain and by selectively fusing the node features. Based on the deformation diagram between the spatial-graph space and our proposed graph co-attention module, we present a multimodal prior-guided segmentation framework, which uses two strategies for two clinical situations: Modality-Specific Learning Strategy and Co-Modality Learning Strategy. Besides, the improved Co-Modality Learning Strategy is used with trainable weights in the multi-task loss for the optimization of the proposed framework. We validated our proposed modules and frameworks on two multimodal MRI datasets: our private liver lesion dataset and a public prostate zone dataset. Our experimental results on both datasets prove the superiority of our proposed approaches.

[A Multi-Label Fusion Based Level Set Method for Multiple Sclerosis Lesion Segmentation].

A Level Set Method for Multiple Sclerosis Lesion Segmentation

Mutual Information-Based Graph Co-Attention Networks for Multimodal Prior-Guided Magnetic Resonance Imaging Segmentation

Level Set Framework of Multi-Atlas Label Fusion with Applications to Magnetic Resonance Imaging Segmentation of Brain Region of Interests and Cardiac Left Ventricles

Towards an accurate and generalizable multiple sclerosis lesion segmentation model using self-ensembled lesion fusion

A Fast Level Set Model of Multi‐atlas Labels Fusion for 3D MRI Tissues Segmentation with Application of Split Bregman Method

Segmentation of Multiple Sclerosis lesion in brain MR images using Fuzzy C-Means

Non-locally regularized segmentation of multiple sclerosis lesion from multi-channel MRI data

Level Set Formulation Based on Edge and Region Information with Application to Accurate Lesion Segmentation of Brain Magnetic Resonance Images

Brain Tumor Segmentation through Data Fusion of T2-Weighted Image and MR Spectroscopy

A Framework of Fuzzy Information Fusion for the Segmentation of Brain Tumor Tissues on MR Images

Multiple Sclerosis Lesion Analysis in Brain Magnetic Resonance Images: Techniques and Clinical Applications

Multiphase Level Set-based Multi-objective Image Segmentation Cooperating with Spatial Fuzzy C-means

Improving Multiple Sclerosis Lesion Segmentation Across Clinical Sites: A Federated Learning Approach with Noise-Resilient Training

Region-to-boundary deep learning model with multi-scale feature fusion for medical image segmentation

Fuzzy information fusion scheme used to segment brain tumor from MR images

MTANS: Multi-Scale Mean Teacher Combined Adversarial Network with Shape-Aware Embedding for Semi-Supervised Brain Lesion Segmentation

A Multi-Scale and Multi-Level Fusion Approach for Deep Learning-Based Liver Lesion Diagnosis in Magnetic Resonance Images with Visual Explanation

Segmentation of Multiple Sclerosis Lesions in Brain MR Images Using Spatially Constrained Possibilistic Fuzzy C-Means Classification

Multiple Sclerosis Lesion Segmentation from Brain MRI via Fully Convolutional Neural Networks

Learning rich features with hybrid loss for brain tumor segmentation