Abstract:Multimodal magnetic resonance imaging (MRI) provides complementary information about targets, and the segmentation of multimodal MRI is widely used as an essential preprocessing step for initial diagnosis, stage differentiation, and post-treatment efficacy evaluation in clinical situations. For the main modality or each of the modalities, it is important to enhance the visual information by modeling the connection and effectively fusing the features among them. However, the existing methods for multimodal segmentation have a drawback; they coincidentally drop information of individual modality during the fusion process. Recently, graph learning-based methods have been applied in segmentation, and these methods have achieved considerable improvements by modeling the relationships across feature regions and reasoning using global information. In this paper, we propose a graph learning-based approach to efficiently extract modality-specific features and establish regional correspondence effectively among all modalities. In detail, after projecting features into a graph domain and employing graph convolution to propagate information across all regions for learning global modality-specific features, we propose a mutual information-based graph co-attention module to learn the weight coefficients of one bipartite graph constructed by the fully connected graphs having different modalities in the graph domain and by selectively fusing the node features. Based on the deformation diagram between the spatial-graph space and our proposed graph co-attention module, we present a multimodal prior-guided segmentation framework, which uses two strategies for two clinical situations: Modality-Specific Learning Strategy and Co-Modality Learning Strategy. Besides, the improved Co-Modality Learning Strategy is used with trainable weights in the multi-task loss for the optimization of the proposed framework. We validated our proposed modules and frameworks on two multimodal MRI datasets: our private liver lesion dataset and a public prostate zone dataset. Our experimental results on both datasets prove the superiority of our proposed approaches.

Multiphase Level Set-based Multi-objective Image Segmentation Cooperating with Spatial Fuzzy C-means

Multi-phase Liver Tumor Segmentation with Spatial Aggregation and Uncertain Region Inpainting

Mutual Information-Based Graph Co-Attention Networks for Multimodal Prior-Guided Magnetic Resonance Imaging Segmentation

Coarse-to-fine Image Co-segmentation with Intra and Inter Rank Constraints

Ultrasound Image Segmentation Based on Multi-Scale Fuzzy C-Means and Particle Swarm Optimization

Segmentation of Multiple Sclerosis Lesions in Brain MR Images Using Spatially Constrained Possibilistic Fuzzy C-Means Classification

Canny operator based level set segmentation algorithm for medical images

A Modified Level Set Algorithm Based on Point Distance Shape Constraint for Lesion and Organ Segmentation

Active contour model-based medical image segmentation method collaborative with fuzzy C-means

A Hybrid Method for Image Segmentation Based on Artificial Fish Swarm Algorithm and Fuzzy c-Means Clustering

Mr Image Segmentation Algorithm Based On Non-Local Fuzzy C-Means Clustering

Non-Local and Kernel Function Fuzzy Segmentation Algorithm for Infant Brain Magnetic Resonance Image

Novel fuzzy clustering algorithm with variable multi-pixel fitting spatial information for image segmentation

FMCSSE: fuzzy modified cuckoo search with spatial exploration for biomedical image segmentation

Robust Global Minimization of Active Contour Model for Multi-Object Medical Image Segmentation

Incorporating Adaptive Local Information Into Fuzzy Clustering for Image Segmentation

A Hybrid Approach for Image Segmentation Using Fuzzy Clustering and Level Set Method

The survey of fuzzy clustering method for image segmentation

Multiobjective multiple features fusion: A case study in image segmentation

Sparse Regularization-Based Fuzzy C -Means Clustering Incorporating Morphological Grayscale Reconstruction and Wavelet Frames

Multi-value Image Segmentation Based on FCM Algorithm and Graph Cut Theory.