Abstract:Multimodal magnetic resonance imaging (MRI) provides complementary information about targets, and the segmentation of multimodal MRI is widely used as an essential preprocessing step for initial diagnosis, stage differentiation, and post-treatment efficacy evaluation in clinical situations. For the main modality or each of the modalities, it is important to enhance the visual information by modeling the connection and effectively fusing the features among them. However, the existing methods for multimodal segmentation have a drawback; they coincidentally drop information of individual modality during the fusion process. Recently, graph learning-based methods have been applied in segmentation, and these methods have achieved considerable improvements by modeling the relationships across feature regions and reasoning using global information. In this paper, we propose a graph learning-based approach to efficiently extract modality-specific features and establish regional correspondence effectively among all modalities. In detail, after projecting features into a graph domain and employing graph convolution to propagate information across all regions for learning global modality-specific features, we propose a mutual information-based graph co-attention module to learn the weight coefficients of one bipartite graph constructed by the fully connected graphs having different modalities in the graph domain and by selectively fusing the node features. Based on the deformation diagram between the spatial-graph space and our proposed graph co-attention module, we present a multimodal prior-guided segmentation framework, which uses two strategies for two clinical situations: Modality-Specific Learning Strategy and Co-Modality Learning Strategy. Besides, the improved Co-Modality Learning Strategy is used with trainable weights in the multi-task loss for the optimization of the proposed framework. We validated our proposed modules and frameworks on two multimodal MRI datasets: our private liver lesion dataset and a public prostate zone dataset. Our experimental results on both datasets prove the superiority of our proposed approaches.

Image Recognition with Missing-Features Based on Gaussian Mixture Model and Graph Constrained Nonnegative Matrix Factorization

A Novel Image Classification Method Based on Manifold Learning and Gaussian Mixture Model

Facial feature points detecting based on Gaussian Mixture Models

Missing-Data Classification with the Extended Full-Dimensional Gaussian Mixture Model: Applications to EMG-Based Motion Recognition

Image Retrieval Based on Multiview Constrained Nonnegative Matrix Factorization and Gaussian Mixture Model Spectral Clustering Method

Image Feature Extraction via Graph Embedding Regularized Projective Non-negative Matrix Factorization.

Gaussian Mixture Model Based Probabilistic Modeling of Images for Medical Image Segmentation

Mutual Information-Based Graph Co-Attention Networks for Multimodal Prior-Guided Magnetic Resonance Imaging Segmentation

Double Gaussian Mixture Model for Image Segmentation with Spatial Relationships

Medical Image Registration Algorithm Based on Bounded Generalized Gaussian Mixture Model

A hierarchical nonparametric Bayesian approach for medical images and gene expressions classification

Detection of Tuberculosis in Sputum Smear Images by Gaussian Mixture Models

Medical Image Categorization Based on Gaussian Mixture Model

Retinal Image Registration Via Feature-Guided Gaussian Mixture Model.

Deep Gaussian mixture model for unsupervised image segmentation

Image Segmentation Based on Spatially Coherent Gaussian Mixture Model

Gaussian Mixture Model Clustering with Incomplete Data

Categorization Method Research for Medical Image Using Gaussian Mixture Model

Sequential Three-Way Decision Based on Gaussian Mixture Model and EM Algorithm

Robust Image Matching Via Feature Guided Gaussian Mixture Model

Accurate Image Segmentation Using Gaussian Mixture Model with Saliency Map