Abstract:Multimodal magnetic resonance imaging (MRI) provides complementary information about targets, and the segmentation of multimodal MRI is widely used as an essential preprocessing step for initial diagnosis, stage differentiation, and post-treatment efficacy evaluation in clinical situations. For the main modality or each of the modalities, it is important to enhance the visual information by modeling the connection and effectively fusing the features among them. However, the existing methods for multimodal segmentation have a drawback; they coincidentally drop information of individual modality during the fusion process. Recently, graph learning-based methods have been applied in segmentation, and these methods have achieved considerable improvements by modeling the relationships across feature regions and reasoning using global information. In this paper, we propose a graph learning-based approach to efficiently extract modality-specific features and establish regional correspondence effectively among all modalities. In detail, after projecting features into a graph domain and employing graph convolution to propagate information across all regions for learning global modality-specific features, we propose a mutual information-based graph co-attention module to learn the weight coefficients of one bipartite graph constructed by the fully connected graphs having different modalities in the graph domain and by selectively fusing the node features. Based on the deformation diagram between the spatial-graph space and our proposed graph co-attention module, we present a multimodal prior-guided segmentation framework, which uses two strategies for two clinical situations: Modality-Specific Learning Strategy and Co-Modality Learning Strategy. Besides, the improved Co-Modality Learning Strategy is used with trainable weights in the multi-task loss for the optimization of the proposed framework. We validated our proposed modules and frameworks on two multimodal MRI datasets: our private liver lesion dataset and a public prostate zone dataset. Our experimental results on both datasets prove the superiority of our proposed approaches.

Finding Frequent Approximate Subgraphs in Medical Image Database

Mutual Information-Based Graph Co-Attention Networks for Multimodal Prior-Guided Magnetic Resonance Imaging Segmentation

A graph-based approach for the retrieval of multi-modality medical images

Graph Convolutional Networks for Multi-modality Medical Imaging: Methods, Architectures, and Clinical Applications

Cross Chest Graph for Disease Diagnosis with Structural Relational Reasoning

Entropic graph estimation integrated with SIFT features for medical image non-rigid registration

Graph matching survey for medical imaging: On the way to deep learning

Depression Classification Using Frequent Subgraph Mining Based on Pattern Growth of Frequent Edge in Functional Magnetic Resonance Imaging Uncertain Network

Mining interesting association rules in medical images

A Topo-Graph Model for Indistinct Target Boundary Definition from Anatomical Images

Mining Image Sequence Similarity Patterns In Brain Images

Deep Lesion Graphs in the Wild: Relationship Learning and Organization of Significant Radiology Image Findings in a Diverse Large-Scale Lesion Database

GraphCL: Graph-based Clustering for Semi-Supervised Medical Image Segmentation

The Generation and Application of Medical Image Grid Based on Differential Geometric Features.

Graph-Based Deep Learning for Medical Diagnosis and Analysis: Past, Present and Future

FIT-graph: A multi-grained evolutionary graph based framework for disease diagnosis

Supervised Graph Hashing for Histopathology Image Retrieval and Classification

Extracting Frequent Connected Subgraphs from Large Graph Sets.

Brain CT Database for Content-Based Image Retrieval

Topological structure and global features enhanced graph reasoning model for non-small cell lung cancer segmentation from CT

Efficient Data-Sketches and Fine-Tuning for Early Detection of Distributional Drift in Medical Imaging