Abstract:Multimodal magnetic resonance imaging (MRI) provides complementary information about targets, and the segmentation of multimodal MRI is widely used as an essential preprocessing step for initial diagnosis, stage differentiation, and post-treatment efficacy evaluation in clinical situations. For the main modality or each of the modalities, it is important to enhance the visual information by modeling the connection and effectively fusing the features among them. However, the existing methods for multimodal segmentation have a drawback; they coincidentally drop information of individual modality during the fusion process. Recently, graph learning-based methods have been applied in segmentation, and these methods have achieved considerable improvements by modeling the relationships across feature regions and reasoning using global information. In this paper, we propose a graph learning-based approach to efficiently extract modality-specific features and establish regional correspondence effectively among all modalities. In detail, after projecting features into a graph domain and employing graph convolution to propagate information across all regions for learning global modality-specific features, we propose a mutual information-based graph co-attention module to learn the weight coefficients of one bipartite graph constructed by the fully connected graphs having different modalities in the graph domain and by selectively fusing the node features. Based on the deformation diagram between the spatial-graph space and our proposed graph co-attention module, we present a multimodal prior-guided segmentation framework, which uses two strategies for two clinical situations: Modality-Specific Learning Strategy and Co-Modality Learning Strategy. Besides, the improved Co-Modality Learning Strategy is used with trainable weights in the multi-task loss for the optimization of the proposed framework. We validated our proposed modules and frameworks on two multimodal MRI datasets: our private liver lesion dataset and a public prostate zone dataset. Our experimental results on both datasets prove the superiority of our proposed approaches.

Multi-Atlas Segmentation of Anatomical Brain Structures Using Hierarchical Hypergraph Learning

Multi-Atlas Based Segmentation of Brainstem Nuclei from MR Images by Deep Hyper-Graph Learning

Mutual Information-Based Graph Co-Attention Networks for Multimodal Prior-Guided Magnetic Resonance Imaging Segmentation

Cross-Modality Multi-Atlas Segmentation via Deep Registration and Label Fusion

A Multi-Atlas Segmentation Algorithm with an Improved Sparse Representation on Brain MR Images

Supervoxel Based Method for Multi-Atlas Segmentation of Brain MR Images

A Robust Discriminative Multi-Atlas Label Fusion Method for Hippocampus Segmentation from MR Image

Patch-wise Label Propagation for MR Brain Segmentation Based on Multi-Atlas Images

Novel Multi-Atlas Segmentation by Matrix Completion

Integrated 3d flow-based multi-atlas brain structure segmentation.

A New Multi-Atlas Based Deep Learning Segmentation Framework with Differentiable Atlas Feature Warping

Automatic Hippocampus Segmentation of Magnetic Resonance Imaging Images Using Multiple Atlases

Multi-atlas learner fusion: An efficient segmentation approach for large-scale data

MAS-CL: An End-to-End Multi-Atlas Supervised Contrastive Learning Framework for Brain ROI Segmentation

Automatic Labeling of MR Brain Images by Hierarchical Learning of Atlas Forests.

Interactive Registration and Segmentation for Multi-Atlas-Based Labeling of Brain MR Image

Dynamic Hierarchical Multi-Scale Fusion Network with Axial MLP for Medical Image Segmentation

Neural Multi-Atlas Label Fusion: Application to Cardiac MR Images

Cross-Modality Multi-Atlas Segmentation Using Deep Neural Networks

Label Fusion Based Brain MR Image Segmentation Via a Latent Selective Model

Integrating Semi-supervised and Supervised Learning Methods for Label Fusion in Multi-Atlas Based Image Segmentation