Abstract:Multimodal magnetic resonance imaging (MRI) provides complementary information about targets, and the segmentation of multimodal MRI is widely used as an essential preprocessing step for initial diagnosis, stage differentiation, and post-treatment efficacy evaluation in clinical situations. For the main modality or each of the modalities, it is important to enhance the visual information by modeling the connection and effectively fusing the features among them. However, the existing methods for multimodal segmentation have a drawback; they coincidentally drop information of individual modality during the fusion process. Recently, graph learning-based methods have been applied in segmentation, and these methods have achieved considerable improvements by modeling the relationships across feature regions and reasoning using global information. In this paper, we propose a graph learning-based approach to efficiently extract modality-specific features and establish regional correspondence effectively among all modalities. In detail, after projecting features into a graph domain and employing graph convolution to propagate information across all regions for learning global modality-specific features, we propose a mutual information-based graph co-attention module to learn the weight coefficients of one bipartite graph constructed by the fully connected graphs having different modalities in the graph domain and by selectively fusing the node features. Based on the deformation diagram between the spatial-graph space and our proposed graph co-attention module, we present a multimodal prior-guided segmentation framework, which uses two strategies for two clinical situations: Modality-Specific Learning Strategy and Co-Modality Learning Strategy. Besides, the improved Co-Modality Learning Strategy is used with trainable weights in the multi-task loss for the optimization of the proposed framework. We validated our proposed modules and frameworks on two multimodal MRI datasets: our private liver lesion dataset and a public prostate zone dataset. Our experimental results on both datasets prove the superiority of our proposed approaches.

Biomedical image segmentation for precision radiation oncology

Efficient Segmentation Method Using Quantised and Non‐linear CeNN for Breast Tumour Classification

Mutual Information-Based Graph Co-Attention Networks for Multimodal Prior-Guided Magnetic Resonance Imaging Segmentation

Human Tumor Detection Using Active Contour and Region Growing Segmentation

The Applications of Radiomics in Precision Diagnosis and Treatment of Oncology: Opportunities and Challenges

A medical image segmentation method for rectal tumors based on multi‐scale feature retention and multiple attention mechanisms

Multiple organ segmentation framework for brain metastasis radiotherapy

Radiomics in medical imaging—detection, extraction and segmentation

A Focus on Recent Development of Image Segmentation

Attention guided neural ODE network for breast tumor segmentation in medical images

Novel Robust Automatic Brain-Tumor Detection and Segmentation Using Magnetic Resonance Imaging

Image Segmentation Via Adaptive K-mean Clustering and Knowledge-Based Morphological Operations with Biomedical Applications

Image Segmentation of Triple-Negative Breast Cancer by Incorporating Multiscale and Parallel Attention Mechanisms

An image segmentation algorithm based on multi-scale feature pyramid network

A Review Of Deep Learning Based Methods For Medical Image Multi-Organ Segmentation

Biomedical Image Segmentation Via Constrained Graph Cuts and Pre-Segmentation

Automatic Nuclear Segmentation Using Multiscale Radial Line Scanning With Dynamic Programming.

Deep Learning-Based Image Segmentation on Multimodal Medical Imaging

Lesion Segmentation and Identification of Breast Tumor on Dynamic Contrast-Enhanced Magnetic Resonance Imaging

Deep learning for automatic target volume segmentation in radiation therapy: a review

3D Lightweight Network for Simultaneous Registration and Segmentation of Organs-at-Risk in CT Images of Head and Neck Cancer