Abstract:Background: Accurate segmentation of pancreatic cancer tumors using positron emission tomography/computed tomography (PET/CT) multimodal images is crucial for clinical diagnosis and prognosis evaluation. However, deep learning methods for automated medical image segmentation require a substantial amount of manually labeled data, making it time-consuming and labor-intensive. Moreover, addition or simple stitching of multimodal images leads to redundant information, failing to fully exploit the complementary information of multimodal images. Therefore, we developed a semisupervised multimodal network that leverages limited labeled samples and introduces a cross-fusion and mutual information minimization (MIM) strategy for PET/CT 3D segmentation of pancreatic tumors. Methods: Our approach combined a cross multimodal fusion (CMF) module with a cross-attention mechanism. The complementary multimodal features were fused to form a multifeature set to enhance the effectiveness of feature extraction while preserving specific features of each modal image. In addition, we designed an MIM module to mitigate redundant high-level modal information and compute the latent loss of PET and CT. Finally, our method employed the uncertainty-aware mean teacher semi-supervised framework to segment regions of interest from PET/CT images using a small amount of labeled data and a large amount of unlabeled data. Results: We evaluated our combined MIM and CMF semisupervised segmentation network (MIM-CMFNet) on a private dataset of pancreatic cancer, yielding an average Dice coefficient of 73.14%, an average Jaccard index score of 60.56%, and an average 95% Hausdorff distance (95HD) of 6.30 mm. In addition, to verify the broad applicability of our method, we used a public dataset of head and neck cancer, yielding an average Dice coefficient of 68.71%, an average Jaccard index score of 57.72%, and an average 95HD of 7.88 mm. Conclusions: The experimental results demonstrate the superiority of our MIM-CMFNet over existing semisupervised techniques. Our approach can achieve a performance similar to that of fully supervised segmentation methods while significantly reducing the data annotation cost by 80%, suggesting it is highly practicable for clinical application.

Semi-supervised interactive fusion network for MR image segmentation

Deep Convolutional Neural Network with a Multi-Scale Attention Feature Fusion Module for Segmentation of Multimodal Brain Tumor.

Attention-enhanced multiscale feature fusion network for pancreas and tumor segmentation

Semisupervised 3D segmentation of pancreatic tumors in positron emission tomography/computed tomography images using a mutual information minimization and cross-fusion strategy

Collaborative multi-feature extraction and scale-aware semantic information mining for medical image segmentation

Medical image segmentation based on self-supervised hybrid fusion network

Feature interaction network based on hierarchical decoupled convolution for 3D medical image segmentation

Self-Supervised Multi-Modal Hybrid Fusion Network for Brain Tumor Segmentation.

Exploration of Multi-Scale Image Fusion Systems in Intelligent Medical Image Analysis

Pancreatic cancer segmentation in unregistered multi-parametric MRI with adversarial learning and multi-scale supervision

A 2.5D multi-path fusion network framework with focusing on z-axis 3D joint for medical image segmentation

Segmentation of rectal tumor from multi-parametric MRI images using an attention-based fusion network

A Novel Network Based on Densely Connected Fully Convolutional Networks for Segmentation of Lung Tumors on Multi-Modal MR Images

Feature-enhanced Multi-Sequence MRI-based Fusion Mechanism for Breast Tumor Segmentation

A multibranch and multiscale neural network based on semantic perception for multimodal medical image fusion

MLFINet : A multi-level feature interaction 3D medical image segmentation network

A medical image segmentation method for rectal tumors based on multi‐scale feature retention and multiple attention mechanisms

Automated Cervical Tumor Segmentation on MR Images Using Multi-View Feature Attention Network

MRS‐Net: Brain tumour segmentation network based on feature fusion and attention mechanism

MSAIF-Net: A Multistage Spatial Attention-Based Invertible Fusion Network for MR Images.

GSCFN: A graph self-construction and fusion network for semi-supervised brain tissue segmentation in MRI