Abstract:In clinical diagnosis, positron emission tomography and computed tomography (PET-CT) images containing complementary information are fused. Tumor segmentation based on multi-modal PET-CT images is an important part of clinical diagnosis and treatment. However, the existing current PET-CT tumor segmentation methods mainly focus on positron emission tomography (PET) and computed tomography (CT) feature fusion, which weakens the specificity of the modality. In addition, the information interaction between different modal images is usually completed by simple addition or concatenation operations, but this has the disadvantage of introducing irrelevant information during the multi-modal semantic feature fusion, so effective features cannot be highlighted. To overcome this problem, this paper propose a novel Multi-modal Fusion and Calibration Networks (MFCNet) for tumor segmentation based on three-dimensional PET-CT images. First, a Multi-modal Fusion Down-sampling Block (MFDB) with a residual structure is developed. The proposed MFDB can fuse complementary features of multi-modal images while retaining the unique features of different modal images. Second, a Multi-modal Mutual Calibration Block (MMCB) based on the inception structure is designed. The MMCB can guide the network to focus on a tumor region by combining different branch decoding features using the attention mechanism and extracting multi-scale pathological features using a convolution kernel of different sizes. The proposed MFCNet is verified on both the public dataset (Head and Neck cancer) and the in-house dataset (pancreas cancer). The experimental results indicate that on the public and in-house datasets, the average Dice values of the proposed multi-modal segmentation network are 74.14% and 76.20%, while the average Hausdorff distances are 6.41 and 6.84, respectively. In addition, the experimental results show that the proposed MFCNet outperforms the state-of-the-art methods on the two datasets.

M 3 Net: A multi-scale multi-view framework for multi-phase pancreas segmentation based on cross-phase non-local attention

Multi-phase and Multi-level Selective Feature Fusion for Automated Pancreas Segmentation from CT Images.

A Self-Attention Network with Position-Prior Guided Feature Mapping and Clustering for 3D Pancreas Segnentation

Harnessing 2D Networks and 3D Features for Automated Pancreas Segmentation from Volumetric CT Images

A deep learning framework for pancreas segmentation with multi-atlas registration and 3D level-set

Attention-enhanced multiscale feature fusion network for pancreas and tumor segmentation

Cascaded MultiTask 3-D Fully Convolutional Networks for Pancreas Segmentation

mm3DSNet: multi-scale and multi-feedforward self-attention 3D segmentation network for CT scans of hepatobiliary ducts

MDHT-Net: Multi-scale Deformable U-Net with Cos-spatial and Channel Hybrid Transformer for pancreas segmentation

M3BUNet: Mobile Mean Max UNet for Pancreas Segmentation on CT-Scans

MFCNet: A multi-modal fusion and calibration networks for 3D pancreas tumor segmentation on PET-CT images

High-Resolution Recurrent Gated Fusion Network for 3D Pancreas Segmentation.

Automatic multi-tissue segmentation in pancreatic pathological images with selected multi-scale attention network

Iterative 3D Feature Enhancement Network for Pancreas Segmentation from CT Images

Pancreas segmentation using a dual-input v-mesh network

A Fully 3D Cascaded Framework for Pancreas Segmentation.

Pancreas segmentation in CT based on RC-3DUNet with SOM

Pancreas Segmentation Via Spatial Context Based U-net and Bidirectional LSTM.

MCMC Guided CNN Training and Segmentation for Pancreas Extraction

A 2.5D multi-path fusion network framework with focusing on z-axis 3D joint for medical image segmentation

Multi-target segmentation of pancreas and pancreatic tumor based on fusion of attention mechanism