Multi-Modal Tumor Segmentation with Deformable Aggregation and Uncertain Region Inpainting
Yue Zhang,Chengtao Peng,Ruofeng Tong,Lanfen Lin,Yen-Wei Chen,Qingqing Chen,Hongjie Hu,S. Kevin Zhou
DOI: https://doi.org/10.1109/tmi.2023.3275592
IF: 10.6
2023-01-01
IEEE Transactions on Medical Imaging
Abstract:Multi-modal tumor segmentation exploits complementary information from different modalities to help recognize tumor regions. Known multi-modal segmentation methods mainly have deficiencies in two aspects: First, the adopted multi-modal fusion strategies are built upon well-aligned input images, which are vulnerable to spatial misalignment between modalities (caused by respiratory motions, different scanning parameters, registration errors, etc). Second, the performance of known methods remains subject to the uncertainty of segmentation, which is particularly acute in tumor boundary regions. To tackle these issues, in this paper, we propose a novel multi-modal tumor segmentation method with deformable feature fusion and uncertain region refinement. Concretely, we introduce a deformable aggregation module, which integrates feature alignment and feature aggregation in an ensemble, to reduce inter-modality misalignment and make full use of cross-modal information. Moreover, we devise an uncertain region inpainting module to refine uncertain pixels using neighboring discriminative features. Experiments on two clinical multi-modal tumor datasets demonstrate that our method achieves promising tumor segmentation results and outperforms state-of-the-art methods.
engineering, biomedical,imaging science & photographic technology, electrical & electronic,computer science, interdisciplinary applications,radiology, nuclear medicine & medical imaging