Multi-Modal Fusion Learning for Cervical Dysplasia Diagnosis.

Tingting Chen,Xinjun Ma,Xingde Ying,Wenzhe Wang,Chunnv Yuan,Weiguo Lu,Danny Z. Chen,Jian Wu
DOI: https://doi.org/10.1109/isbi.2019.8759303
2019-01-01
Abstract:Fusion of multi-modal information from a patient's screening tests can help improve the diagnostic accuracy of cervical dysplasia. In this paper, we present a novel multi-modal deep learning fusion network, called MultiFuseNet, for cervical dysplasia diagnosis, utilizing multi-modal data from cervical screening results. To exploit the relations among different image modalities, we propose an Attention Mutual-Enhance (AME) module to fuse features of each modality at the feature extraction stage. Specifically, we first develop the Fused Faster R-CNN with AME modules for automatic cervix region detection and fused image feature learning, and then incorporate non-image information into the learning model to jointly learn non-linear correlations among all the modalities. To effectively train the Fused Faster R-CNN, we employ an alternating training scheme. Experimental results show the effectiveness of our method, which achieves an average accuracy of 87.4% (88.6% sensitivity and 86.1% specificity) on a large dataset, outperforming the methods using any single modality alone and the known multi-modal methods.
What problem does this paper attempt to address?