A Novel Fusion Mechanism for Multimodal Neuroimaging of Focal Cortical Dysplasias.

Yanzhu Lu,Hao Yu,Gongpeng Cao,Manli Zhang,Guixia Kang,Lixin Cai
DOI: https://doi.org/10.1145/3637732.3637773
2024-01-01
Abstract:Refractory epilepsy caused by focal cortical dysplasia (FCD) is difficult to observe in imaging, which brings great difficulty in preoperative lesion localization. Currently, the deep-learning Neural Networks for FCD lesion location mainly rely on multimodal images as MRI and PET, which overcomes the limitation of information in single-modality. We propose a network with double-branch encoder and MAP-attention gate (DE-MAPGNet), which is able to improve the effectiveness of multimodal images feature fusion and locate the FCD lesions accurately. One of the encoding branches takes T1, PET, and FLAIR images as input, while the other receives MAP post-processed images that contain salient information of lesions. First of all, since the abnormal features are various in different modalities, the dual-branch encoder extracts targeted features from the modality with gap. Secondly, the integration of double-branch feature maps is carried out at the skip connection. The MAP-attention designed in this paper can weight in the spatial and channel dimensions of the multimodal images respectively, so as to enhance the filtering effect of the model on redundant information and pay more attention to abnormal areas. In this paper, the multimodal images of 28 children with FCD were predicted by using the method of five-fold cross-validation. The final prediction results are superior to the state-of-art in most indicators which has a reference value for the clinical assistant doctors in preoperative lesion localization.
What problem does this paper attempt to address?