SELECTOR: Heterogeneous graph network with convolutional masked autoencoder for multimodal robust prediction of cancer survival

Liangrui Pan,Yijun Peng,Yan Li,Xiang Wang,Wenjuan Liu,Liwen Xu,Qingchun Liang,Shaoliang Peng
2024-03-14
Abstract:Accurately predicting the survival rate of cancer patients is crucial for aiding clinicians in planning appropriate treatment, reducing cancer-related medical expenses, and significantly enhancing patients' quality of life. Multimodal prediction of cancer patient survival offers a more comprehensive and precise approach. However, existing methods still grapple with challenges related to missing multimodal data and information interaction within modalities. This paper introduces SELECTOR, a heterogeneous graph-aware network based on convolutional mask encoders for robust multimodal prediction of cancer patient survival. SELECTOR comprises feature edge reconstruction, convolutional mask encoder, feature cross-fusion, and multimodal survival prediction modules. Initially, we construct a multimodal heterogeneous graph and employ the meta-path method for feature edge reconstruction, ensuring comprehensive incorporation of feature information from graph edges and effective embedding of nodes. To mitigate the impact of missing features within the modality on prediction accuracy, we devised a convolutional masked autoencoder (CMAE) to process the heterogeneous graph post-feature reconstruction. Subsequently, the feature cross-fusion module facilitates communication between modalities, ensuring that output features encompass all features of the modality and relevant information from other modalities. Extensive experiments and analysis on six cancer datasets from TCGA demonstrate that our method significantly outperforms state-of-the-art methods in both modality-missing and intra-modality information-confirmed cases. Our codes are made available at
Computer Vision and Pattern Recognition,Artificial Intelligence,Machine Learning
What problem does this paper attempt to address?
### Problems the Paper Aims to Solve The paper aims to address several key issues in survival rate prediction for cancer patients: 1. **Multimodal Data Integration**: - Current methods face challenges in handling multimodal data (such as pathological images, genomic data, clinical records, etc.), especially in terms of information interaction between modalities. 2. **Impact of Missing Data**: - Multimodal data often suffer from missing data, which negatively affects the accuracy of survival rate predictions. For example, missing data can occur due to patients' unwillingness to bear high medical costs or doctors' failure to comprehensively collect multimodal data. 3. **Improving Prediction Accuracy**: - Researchers have proposed the SELECTOR method, a network based on a heterogeneous graph convolutional masked autoencoder, which effectively addresses the issue of missing data within and between modalities and significantly improves the accuracy of survival rate predictions. By introducing a feature edge reconstruction module, a convolutional masked autoencoder (CMAE), a feature cross-fusion module, and a multimodal survival prediction module, SELECTOR can maintain high prediction performance even in the presence of missing data. Experimental results show that this method significantly outperforms existing methods on six cancer datasets from TCGA.