Abstract:Background and objective: Accurate prognosis prediction for cancer patients plays a significant role in the formulation of treatment strategies, considerably impacting personalized medicine. Recent advancements in this field indicate that integrating information from various modalities, such as genetic and clinical data, and developing multi-modal deep learning models can enhance prediction accuracy. However, most existing multi-modal deep learning methods either overlook patient similarities that benefit prognosis prediction or fail to effectively capture diverse information due to measuring patient similarities from a single perspective. To address these issues, a novel framework called multi-modal multi-view graph convolutional networks (MMGCN) is proposed for cancer prognosis prediction. Methods: Initially, we utilize the similarity network fusion (SNF) algorithm to merge patient similarity networks (PSNs), individually constructed using gene expression, copy number alteration, and clinical data, into a fused PSN for integrating multi-modal information. To capture diverse perspectives of patient similarities, we treat the fused PSN as a multi-view graph by considering each single-edge-type subgraph as a view graph, and propose multi-view graph convolutional networks (GCNs) with a view-level attention mechanism. Moreover, an edge homophily prediction module is designed to alleviate the adverse effects of heterophilic edges on the representation power of GCNs. Finally, comprehensive representations of patient nodes are obtained to predict cancer prognosis. Results: Experimental results demonstrate that MMGCN outperforms state-of-the-art baselines on four public datasets, including METABRIC, TCGA-BRCA, TCGA-LGG, and TCGA-LUSC, with the area under the receiver operating characteristic curve achieving 0.827 ± 0.005, 0.805 ± 0.014, 0.925 ± 0.007, and 0.746 ± 0.013, respectively. Conclusions: Our study reveals the effectiveness of the proposed MMGCN, which deeply explores patient similarities related to different modalities from a broad perspective, in enhancing the performance of multi-modal cancer prognosis prediction. The source code is publicly available at https://github.com/ping-y/MMGCN.

Multimodal Survival Ensemble Network: Integrating Genomic and Histopathological Insights for Enhanced Cancer Prognosis.

Learning Comprehensive Multimodal Representation for Cancer Survival Prediction

A Multimodal Affinity Fusion Network for Predicting the Survival of Breast Cancer Patients

MGCT: Mutual-Guided Cross-Modality Transformer for Survival Outcome Prediction using Integrative Histopathology-Genomic Features

MMGCN: Multi-modal multi-view graph convolutional networks for cancer prognosis prediction

Pathology-genomic fusion via biologically informed cross-modality graph learning for survival analysis

Pancancer survival prediction using a deep learning architecture with multimodal representation and integration

A multimodal ensemble approach for clear cell renal cell carcinoma treatment outcome prediction

Predicting the Survival of Cancer Patients With Multimodal Graph Neural Network

Multimodal Learning for Non-small Cell Lung Cancer Prognosis

Cohort-Individual Cooperative Learning for Multimodal Cancer Survival Analysis

MoME: Mixture of Multimodal Experts for Cancer Survival Prediction

Pathomic Fusion: An Integrated Framework for Fusing Histopathology and Genomic Features for Cancer Diagnosis and Prognosis

Deep Biological Pathway Informed Pathology-Genomic Multimodal Survival Prediction

M2EF-NNs: Multimodal Multi-instance Evidence Fusion Neural Networks for Cancer Survival Prediction

Graph attention-based fusion of pathology images and gene expression for prediction of cancer survival

Multimodal Siamese Model for Breast Cancer Survival Prediction

Cross-modality Attention-based Multimodal Fusion for Non-small Cell Lung Cancer (NSCLC) Patient Survival Prediction

MIF: Multi-Shot Interactive Fusion Model for Cancer Survival Prediction Using Pathological Image and Genomic Data

Multimodal Cross-Task Interaction for Survival Analysis in Whole Slide Pathological Images

Embedding-based Multimodal Learning on Pan-Squamous Cell Carcinomas for Improved Survival Outcomes