Abstract:Background and objective: Accurate prognosis prediction for cancer patients plays a significant role in the formulation of treatment strategies, considerably impacting personalized medicine. Recent advancements in this field indicate that integrating information from various modalities, such as genetic and clinical data, and developing multi-modal deep learning models can enhance prediction accuracy. However, most existing multi-modal deep learning methods either overlook patient similarities that benefit prognosis prediction or fail to effectively capture diverse information due to measuring patient similarities from a single perspective. To address these issues, a novel framework called multi-modal multi-view graph convolutional networks (MMGCN) is proposed for cancer prognosis prediction. Methods: Initially, we utilize the similarity network fusion (SNF) algorithm to merge patient similarity networks (PSNs), individually constructed using gene expression, copy number alteration, and clinical data, into a fused PSN for integrating multi-modal information. To capture diverse perspectives of patient similarities, we treat the fused PSN as a multi-view graph by considering each single-edge-type subgraph as a view graph, and propose multi-view graph convolutional networks (GCNs) with a view-level attention mechanism. Moreover, an edge homophily prediction module is designed to alleviate the adverse effects of heterophilic edges on the representation power of GCNs. Finally, comprehensive representations of patient nodes are obtained to predict cancer prognosis. Results: Experimental results demonstrate that MMGCN outperforms state-of-the-art baselines on four public datasets, including METABRIC, TCGA-BRCA, TCGA-LGG, and TCGA-LUSC, with the area under the receiver operating characteristic curve achieving 0.827 ± 0.005, 0.805 ± 0.014, 0.925 ± 0.007, and 0.746 ± 0.013, respectively. Conclusions: Our study reveals the effectiveness of the proposed MMGCN, which deeply explores patient similarities related to different modalities from a broad perspective, in enhancing the performance of multi-modal cancer prognosis prediction. The source code is publicly available at https://github.com/ping-y/MMGCN.

A Multi-modal Fusion Framework Based on Multi-task Correlation Learning for Cancer Prognosis Prediction

MBFusion: Multi-modal balanced fusion and multi-task learning for cancer diagnosis and prognosis

Multimodal Fusion with Cross-attention Transformer for HCC Early Recurrence Prediction from Multi-Phase CT and Clinical Data

M<SUP>2</SUP>F: A Multi-modal and Multi-task Fusion Network for Glioma Diagnosis and Prognosis

Multi-modal Fusion Network with Intra- and Inter-Modality Attention for Prognosis Prediction in Breast Cancer

Pathomic Fusion: An Integrated Framework for Fusing Histopathology and Genomic Features for Cancer Diagnosis and Prognosis

Multimodal Survival Ensemble Network: Integrating Genomic and Histopathological Insights for Enhanced Cancer Prognosis.

Cross-modality Attention-based Multimodal Fusion for Non-small Cell Lung Cancer (NSCLC) Patient Survival Prediction

Multi-task multi-modal learning for joint diagnosis and prognosis of human cancers.

TTMFN: Two-stream Transformer-based Multimodal Fusion Network for Survival Prediction

SG-Fusion: A swin-transformer and graph convolution-based multi-modal deep neural network for glioma prognosis

MIF: Multi-Shot Interactive Fusion Model for Cancer Survival Prediction Using Pathological Image and Genomic Data

MFCSA-CAT: a Multimodal Fusion Method for Cancer Survival Analysis Based on Cross-Attention Transformer

Hierarchical multimodal fusion framework based on noisy label learning and attention mechanism for cancer classification with pathology and genomic features

A Multimodal Affinity Fusion Network for Predicting the Survival of Breast Cancer Patients

Graph attention-based fusion of pathology images and gene expression for prediction of cancer survival

Multi-modality fusion using canonical correlation analysis methods: Application in breast cancer survival prediction from histology and genomics

MMGCN: Multi-modal multi-view graph convolutional networks for cancer prognosis prediction

A Framework for Deep Multitask Learning With Multiparametric Magnetic Resonance Imaging for the Joint Prediction of Histological Characteristics in Breast Cancer

Knowledge-driven Subspace Fusion and Gradient Coordination for Multi-modal Learning

Cohort-Individual Cooperative Learning for Multimodal Cancer Survival Analysis