Abstract:e16501 Background: Survival prediction of cancer patients has always been an challenging problem.Tumor microenvironment (TME) Analyzation based on whole-slide-images (WSIs) has provide an effective perspective for survival prediction. However, most existing TME analyzation based on cell segmentation or classification relies heavily on labor-intensive cell-level annotations of pathologists. Furthermore, except for each individual cell or local pathological feature, survival prediction also involves local-level pathological features interactions in tumor microenvironments. This requires context-awareness based on histological features to fully infer the patient's survival risk. Therefore, we explored a model based on graph convolutional neural networks (GCNN) to perform survival prediction of cancer patients using WSIs. Methods: We utilized WSIs collected from The Cancer Genome Atlas (TCGA) to develop a graph convolutional neural network for survival prediction of cancer patients. The model leverages the advantages of graph structures to autonomously learn the histopathological contextual features in WSIs, and therefore can incorporate additional and crucial tumor microenvironment interaction information while avoiding the labor-intensive annotations. WSIs of two different cancers, KIRC and LUAD, were randomly divided into training, validation, and testing sets in a ratio of 7:1:2. The performance of the constructed model is evaluated using the test set and the results are compared with other state-of-art methods. Results: Our work is compared with other state-of-art weakly supervised learning methods for survival prediction in computational pathology. Abundant experimental results shown that our method outperformed previous methods on these two cancer types (achieving a 2.9% improvement compared to Multiple Instance Learning (MIL) and a 2.6% improvement compared to Attention MIL), with an overall c-index of 0.646. Additionally, we evaluated the interpretability of our model through attention heatmaps of low-risk and high-risk patients. Conclusions: We have developed a GCNN based model, combined with attention mechanisms, to learn features of heterogeneous tumor microenvironments and their contextual information from memory-efficient representations of highly correlated image patches for cancer patients survival prediction. This model is applicable to any weakly supervised learning task in computational pathology that involves slide-level or patient-level labels, making it an effective supplementary diagnostic tool for oncologists and pathologists.

Multimodal Optimal Transport-based Co-Attention Transformer with Global Structure Consistency for Survival Prediction

Multimodal Co-Attention Transformer for Survival Prediction in Gigapixel Whole Slide Images

Generating Hypergraph-Based High-Order Representations of Whole-Slide Histopathological Images for Survival Prediction

Survival Prediction Via Hierarchical Multimodal Co-Attention Transformer: A Computational Histology-Radiology Solution.

Hierarchical Transformer for Survival Prediction Using Multimodality Whole Slide Images and Genomics

Multimodal Cross-Task Interaction for Survival Analysis in Whole Slide Pathological Images

MGCT: Mutual-Guided Cross-Modality Transformer for Survival Outcome Prediction using Integrative Histopathology-Genomic Features

Pathology-and-genomics Multimodal Transformer for Survival Outcome Prediction

Multimodal Survival Ensemble Network: Integrating Genomic and Histopathological Insights for Enhanced Cancer Prognosis.

TransSurv: Transformer-Based Survival Analysis Model Integrating Histopathological Images and Genomic Data for Colorectal Cancer.

TTMFN: Two-stream Transformer-based Multimodal Fusion Network for Survival Prediction

Multi-scope Analysis Driven Hierarchical Graph Transformer for Whole Slide Image Based Cancer Survival Prediction

Modeling Dense Multimodal Interactions Between Biological Pathways and Histology for Survival Prediction

Transformer-Based Multimodal Fusion for Survival Prediction by Integrating Whole Slide Images, Clinical, and Genomic Data

HVTSurv: Hierarchical Vision Transformer for Patient-Level Survival Prediction from Whole Slide Image

Path-GPTOmic: A Balanced Multi-modal Learning Framework for Survival Outcome Prediction

Multimodal Prototyping for cancer survival prediction

Annotations-free Survival Prediction with WSIs Using Graph Convolutional Neural Networks.

Learning Comprehensive Multimodal Representation for Cancer Survival Prediction

CoADS: Cross Attention Based Dual-Space Graph Network for Survival Prediction of Lung Cancer Using Whole Slide Images.

Self-Supervised Learning Guided Transformer for Survival Prediction of Lung Cancer Using Pathological Images.