Abstract:Background and objectives: Graph neural network (GNN) has been extensively used in histopathology whole slide image (WSI) analysis due to the efficiency and flexibility in modelling relationships among entities. However, most existing GNN-based WSI analysis methods only consider the pairwise correlation of patches from one single perspective (e.g. spatial affinity or embedding similarity) yet ignore the intrinsic non-pairwise relationships present in gigapixel WSI, which are likely to contribute to feature learning and downstream tasks. The objective of this study is therefore to explore the non-pairwise relationships in histopathology WSI and exploit them to guide the learning of slide-level representations for better classification performance. Methods: In this paper, we propose a novel Masked HyperGraph Learning (MaskHGL) framework for weakly supervised histopathology WSI classification. Compared with most GNN-based WSI classification methods, MaskHGL exploits the non-pairwise correlations between patches with hypergraph and global message passing conducted by hypergraph convolution. Concretely, multi-perspective hypergraphs are first built for each WSI, then hypergraph attention is introduced into the jointed hypergraph to propagate the non-pairwise relationships and thus yield more discriminative node representation. More importantly, a masked hypergraph reconstruction module is devised to guide the hypergraph learning which can generate more powerful robustness and generalization than the method only using hypergraph modelling. Additionally, a self-attention-based node aggregator is also applied to explore the global correlation of patches in WSI and produce the slide-level representation for classification. Results: The proposed method is evaluated on two public TCGA benchmark datasets and one in-house dataset. On the public TCGA-LUNG (1494 WSIs) and TCGA-EGFR (696 WSIs) test set, the area under receiver operating characteristic (ROC) curve (AUC) were 0.9752±0.0024 and 0.7421±0.0380, respectively. On the USTC-EGFR (754 WSIs) dataset, MaskHGL achieved significantly better performance with an AUC of 0.8745±0.0100, which surpassed the second-best state-of-the-art method SlideGraph+ 2.64%. Conclusions: MaskHGL shows a great improvement, brought by considering the intrinsic non-pairwise relationships within WSI, in multiple downstream WSI classification tasks. In particular, the designed masked hypergraph reconstruction module promisingly alleviates the data scarcity and greatly enhances the robustness and classification ability of our MaskHGL. Notably, it has shown great potential in cancer subtyping and fine-grained lung cancer gene mutation prediction from hematoxylin and eosin (H&E) stained WSIs.

Generating Hypergraph-Based High-Order Representations of Whole-Slide Histopathological Images for Survival Prediction

Big-Hypergraph Factorization Neural Network for Survival Prediction From Whole Slide Image

GraphLSurv: A Scalable Survival Prediction Network with Adaptive and Sparse Structure Learning for Histopathological Whole-Slide Images

A Hierarchical Hypergraph Attention Network for Survival Analysis from Pathological Images

Ranking-Based Survival Prediction on Histopathological Whole-Slide Images

Multi-scope Analysis Driven Hierarchical Graph Transformer for Whole Slide Image Based Cancer Survival Prediction

Hierarchical Transformer for Survival Prediction Using Multimodality Whole Slide Images and Genomics

Wsisa: Making Survival Prediction From Whole Slide Histopathological Images

Masked hypergraph learning for weakly supervised histopathology whole slide image classification

Self-supervised learning-based Multi-Scale feature Fusion Network for survival analysis from whole slide images

Pathology-genomic fusion via biologically informed cross-modality graph learning for survival analysis

Multi-Scale Heterogeneity-Aware Hypergraph Representation for Histopathology Whole Slide Images

HVTSurv: Hierarchical Vision Transformer for Patient-Level Survival Prediction from Whole Slide Image

An Explainable Coarse-to-Fine Survival Analysis Method on Multi-Center Whole Slide Images.

HSG-MGAF Net: Heterogeneous Subgraph-Guided Multiscale Graph Attention Fusion Network for Interpretable Prediction of Whole-Slide Image

Annotations-free Survival Prediction with WSIs Using Graph Convolutional Neural Networks.

Survival Prediction Via Hierarchical Multimodal Co-Attention Transformer: A Computational Histology-Radiology Solution.

Whole Slide Images based Cancer Survival Prediction using Attention Guided Deep Multiple Instance Learning Networks

Dual-stream multi-dependency graph neural network enables precise cancer survival analysis

MGCT: Mutual-Guided Cross-Modality Transformer for Survival Outcome Prediction using Integrative Histopathology-Genomic Features

Slide-based Graph Collaborative Training for Histopathology Whole Slide Image Analysis