Abstract:In computational pathology, extracting spatial features from gigapixel whole slide images (WSIs) is a fundamental task, but due to their large size, WSIs are typically segmented into smaller tiles. A critical aspect of this analysis is aggregating information from these tiles to make predictions at the WSI level. We introduce a model that combines a message-passing graph neural network (GNN) with a state space model (Mamba) to capture both local and global spatial relationships among the tiles in WSIs. The model's effectiveness was demonstrated in predicting progression-free survival among patients with early-stage lung adenocarcinomas (LUAD). We compared the model with other state-of-the-art methods for tile-level information aggregation in WSIs, including tile-level information summary statistics-based aggregation, multiple instance learning (MIL)-based aggregation, GNN-based aggregation, and GNN-transformer-based aggregation. Additional experiments showed the impact of different types of node features and different tile sampling strategies on the model performance. This work can be easily extended to any WSI-based analysis. Code: <a class="link-external link-https" href="https://github.com/rina-ding/gat-mamba" rel="external noopener nofollow">this https URL</a>.

What problem does this paper attempt to address?

This paper attempts to address the problem of how to extract spatial features from Whole Slide Images (WSIs) in computational pathology and effectively integrate these features when predicting the Progression-Free Survival (PFS) of early-stage non-small cell lung cancer (NSCLC) patients. ### Specific Problems: 1. **Extracting Spatial Features**: Due to the large size of WSIs, they are usually divided into smaller tiles. How to extract and integrate spatial features from these tiles to make predictions at the WSI level. 2. **Integrating Local and Global Relationships**: Existing methods often capture only local spatial relationships when integrating tile information, failing to capture global long-distance dependencies. How to combine local and global spatial relationships to improve prediction accuracy. 3. **Improving Prediction Performance**: How to surpass existing methods in predicting the Progression-Free Survival of early-stage lung adenocarcinoma (LUAD) patients by combining Graph Neural Network (GNN) and State Space Model (Mamba). ### Background: - **Non-Small Cell Lung Cancer (NSCLC)**: One of the most common types of lung cancer. Early-stage patients usually undergo radical surgery, but about 30-55% of patients experience disease recurrence within 5 years post-surgery. - **Current Clinical Standard**: The TNM staging system has limitations in assessing tumor size, local invasion, lymph node metastasis, and distant metastasis, making it ineffective for risk stratification. - **Quantitative Tissue Morphological Features**: Recent studies have shown that quantitative tissue morphological features extracted from H&E-stained WSIs can be used to predict recurrence or survival in early-stage NSCLC patients. ### Methodological Innovations: - **Combining GNN and Mamba**: The paper proposes a method that combines message-passing GNN and Mamba state space model to capture both local and global spatial relationships between tiles in WSIs. - **Systematic Exploration of Node Features and Sampling Strategies**: Investigated the impact of different node features and tile sampling strategies on model performance. - **Experimental Validation**: Extensive experiments were conducted using two public datasets to compare the performance of the proposed method with existing baseline models. ### Objectives: - **Improve Prediction Accuracy**: Enhance the accuracy of predicting the Progression-Free Survival of early-stage LUAD patients by combining local and global spatial relationships. - **Expand Application Scope**: The method can be easily extended to other WSI-based analysis tasks.

Combining Graph Neural Network and Mamba to Capture Local and Global Tissue Spatial Relationships in Whole Slide Images

Generating Hypergraph-Based High-Order Representations of Whole-Slide Histopathological Images for Survival Prediction

SurvMamba: State Space Model with Multi-grained Multi-modal Interaction for Survival Prediction

MGCT: Mutual-Guided Cross-Modality Transformer for Survival Outcome Prediction using Integrative Histopathology-Genomic Features

MERGE: Multi-faceted Hierarchical Graph-based GNN for Gene Expression Prediction from Whole Slide Histopathology Images

Marrying Global-Local Spatial Context for Image Patches in Computer-Aided Assessment.

How GNNs Facilitate CNNs in Mining Geometric Information from Large-Scale Medical Images.

Integrating multiscale topology in digital pathology with pyramidal graph convolutional networks

Gigapixel Whole-Slide Images Classification using Locally Supervised Learning

Annotations-free Survival Prediction with WSIs Using Graph Convolutional Neural Networks.

Hierarchical Transformer for Survival Prediction Using Multimodality Whole Slide Images and Genomics

Unleash the Power of State Space Model for Whole Slide Image with Local Aware Scanning and Importance Resampling

MLAGG-Net: Multi-level aggregation and global guidance network for pancreatic lesion segmentation in histopathological images

Multi-scale representation attention based deep multiple instance learning for gigapixel whole slide image analysis

Mamba2MIL: State Space Duality Based Multiple Instance Learning for Computational Pathology

Dual-stream multi-dependency graph neural network enables precise cancer survival analysis

MAg: a simple learning-based patient-level aggregation method for detecting microsatellite instability from whole-slide images

Self-supervised learning-based Multi-Scale feature Fusion Network for survival analysis from whole slide images

PathoGraph: An Attention-Based Graph Neural Network Capable of Prognostication Based on CD276 Labelling of Malignant Glioma Cells

Survival Prediction Based on Histopathology Imaging and Clinical Data: A Novel, Whole Slide CNN Approach

Topological Feature Extraction and Visualization of Whole Slide Images using Graph Neural Networks