Abstract:Abstract Deep learning is a powerful tool for assessing pathology data obtained from digitized biopsy slides. In the context of supervised learning, most methods typically divide a whole slide image (WSI) into patches, aggregate convolutional neural network outcomes on them and estimate overall disease grade. However, patch-based methods introduce label noise in training by assuming that each patch is independent with the same label as the WSI and neglect the important contextual information that is significant in disease grading. Here we present a Graph-Transformer (GT) based framework for processing pathology data, called GTP, that interprets morphological and spatial information at the WSI-level to predict disease grade. To demonstrate the applicability of our approach, we selected 3,024 hematoxylin and eosin WSIs of lung tumors and with normal histology from the Clinical Proteomic Tumor Analysis Consortium, the National Lung Screening Trial, and The Cancer Genome Atlas, and used GTP to distinguish adenocarcinoma (LUAD) and squamous cell carcinoma (LSCC) from those that have normal histology. Our model achieved consistently high performance on binary (tumor versus normal: mean overall accuracy = 0.975 ± 0.013) as well as three-label (normal versus LUAD versus LSCC: mean accuracy = 0.932 ± 0.019) classification on held-out test data, underscoring the power of GT-based deep learning for WSI-level classification. We also introduced a graphbased saliency mapping technique, called GraphCAM, that captures regional as well as contextual information and allows our model to highlight WSI regions that are highly associated with the class label. Taken together, our findings demonstrate GTP as a novel interpretable and effective deep learning framework for WSI-level classification.

An efficient context-aware approach for whole slide image classification

Automatic Whole Slide Pathology Image Diagnosis Framework Via Unit Stochastic Selection and Attention Fusion

Slide-based Graph Collaborative Training for Histopathology Whole Slide Image Analysis

WSI-VQA: Interpreting Whole Slide Images by Generative Visual Question Answering

Gigapixel Whole-Slide Images Classification using Locally Supervised Learning

PATHS: A Hierarchical Transformer for Efficient Whole Slide Image Analysis

Multi-class Cancer Classification of Whole Slide Images Through Transformer and Multiple Instance Learning.

Interpretable Classification of Pathology Whole-Slide Images Using Attention Based Context-Aware Graph Convolutional Neural Network

Generalizable Whole Slide Image Classification with Fine-Grained Visual-Semantic Interaction

A graph-transformer for whole slide image classification

A deep learning based graph-transformer for whole slide image classification

Enhancing Whole Slide Image Classification with Discriminative and Contrastive Learning

Robust whole slide image analysis for cervical cancer screening using deep learning

Trusted multi-scale classification framework for whole slide image

A self-supervised framework for learning whole slide representations

CWC-transformer: a visual transformer approach for compressed whole slide image classification

HIGT: Hierarchical Interaction Graph-Transformer for Whole Slide Image Analysis

An Efficient Cervical Whole Slide Image Analysis Framework Based on Multi-scale Semantic and Location Deep Features

Efficient Whole Slide Image Classification through Fisher Vector Representation

WsiCaption: Multiple Instance Generation of Pathology Reports for Gigapixel Whole-Slide Images

PathTR: Context-Aware Memory Transformer for Tumor Localization in Gigapixel Pathology Images.