Abstract:Cancer diagnosis, prognosis, and therapy response predictions from tissue specimens highly depend on the phenotype and topological distribution of constituting histological entities. Thus, adequate tissue representations for encoding histological entities is imperative for computer aided cancer patient care. To this end, several approaches have leveraged cell-graphs, capturing the cell-microenvironment, to depict the tissue. These allow for utilizing graph theory and machine learning to map the tissue representation to tissue functionality, and quantify their relationship. Though cellular information is crucial, it is incomplete alone to comprehensively characterize complex tissue structure. We herein treat the tissue as a hierarchical composition of multiple types of histological entities from fine to coarse level, capturing multivariate tissue information at multiple levels. We propose a novel multi-level hierarchical entity-graph representation of tissue specimens to model the hierarchical compositions that encode histological entities as well as their intra- and inter-entity level interactions. Subsequently, a hierarchical graph neural network is proposed to operate on the hierarchical entity-graph and map the tissue structure to tissue functionality. Specifically, for input histology images, we utilize well-defined cells and tissue regions to build HierArchical Cell-to-Tissue (HACT) graph representations, and devise HACT-Net, a message passing graph neural network, to classify the HACT representations. As part of this work, we introduce the BReAst Carcinoma Subtyping (BRACS) dataset, a large cohort of Haematoxylin & Eosin stained breast tumor regions-of-interest, to evaluate and benchmark our proposed methodology against pathologists and state-of-the-art computer-aided diagnostic approaches. Through comparative assessment and ablation studies, our proposed method is demonstrated to yield superior classification results compared to alternative methods as well as individual pathologists. The code, data, and models can be accessed at https://github.com/histocartography/hact-net.

A Scalable Graph-Based Framework for Multi-Organ Histology Image Classification

A Multi-scale Graph Network with Multi-head Attention for Histopathology Image Diagnosisn

Spatial-Hierarchical Graph Neural Network with Dynamic Structure Learning for Histological Image Classification

HoVer-Net: Simultaneous Segmentation and Classification of Nuclei in Multi-Tissue Histology Images

Real-time and accurate deep learning-based multi-organ nucleus segmentation in histology images

Detecting and Classifying Nuclei Using Multi-Scale Fully Convolutional Network.

HACT-Net: A Hierarchical Cell-to-Tissue Graph Neural Network for Histopathological Image Classification

Dual-branch hybrid encoding embedded network for histopathology image classification.

Supervised Graph Hashing for Histopathology Image Retrieval and Classification

Large Scale Tissue Histopathology Image Classification, Segmentation, and Visualization Via Deep Convolutional Activation Features

Cascaded Graph Convolution Approach for Nuclei Detection in Histopathology Images

Improving Histopathological Image Segmentation and Classification Using Graph Convolution Network.

Hierarchical graph representations in digital pathology

TSHVNet: Simultaneous Nuclear Instance Segmentation and Classification in Histopathological Images Based on Multiattention Mechanisms

A Hierarchical Graph V-Net with Semi-Supervised Pre-Training for Histological Image Based Breast Cancer Classification.

When Machine Vision Meets Histology: A Comparative Evaluation of Model Architecture for Classification of Histology Sections

Graph Neural Networks for the Cross-Domain Histopathological Image Classification

Exploiting Geometric Features Via Hierarchical Graph Pyramid Transformer for Cancer Diagnosis Using Histopathological Images

Improving High Resolution Histology Image Classification with Deep Spatial Fusion Network

Structure Embedded Nucleus Classification for Histopathology Images

Nuclei Classification Using Dual View CNNs with Multi-crop Module in Histology Images.