Abstract:Graph-based approaches are successful for histology image classification tasks but still face many challenges, such as: 1) the lack of nuclei-level labels and the significant variations between histology images make it extremely difficult to extract discriminative high-level nuclei features like nuclei type, texture and micro-environment; 2) graph-based approaches cannot handle large-scale cell graph nodes typically contained in histology images; and 3) graph neural networks (GNNs) struggle to learn the long-range dependency of cell graphs. To address the above challenges, we propose a scalable graph-based framework for multi-organ histology image classification. We develop a two-step masked nuclei patches supervised training approach to extract discriminative high-level nuclei features for histology images without nuclei-level labels. Additionally, we introduce a nuclei sampling strategy to make our graph-based framework scalable for large-scale cell graphs. Furthermore, we propose HierArchical Transformer Graph Neural Network (HAT-Net+) for cell graph classi- fications. HAT-Net+ adopts Transformer to model the long-range dependency of cell graphs and a parameter-free approach to adaptively fuse different hierarchical graph representations of each layer. We achieved the state-of-the-art results on four public histology image classification datasets: CRC dataset (100), Extended CRC dataset (98), UZH dataset (96.9) and BACH dataset (88). Unlike other methods, our approach can be used in various histology image classification tasks, even for images without nuclei-level labels, indicating its potential in cancer diagnosis. The code is available at https://github.com/suyouooooo/HAT-Net.

Compositional Boosting for Computing Hierarchical Image Structures

Learning to Infer Unseen Single-/ Multi-Attribute-Object Compositions with Graph Networks.

Large-scale Outdoor Scene Classification by Boosting a Set of Highly Discriminative and Low Redundant Graphlets

Structural Data Recognition with Graph Model Boosting

Object Category Recognition Using Generative Template Boosting.

Towards a Unified Compositional Model for Visual Pattern Modeling

AdaBoostNet: An Efficient Hierarchical Neural Network for Image Classification

Learning a Probabilistic Topology Discovering Model for Scene Categorization.

Attributed Graph Subspace Clustering with Graph-Boosting.

Layered Graph Matching by Composite Cluster Sampling with Collaborative and Competitive Interactions

Layered Graph Matching with Composite Cluster Sampling

MMF: Multi-Task Multi-Structure Fusion for Hierarchical Image Classification

Hierarchical Relationships: A New Perspective to Enhance Scene Graph Generation

Incorporating Structural Alternatives and Sharing into Hierarchy for Multiclass Object Recognition and Detection

Unsupervised Learning of Dictionaries of Hierarchical Compositional Models

Learning Unseen Concepts Via Hierarchical Decomposition and Composition

Hierarchical Image Segmentation Based on Multi-feature Fusion and Graph Cut Optimization.

Boosting Point Cloud Understanding Through Graph Convolutional Network with Scale Measurement and High-Frequency Enhancement

A stochastic graph grammar for compositional object representation and recognition

Boosting Scene Understanding by Hierarchical Pachinko Allocation

A Scalable Graph-Based Framework for Multi-Organ Histology Image Classification