A Scalable Graph-Based Framework for Multi-Organ Histology Image Classification
Yu Bai,Yue Mi,Yihan Su,Bo Zhang,Zheng Zhang,Jingyun Wu,Haiwen Huang,Yongping Xiong,Xiangyang Gong,Wendong Wang
DOI: https://doi.org/10.1109/jbhi.2022.3199110
IF: 7.7
2022-11-01
IEEE Journal of Biomedical and Health Informatics
Abstract:Graph-based approaches are successful for histology image classification tasks but still face many challenges, such as: 1) the lack of nuclei-level labels and the significant variations between histology images make it extremely difficult to extract discriminative high-level nuclei features like nuclei type, texture and micro-environment; 2) graph-based approaches cannot handle large-scale cell graph nodes typically contained in histology images; and 3) graph neural networks (GNNs) struggle to learn the long-range dependency of cell graphs. To address the above challenges, we propose a scalable graph-based framework for multi-organ histology image classification. We develop a two-step masked nuclei patches supervised training approach to extract discriminative high-level nuclei features for histology images without nuclei-level labels. Additionally, we introduce a nuclei sampling strategy to make our graph-based framework scalable for large-scale cell graphs. Furthermore, we propose HierArchical Transformer Graph Neural Network (HAT-Net+) for cell graph classi- fications. HAT-Net+ adopts Transformer to model the long-range dependency of cell graphs and a parameter-free approach to adaptively fuse different hierarchical graph representations of each layer. We achieved the state-of-the-art results on four public histology image classification datasets: CRC dataset (100), Extended CRC dataset (98), UZH dataset (96.9) and BACH dataset (88). Unlike other methods, our approach can be used in various histology image classification tasks, even for images without nuclei-level labels, indicating its potential in cancer diagnosis. The code is available at https://github.com/suyouooooo/HAT-Net.
computer science, interdisciplinary applications,mathematical & computational biology,medical informatics, information systems