Abstract:Histopathological images are a rich but incompletely explored data type for studying cancer. Manual inspection is time consuming, making it challenging to use for image data mining. Here we show that convolutional neural networks (CNNs) can be systematically applied across cancer types, enabling comparisons to reveal shared spatial behaviors. We develop CNN architectures to analyze 27,815 hematoxylin and eosin scanned images from The Cancer Genome Atlas for tumor/normal, cancer subtype, and mutation classification. Our CNNs are able to classify TCGA pathologist-annotated tumor/normal status of whole slide images (WSIs) in 19 cancer types with consistently high AUCs (0.995 ± 0.008), as well as subtypes with lower but significant accuracy (AUC 0.87 ± 0.1). Remarkably, tumor/normal CNNs trained on one tissue are effective in others (AUC 0.88 ± 0.11), with classifier relationships also recapitulating known adenocarcinoma, carcinoma, and developmental biology. Moreover, classifier comparisons reveal intra-slide spatial similarities, with an average tile-level correlation of 0.45 ± 0.16 between classifier pairs. Breast cancers, bladder cancers, and uterine cancers have spatial patterns that are particularly easy to detect, suggesting these cancers can be canonical types for image analysis. Patterns for TP53 mutations can also be detected, with WSI self- and cross-tissue AUCs ranging from 0.65-0.80. Finally, we comparatively evaluate CNNs on 170 breast and colon cancer images with pathologist-annotated nuclei, finding that both cellular and intercellular regions contribute to CNN accuracy. These results demonstrate the power of CNNs not only for histopathological classification, but also for cross-comparisons to reveal conserved spatial behaviors across tumors.

Sampling Based Tumor Recognition in Whole-Slide Histology Image with Deep Learning Approaches.

Fast Tumor Detector in Whole-Slide Image With Dynamic Programing Based Monte Carlo Sampling

Representative Region Based Active Learning for Histological Classification of Colorectal Cancer

Deep Learning-Based Semantic Segmentation of Non-Melanocytic Skin Tumors in Whole-Slide Histopathological Images.

Identify Representative Samples by Conditional Random Field of Cancer Histology Images

High-throughput adaptive sampling for whole-slide histopathology image analysis (HASHI) via convolutional neural networks: Application to invasive breast cancer detection

Accurate and reproducible invasive breast cancer detection in whole-slide images: A Deep Learning approach for quantifying tumor extent

Deep Learning Classification of Colorectal Lesions Based on Whole Slide Images

Whole-slide-imaging Cancer Metastases Detection and Localization with Limited Tumorous Data

Detection and Localization of Melanoma Skin Cancer in Histopathological Whole Slide Images

Multi-class Breast Tumor Region Detection for Gigapixel Pathology Images Using Deep Neural Network with Rescale Approach

A Deep Learning Approach for Tissue Spatial Quantification and Genomic Correlations of Histopathological Images

A novel deep learning-based algorithm combining histopathological features with tissue areas to predict colorectal cancer survival from whole-slide images

Patch-based Convolutional Neural Network for Whole Slide Tissue Image Classification

Deep learning-based cross-classifications reveal conserved spatial behaviors within tumor histological images

Overcoming the limitations of patch-based learning to detect cancer in whole slide images

Fast whole-slide cartography in colon cancer histology using superpixels and CNN classification

Identifying Molecular Subtypes of Whole-Slide Image in Colorectal Cancer Via Deep Learning

Computer-aided Detection of Squamous Carcinoma of the Cervix in Whole Slide Images

Multiscale Detection of Cancerous Tissue in High Resolution Slide Scans

Diagnosis and Survival Prediction of Colorectal Cancer Using Deep Learning from the Histopathological HE Images