Abstract:Morphological features of cancer cell nuclei are linked to gene expression signatures and genomic alterations. In addition, pathologists have leveraged nuclear morphology as diagnostic and prognostic markers. To enable the use of nuclear morphology in digital pathology, we developed a pan-tissue, deep-learning-based digital pathology pipeline for exhaustive nucleus detection, instance segmentation, and classification. We collected > 29,000 manual nucleus annotations from hematoxylin and eosin (H&E)-stained pathology images from 21 tumor types at 40x and 20x magnification from The Cancer Genome Atlas (TCGA) project, as well as a proprietary set of H&E-stained tissue biopsies of skin, liver non-alcoholic steatohepatitis (NASH), colon inflammatory bowel disease (IBD), and kidney lupus. Annotations were used to train an object detection and segmentation model for identifying nuclei. Application of the model to held-out test data, including held-out tissue types, demonstrated performance comparable to state-of-the-art models described in the literature (mean Dice score = 0.80, aggregated Jaccard index = 0.60). We deployed our model to segment nuclei in H&E slides from the breast cancer (BRCA, N = 941) and prostate adenocarcinoma (PRAD, N = 457) TCGA cohorts. We extracted interpretable features describing the shape (circularity, eccentricity), size, staining intensity (mean and standard deviation), and texture of each nucleus. Nuclei were assigned as cancer or other cell types using separately trained convolutional neural networks for BRCA and PRAD. We used the mean and standard deviation of each feature sampled from a random subset of cancer nuclei to summarize the nuclear morphology on each slide (mean (range) = 10,068 (5,981-10,452) cancer cells from each BRCA slide; mean (range) = 10,053 (5,029-10,495) cancer cells from each PRAD slide). We used nuclear features to construct random forest classification models for predicting markers of genomic instability and prognosis: whole-genome doubling (WGD) and homologous recombination deficiency (HRD) status separately in BRCA and PRAD, HER2 subtype in BRCA, and Gleason grade in PRAD. Nuclear features were predictive of WGD (area under the receiver operating characteristic curve (AUROC) = 0.78 BRCA, = 0.69 PRAD) and binarized HRD status (AUROC = 0.65 BRCA, = 0.68 PRAD) on held-out test sets. Nuclear features were predictive of HER2-enriched breast cancer vs. other molecular subtypes (AUROC = 0.72), and distinguished between low risk (6) and moderate/high risk (7-10) Gleason grade in PRAD (AUROC = 0.72). In summary, we present a powerful pan-tissue approach for nucleus segmentation and featurization, which enables the construction of predictive models and the identification of features linking nuclear morphology with clinically-relevant prognostic biomarkers across multiple cancer types. Citation Format: John Abel, Suyog Jain, Deepta Rajan, Ken Leidal, Harshith Padigela, Aaditya Prakash, Jake Conway, Michael Nercessian, Christian Kirkup, Robert Egger, Ben Trotter, Andrew Beck, Ilan Wapinski, Michael G. Drage, Limin Yu, Amaro Taylor-Weiner. AI-powered segmentation and analysis of nuclei morphology predicts genomic and clinical markers in multiple cancer types [abstract]. In: Proceedings of the American Association for Cancer Research Annual Meeting 2022; 2022 Apr 8-13. Philadelphia (PA): AACR; Cancer Res 2022;82(12_Suppl) nr 464.

Abstract 5409: NuKit: a deep learning platform for histopathological Images

NuKit: A deep learning platform for fast nucleus segmentation of histopathological images

Evaluation Of Deep Learning Strategies For Nucleus Segmentation In Fluorescence Images

A Robust Deep Learning Approach for Joint Nuclei Detection and Cell Classification in Pan-Cancer Histology Images

ChampKit: A framework for rapid evaluation of deep neural networks for patch-based histopathology classification

Nuclei probability and centroid map network for nuclei instance segmentation in histology images

High-resolution deep transferred ASPPU-Net for nuclei segmentation of histopathology images

NuClick: A Deep Learning Framework for Interactive Segmentation of Microscopy Images

A survey on recent trends in deep learning for nucleus segmentation from histopathology images

NuInsSeg: A Fully Annotated Dataset for Nuclei Instance Segmentation in H&E-Stained Histological Images

NuLite -- Lightweight and Fast Model for Nuclei Instance Segmentation and Classification

A deep learning algorithm for one-step contour aware nuclei segmentation of histopathology images

Abstract 464: AI-powered segmentation and analysis of nuclei morphology predicts genomic and clinical markers in multiple cancer types

Attention augmented distance regression and classification network for nuclei instance segmentation and type classification in histology images

Data-Efficient Computational Pathology Platform for Faster and Cheaper Breast Cancer Subtype Identifications: Development of a Deep Learning Model

Nucleus segmentation from the histopathological images of liver cancer through an efficient deep learning framework

Deep Learning Model for Cell Nuclei Segmentation and Lymphocyte Identification in Whole Slide Histology Images

A Comprehensive Overview of Computational Nuclei Segmentation Methods in Digital Pathology

A deep learning-based toolkit for 3D nuclei segmentation and quantitative analysis in cellular and tissue context

A Deep Learning Approach for Histology-Based Nucleus Segmentation and Tumor Microenvironment Characterization