Abstract 464: AI-powered segmentation and analysis of nuclei morphology predicts genomic and clinical markers in multiple cancer types

John Abel,Suyog Jain,Deepta Rajan,Ken Leidal,Harshith Padigela,Aaditya Prakash,Jake Conway,Michael Nercessian,Christian Kirkup,Robert Egger,Ben Trotter,Andrew Beck,Ilan Wapinski,Michael G. Drage,Limin Yu,Amaro Taylor-Weiner

DOI: https://doi.org/10.1158/1538-7445.am2022-464

IF: 11.2

2022-06-17

Cancer Research

Abstract:Morphological features of cancer cell nuclei are linked to gene expression signatures and genomic alterations. In addition, pathologists have leveraged nuclear morphology as diagnostic and prognostic markers. To enable the use of nuclear morphology in digital pathology, we developed a pan-tissue, deep-learning-based digital pathology pipeline for exhaustive nucleus detection, instance segmentation, and classification. We collected > 29,000 manual nucleus annotations from hematoxylin and eosin (H&E)-stained pathology images from 21 tumor types at 40x and 20x magnification from The Cancer Genome Atlas (TCGA) project, as well as a proprietary set of H&E-stained tissue biopsies of skin, liver non-alcoholic steatohepatitis (NASH), colon inflammatory bowel disease (IBD), and kidney lupus. Annotations were used to train an object detection and segmentation model for identifying nuclei. Application of the model to held-out test data, including held-out tissue types, demonstrated performance comparable to state-of-the-art models described in the literature (mean Dice score = 0.80, aggregated Jaccard index = 0.60). We deployed our model to segment nuclei in H&E slides from the breast cancer (BRCA, N = 941) and prostate adenocarcinoma (PRAD, N = 457) TCGA cohorts. We extracted interpretable features describing the shape (circularity, eccentricity), size, staining intensity (mean and standard deviation), and texture of each nucleus. Nuclei were assigned as cancer or other cell types using separately trained convolutional neural networks for BRCA and PRAD. We used the mean and standard deviation of each feature sampled from a random subset of cancer nuclei to summarize the nuclear morphology on each slide (mean (range) = 10,068 (5,981-10,452) cancer cells from each BRCA slide; mean (range) = 10,053 (5,029-10,495) cancer cells from each PRAD slide). We used nuclear features to construct random forest classification models for predicting markers of genomic instability and prognosis: whole-genome doubling (WGD) and homologous recombination deficiency (HRD) status separately in BRCA and PRAD, HER2 subtype in BRCA, and Gleason grade in PRAD. Nuclear features were predictive of WGD (area under the receiver operating characteristic curve (AUROC) = 0.78 BRCA, = 0.69 PRAD) and binarized HRD status (AUROC = 0.65 BRCA, = 0.68 PRAD) on held-out test sets. Nuclear features were predictive of HER2-enriched breast cancer vs. other molecular subtypes (AUROC = 0.72), and distinguished between low risk (6) and moderate/high risk (7-10) Gleason grade in PRAD (AUROC = 0.72). In summary, we present a powerful pan-tissue approach for nucleus segmentation and featurization, which enables the construction of predictive models and the identification of features linking nuclear morphology with clinically-relevant prognostic biomarkers across multiple cancer types. Citation Format: John Abel, Suyog Jain, Deepta Rajan, Ken Leidal, Harshith Padigela, Aaditya Prakash, Jake Conway, Michael Nercessian, Christian Kirkup, Robert Egger, Ben Trotter, Andrew Beck, Ilan Wapinski, Michael G. Drage, Limin Yu, Amaro Taylor-Weiner. AI-powered segmentation and analysis of nuclei morphology predicts genomic and clinical markers in multiple cancer types [abstract]. In: Proceedings of the American Association for Cancer Research Annual Meeting 2022; 2022 Apr 8-13. Philadelphia (PA): AACR; Cancer Res 2022;82(12_Suppl) nr 464.

oncology