Abstract:Breast cancer prognostication guides treatment decisions, with transcriptomic assays like Prosigna and Oncotype DX providing valuable risk of recurrence (ROR) scores in ER+/HER2– patients. However, these tests are costly and require sufficient biopsy tissue for accuracy. Pathology images are routinely available and could expedite rapid risk stratification at lower cost. We propose a multi-modal, multi-task deep neural network that learns transcriptomic ROR using digital pathology images and hormone receptor expression status. Because the incorporation of both pathology images and clinical data has proven beneficial in other deep learning models, we hypothesized that incorporation of this complementary information would improve ROR predictions compared to using pathology images alone. We modified the "clustering-constrained-attention multiple-instance learning" (CLAM) deep learning method to perform multi-task regression and infer continuous risk scores (instead of discrete risk categories). We evaluated (1) PAM50 ROR-P, calculated using a multivariate model based on intrinsic subtype centroids and a proliferation score, and (2) a 21-gene assay recurrence score (RS) (research version of Oncotype DX), calculated from component scores relating to proliferation, ER, HER2, and invasion. Using data from 899 patients in the Carolina Breast Cancer Study, a population-based study of diverse patients including 50% Black women and 50% women under age 50, we constructed training (n=714), validation (n=93), and test (n=92) sets. Genomic data were assayed by Nanostring, and digital pathology was based on H&E-stained whole slide images (WSIs). Models were assessed using Pearson correlation between measured and predicted ROR-P or RS. The model with the highest correlations on the test set used H&E WSIs and ER/PR/HER2 expression status as inputs and generated 10 independent outputs: ROR-P, four PAM50 centroid correlations (i.e., Basal, HER2-enriched, Luminal A, and Luminal B), RS, and four RS component scores. Prediction was strong when considering all participants in the test set (ROR-P, 0.78; RS, 0.83, N, 92) but was reduced among more clinically homogeneous ER+/HER2– patients (ROR-P, 0.55; RS, 0.49; N, 49). For RS, correlations were higher in models that included ER/PR/HER2 (Overall, 0.83; ER+/HER2–, 0.50) than in those that relied on pathology alone (Overall, 0.60, ER+/HER2–, 0.34). Future models will consider strategies to enhance performance in ER+/HER2– patients, such as oversampling of ER+/HER2– patients and use of loss functions designed to focus learning on mis-predicted examples. In summary, multi-task, tissue-based regression deep learning models recapitulate transcription-based risk assays with high correlations and, with optimization (especially integration of additional clinicopathologic data), hold promise for personalized treatment decisions from early, routinely-collected biopsy images. Citation Format: Jakub R Kaczmarzyk, Luke A Torre-Healy, Richard A Moffitt, Rajarsi Gupta, Alina M Hamilton, Tahsin M Kurc, Katherine A Hoadley, Melissa A Troester, Joel H Saltz. Early risk stratification of ER+/HER2– breast cancer patients using digital pathology and multi-task, weakly-supervised deep learning [abstract]. In: Proceedings of the AACR Special Conference in Cancer Research: Advances in Breast Cancer Research; 2023 Oct 19-22; San Diego, California. Philadelphia (PA): AACR; Cancer Res 2024;84(3 Suppl_1) nr B082.

Clinical evaluation of deep learning-based risk profiling in breast cancer histopathology and comparison to an established multigene assay

Validation of an AI-based solution for breast cancer risk stratification using routine digital histopathology images

Deep learning-based risk stratification of preoperative breast biopsies using digital whole slide images

Abstract B082: Early risk stratification of ER+/HER2– breast cancer patients using digital pathology and multi-task, weakly-supervised deep learning

Computational Pathology Improves Risk Stratification of a Multi-Gene Assay for Early Stage ER+ Breast Cancer

Abstract PO3-07-05: Multi-site validation of a deep learning solution for ER/PR profiling of breast cancer from H&E-stained pathology slides

A blind validation of a deep-learning solution providing HER2, ER, and PR results from H&E-stained breast cancer specimens.

Deep Learning and Digital Pathology Power Precision Treatment of Triple-Negative Breast Cancer

Development and deployment of a histopathology-based deep learning algorithm for patient prescreening in a clinical trial

Multi-modal AI for comprehensive breast cancer prognostication

Multi-stain modelling of histopathology slides for breast cancer prognosis prediction

Early Breast Cancer Risk Assessment: Integrating Histopathology with Artificial Intelligence

Determining breast cancer biomarker status and associated morphological features using deep learning

Screen Them All: High-Throughput Pan-Cancer Genetic and Phenotypic Biomarker Screening from H&E Whole Slide Images

Abstract 890: Multimodal modeling of digitized histopathology slides improves risk stratification in hormone receptor-positive breast cancer patients

A multi-model approach integrating whole-slide imaging and clinicopathologic features to predict breast cancer recurrence risk

Predicting Breast Cancer Chemotherapeutic Response Using a Novel Tool for Microarray Data Analysis.

Artificial Intelligence Predicts Multiclass Molecular Signatures and Subtypes Directly from Breast Cancer Histology

Prediction of Breast Cancer Recurrence Risk Using a Multi-Model Approach Integrating Whole Slide Imaging and Clinicopathologic Features

Integration of clinical features and deep learning on pathology for the prediction of breast cancer recurrence assays and risk of recurrence

Uncovering the Prognostic Gene Signatures for the Improvement of Risk Stratification in Cancers by Using Deep Learning Algorithm Coupled with Wavelet Transform