Abstract:Checkpoint blockade immunotherapy is a cornerstone of lung cancer treatment, but there is a need to improve the identification of patients who will respond favorably. Here, we explored a deep learning approach to predict immunotherapy outcomes from hematoxylin and eosin (H&E) images in non-small cell lung cancer (NSCLC). We included 150 unique cases with metastatic NSCLC (113 adenocarcinoma, 29 squamous cell, 8 other) treated with anti-PD-1/PD-L1 immunotherapy (56 nivolumab, 49 atezolizumab, 44 pembrolizumab, 1 durvalumab) as mono or combination (14 with chemotherapy, 1 with ipilimumab) therapy in a single institution. Each case consisted of a representative H&E whole slide image (53 biopsies, 50 needle core biopsies, 47 resections) obtained prior to immunotherapy, and the outcome reported as the 1-year overall survival (OS). PD-L1 status (tumor proportion score ≥ 1%) was known for 70 cases. We preprocessed the H&E images using two deep learning models previously developed using The Cancer Genome Atlas dataset. First, we used a classification model to identify tumor regions and randomly sampled a fixed number of tumor patches for each case. Then, we used a self-supervised pathology foundation model to obtain a compressed visual representation of each patch, known as an embedding. Next, using our dataset, we trained a deep multiple instance learning (DeepMIL) model with a gated attention mechanism to predict the binary 1-year OS status (0=deceased, 1=alive) for each case. As a baseline, we also trained a linear-probe (logistic regression) model using the averaged embeddings. Given the small dataset size, 5-fold cross-validation was used to train and evaluate both the DeepMIL and linear-probe models, with cases randomly split across folds. For evaluation, we used survival analysis to compare the 0/1 case groups. Overall, across all 150 cases, univariable Cox regression showed that 1-year OS was more strongly associated with the DeepMIL status (46/104, HR=0.55, p=0.03) than the linear-probe status (55/95, HR=0.81, p=0.44). Results were consistent on the subset of 70 cases with known PD-L1 status, whereby OS was most strongly associated with the DeepMIL status (19/51, HR=0.40, p=0.04) compared to the linear-probe status (31/39, HR=0.46, p=0.09) and PD-L1 status (30/40, HR=0.65, p=0.32). In multivariable Cox regression adjusting for age group and smoking status, OS remained more strongly associated with the DeepMIL status (HR=0.45, p=0.08) than PD-L1 status (HR=0.72, p=0.47). In conclusion, the DeepMIL status predicted from H&E images showed a stronger association with outcomes compared to PD-L1 status, a standard biomarker for immunotherapy in NSCLC. These exploratory results demonstrate the potential of deep learning using pathology foundation models to improve immunotherapy outcomes prediction, even with small datasets. Such approaches may even enable the discovery of novel biomarkers from H&E images to advance precision medicine. Citation Format: Jessica Loo, Yang Wang, Pok Fai Wong, Ellery Wulczyn, Jeremy Lai, Peter Cimermancic, David F. Steiner, Shamira S. Weaver. Predicting immunotherapy outcomes from H&E images in lung cancer [abstract]. In: Proceedings of the American Association for Cancer Research Annual Meeting 2024; Part 1 (Regular s); 2024 Apr 5-10; San Diego, CA. Philadelphia (PA): AACR; Cancer Res 2024;84(6_Suppl) nr 7380.

Deep learning-based image analysis predicts PD-L1 status from H&E-stained histopathology images in breast cancer

PD-L1 Prediction of Lung Cancer by a Radiomics Method Using Rough Tumor Mask in CT Images

Prediction of PD-L1 tumor positive score in lung squamous cell carcinoma with H&E staining whole slide images and deep learning

Application of Automated Segmentation and Classification of PD-L1 Immunohistochemistry Images in Cancer Treatment Prediction.

Weakly Supervised Deep Learning Predicts Immunotherapy Response in Solid Tumors Based on PD-L1 Expression

Predicting PD-L1 status of esophageal cancer from H&E images based on FusedNet model.

Using deep learning to predict anti-PD-1 response in melanoma and lung cancer patients from histopathology images

Deep learning-based image analysis predicts PD-L1 status from 18F-FDG PET/CT images in non-small-cell lung cancer

Deep learning-enabled breast cancer hormonal receptor status determination from base-level H&E stains

Deep learning analysis of histopathological images predicts immunotherapy prognosis and reveals tumour microenvironment features in non-small cell lung cancer

Abstract 7380: Predicting immunotherapy outcomes from H&E images in lung cancer

A systematic pan-cancer study on deep learning-based prediction of multi-omic biomarkers from routine pathology images

Teacher-student collaborated multiple instance learning for pan-cancer PDL1 expression prediction from histopathology slides

[Intelligent Prediction of HER2 Status Based on Breast Histopathology].

Non-invasive measurement of PD-L1 status and prediction of immunotherapy response using deep learning of PET/CT images

Assessing PD-L1 expression in non-small cell lung cancer and predicting responses to immune checkpoint inhibitors using deep learning on computed tomography images

PD-L1 Classification of Weakly-Labeled Whole Slide Images of Breast Cancer

Comparing deep learning and pathologist quantification of cell-level PD-L1 expression in non-small cell lung cancer whole-slide images

Predicting Ki67, ER, PR, and HER2 Statuses from H&E-stained Breast Cancer Images

HER2 and FISH Status Prediction in Breast Biopsy H&E-Stained Images Using Deep Learning

Deep Semi Supervised Generative Learning for Automated PD-L1 Tumor Cell Scoring on NSCLC Tissue Needle Biopsies