Abstract:Checkpoint blockade immunotherapy is a cornerstone of lung cancer treatment, but there is a need to improve the identification of patients who will respond favorably. Here, we explored a deep learning approach to predict immunotherapy outcomes from hematoxylin and eosin (H&E) images in non-small cell lung cancer (NSCLC). We included 150 unique cases with metastatic NSCLC (113 adenocarcinoma, 29 squamous cell, 8 other) treated with anti-PD-1/PD-L1 immunotherapy (56 nivolumab, 49 atezolizumab, 44 pembrolizumab, 1 durvalumab) as mono or combination (14 with chemotherapy, 1 with ipilimumab) therapy in a single institution. Each case consisted of a representative H&E whole slide image (53 biopsies, 50 needle core biopsies, 47 resections) obtained prior to immunotherapy, and the outcome reported as the 1-year overall survival (OS). PD-L1 status (tumor proportion score ≥ 1%) was known for 70 cases. We preprocessed the H&E images using two deep learning models previously developed using The Cancer Genome Atlas dataset. First, we used a classification model to identify tumor regions and randomly sampled a fixed number of tumor patches for each case. Then, we used a self-supervised pathology foundation model to obtain a compressed visual representation of each patch, known as an embedding. Next, using our dataset, we trained a deep multiple instance learning (DeepMIL) model with a gated attention mechanism to predict the binary 1-year OS status (0=deceased, 1=alive) for each case. As a baseline, we also trained a linear-probe (logistic regression) model using the averaged embeddings. Given the small dataset size, 5-fold cross-validation was used to train and evaluate both the DeepMIL and linear-probe models, with cases randomly split across folds. For evaluation, we used survival analysis to compare the 0/1 case groups. Overall, across all 150 cases, univariable Cox regression showed that 1-year OS was more strongly associated with the DeepMIL status (46/104, HR=0.55, p=0.03) than the linear-probe status (55/95, HR=0.81, p=0.44). Results were consistent on the subset of 70 cases with known PD-L1 status, whereby OS was most strongly associated with the DeepMIL status (19/51, HR=0.40, p=0.04) compared to the linear-probe status (31/39, HR=0.46, p=0.09) and PD-L1 status (30/40, HR=0.65, p=0.32). In multivariable Cox regression adjusting for age group and smoking status, OS remained more strongly associated with the DeepMIL status (HR=0.45, p=0.08) than PD-L1 status (HR=0.72, p=0.47). In conclusion, the DeepMIL status predicted from H&E images showed a stronger association with outcomes compared to PD-L1 status, a standard biomarker for immunotherapy in NSCLC. These exploratory results demonstrate the potential of deep learning using pathology foundation models to improve immunotherapy outcomes prediction, even with small datasets. Such approaches may even enable the discovery of novel biomarkers from H&E images to advance precision medicine. Citation Format: Jessica Loo, Yang Wang, Pok Fai Wong, Ellery Wulczyn, Jeremy Lai, Peter Cimermancic, David F. Steiner, Shamira S. Weaver. Predicting immunotherapy outcomes from H&E images in lung cancer [abstract]. In: Proceedings of the American Association for Cancer Research Annual Meeting 2024; Part 1 (Regular s); 2024 Apr 5-10; San Diego, CA. Philadelphia (PA): AACR; Cancer Res 2024;84(6_Suppl) nr 7380.

Predicting PD-L1 status of esophageal cancer from H&E images based on FusedNet model.

Deep learning-based image analysis predicts PD-L1 status from H&E-stained histopathology images in breast cancer

PD-L1 Prediction of Lung Cancer by a Radiomics Method Using Rough Tumor Mask in CT Images

Deep Learning Identifies HAT1 As a Morphological Regulator in Esophageal Squamous Carcinoma Cells Through Controlling Cell Senescence

Prediction of HER2 Status Based on Deep Learning in H&E-Stained Histopathology Images of Bladder Cancer

A 18F-FDG PET/CT-based Deep Learning-Radiomics-clinical Model for Prediction of Cervical Lymph Node Metastasis in Esophageal Squamous Cell Carcinoma

Self-supervised Contrastive Learning Using CT Images for PD-1/PD-L1 Expression Prediction in Hepatocellular Carcinoma

Deep learning-based image analysis predicts PD-L1 status from 18F-FDG PET/CT images in non-small-cell lung cancer

Assessing PD-L1 expression in non-small cell lung cancer and predicting responses to immune checkpoint inhibitors using deep learning on computed tomography images

A deep learning model using hyperspectral image for EUS‐FNA cytology diagnosis in pancreatic ductal adenocarcinoma

From pixels to patient care: deep learning-enabled pathomics signature offers precise outcome predictions for immunotherapy in esophageal squamous cell cancer

Using deep learning to predict anti-PD-1 response in melanoma and lung cancer patients from histopathology images

Deep learning analysis of histopathological images predicts immunotherapy prognosis and reveals tumour microenvironment features in non-small cell lung cancer

Automated PD-L1 status prediction in lung cancer with multi-modal PET/CT fusion

Non-Invasive Measurement Using Deep Learning Algorithm Based on Multi-Source Features Fusion to Predict PD-L1 Expression and Survival in NSCLC

Weakly Supervised Deep Learning Predicts Immunotherapy Response in Solid Tumors Based on PD-L1 Expression

A Deep Learning Model and Human-Machine Fusion for Prediction of EBV-associated Gastric Cancer from Histopathology

Abstract 7380: Predicting immunotherapy outcomes from H&E images in lung cancer

Non-invasive assessment of programmed cell death ligand-1 expression using 18F-FDG PET-CT imaging in esophageal squamous cell carcinoma

Teacher-student collaborated multiple instance learning for pan-cancer PDL1 expression prediction from histopathology slides

HER2 and FISH Status Prediction in Breast Biopsy H&E-Stained Images Using Deep Learning