Abstract:Hematoxylin and eosin (H&E) stained histologic sections contain invaluable information that remains largely untapped because of its complexity. To this end, AI applications employing deep learning (DL) can facilitate the translation of image data to enable human interpretation and yield novel oncological insights that would have otherwise remained imperceptible. DL-based methodologies are multimodal, capable of integrating imaging with clinicogenomic data to furnish a more holistic perspective and affording more accurate predictions in oncology. Here, we developed an unsupervised DL workflow to analyze 1,799 H&E images of lung cancer (NSCLC n = 951; SCLC n = 50; others n = 798) incorporating comprehensive patient-level clinical data (electronic health records [ConcertAI]) integrated with genomics (WES and RNA-seq [Caris Labs]). There are three steps in our approach: (1) image preprocessing and filtering, yielding > 30 million image patches; (2) utilizing pretrained SimCLR models from 57 public oncology histopathology datasets with ResNet-18 as a backbone structure to extract 512-dimensional-feature vectors for each patch; (3) using three unsupervised clustering methods (kmeans, DBSCAN, Leiden clustering) to cluster patches and selected Leiden clustering. We identified 635 primary imaging clusters using an elbow method and generated an image feature matrix by calculating correlations between each patch and cluster centroids; these were aggregated and mapped back to source slides. In this proof-of-concept, distinct image feature patterns characterized SCLC and NSCLC samples. For SCLC, one of the salient features was the presence of hemorrhage, which may be associated with higher rates of fine-needle aspiration biopsy procedure for SCLC compared with NSCLC which was confirmed in the EHR data (p = 0.032). Derived morphological clusters were correlated with tumor-immune genomic features (Tumor Mutational Burden [TMB], Immunologic Constant of Rejection [ICR], and Miracle scores1) serving as predictors of response to immune-checkpoint inhibitor therapy. By applying linear models, we detected 11, 96 and 249 significantly associated imaging clusters, respectively, highly enriched with immune cells e.g., plasma cells, macrophages, lymphocytes, and supporting an infiltrated and inflamed tumor-immune microenvironment. In summary, a multimodal, unsupervised deep learning workflow combining H&E imaging with clinicogenomic data was developed to identify histologic feature clusters associated with well-established tumor-immune genomic signatures of NSCLC immune infiltration and molecular phenotypes. These studies demonstrate enormous potential to yield histopathological and translational insights in NSCLC and SCLC that can empower clinicians to make better therapeutic response predictions. Citation Format: Si Wu, Yujie Zhao, Hugo Luo, Kevin Kolahi, Thanh Bui, Xu Shi, Aditee Shrotre, Alexander Liede, Xi Zhao, Josue Samayoa, Weilong Zhao. Integrating real-world histopathological and clinicogenomic data from 1799 lung cancer patients by applying unsupervised deep learning [abstract]. In: Proceedings of the American Association for Cancer Research Annual Meeting 2024; Part 1 (Regular s); 2024 Apr 5-10; San Diego, CA. Philadelphia (PA): AACR; Cancer Res 2024;84(6_Suppl) nr 2310.

Feasibility of a deep learning algorithm to distinguish large cell neuroendocrine from small cell lung carcinoma in cytology specimens

Artificial Classification of Cervical Squamous Lesions in ThinPrep Cytologic Tests Using a Deep Convolutional Neural Network.

Deep Learning Facilitates Distinguishing Histologic Subtypes of Pulmonary Neuroendocrine Tumors on Digital Whole-Slide Images

Comparison of Fine-Tuned Deep Convolutional Neural Networks for the Automated Classification of Lung Cancer Cytology Images with Integration of Additional Classifiers

A Multicenter Proof-of-concept Study on Deep Learning-Based Intraoperative Discrimination of Primary Central Nervous System Lymphoma

Deep Learning-Based Intraoperative Differentiation of Primary CNS Lymphoma and Glioma: a Discovery, Multicenter Validation, and Proof-of Concept Study

Discriminant model for cytologic distinction of large cell neuroendocrine carcinoma from small cell carcinoma of the lung

Machine learning enabled classification of lung cancer cell lines co-cultured with fibroblasts with lightweight convolutional neural network for initial diagnosis

Improving differential diagnosis of pulmonary large cell neuroendocrine carcinoma and small cell lung cancer via a transcriptomic, biological pathway-based machine learning model

Deep learning classification of lung cancer histology using CT images

Deep Convolutional Neural Network-Based Classification of Cancer Cells on Cytological Pleural Effusion Images.

Artificial Intelligence Recognition Model Using Liquid-based Cytology Images to Discriminate Malignancy and Histological Types of Non-small-cell Lung Cancer

Abstract 2310: Integrating real-world histopathological and clinicogenomic data from 1799 lung cancer patients by applying unsupervised deep learning

MA03.07 Development and Validation of Deep Learning Model for Recognition of Histologic Subtype of Lung Adenocarcinoma from CT Images

Improving differential diagnosis of pulmonary large cell neuroendocrine carcinoma (LCNEC) and small cell lung cancer via a transcriptomic, biological pathway-based ridge regression model.

Deep learning-based identification of esophageal cancer subtypes through analysis of high-resolution histopathology images

Direct identification of ALK and ROS1 fusions in non-small cell lung cancer from hematoxylin and eosin-stained slides using deep learning algorithms

Attention-Based Deep Neural Networks for Detection of Cancerous and Precancerous Esophagus Tissue on Histopathological Slides

Prediction of Epidermal Growth Factor Receptor Mutation Subtypes in Non-Small Cell Lung Cancer From Hematoxylin and Eosin-Stained Slides Using Deep Learning

Enhanced NSCLC Subtyping and Staging Through Attention-Augmented Multi-Task Deep Learning: A Novel Diagnostic Tool

Machine Learning and Deep Learning CT-Based Models for Predicting the Primary Central Nervous System Lymphoma and Glioma Types: A Multicenter Retrospective Study