Abstract:Hematoxylin and eosin (H&E) stained histologic sections contain invaluable information that remains largely untapped because of its complexity. To this end, AI applications employing deep learning (DL) can facilitate the translation of image data to enable human interpretation and yield novel oncological insights that would have otherwise remained imperceptible. DL-based methodologies are multimodal, capable of integrating imaging with clinicogenomic data to furnish a more holistic perspective and affording more accurate predictions in oncology. Here, we developed an unsupervised DL workflow to analyze 1,799 H&E images of lung cancer (NSCLC n = 951; SCLC n = 50; others n = 798) incorporating comprehensive patient-level clinical data (electronic health records [ConcertAI]) integrated with genomics (WES and RNA-seq [Caris Labs]). There are three steps in our approach: (1) image preprocessing and filtering, yielding > 30 million image patches; (2) utilizing pretrained SimCLR models from 57 public oncology histopathology datasets with ResNet-18 as a backbone structure to extract 512-dimensional-feature vectors for each patch; (3) using three unsupervised clustering methods (kmeans, DBSCAN, Leiden clustering) to cluster patches and selected Leiden clustering. We identified 635 primary imaging clusters using an elbow method and generated an image feature matrix by calculating correlations between each patch and cluster centroids; these were aggregated and mapped back to source slides. In this proof-of-concept, distinct image feature patterns characterized SCLC and NSCLC samples. For SCLC, one of the salient features was the presence of hemorrhage, which may be associated with higher rates of fine-needle aspiration biopsy procedure for SCLC compared with NSCLC which was confirmed in the EHR data (p = 0.032). Derived morphological clusters were correlated with tumor-immune genomic features (Tumor Mutational Burden [TMB], Immunologic Constant of Rejection [ICR], and Miracle scores1) serving as predictors of response to immune-checkpoint inhibitor therapy. By applying linear models, we detected 11, 96 and 249 significantly associated imaging clusters, respectively, highly enriched with immune cells e.g., plasma cells, macrophages, lymphocytes, and supporting an infiltrated and inflamed tumor-immune microenvironment. In summary, a multimodal, unsupervised deep learning workflow combining H&E imaging with clinicogenomic data was developed to identify histologic feature clusters associated with well-established tumor-immune genomic signatures of NSCLC immune infiltration and molecular phenotypes. These studies demonstrate enormous potential to yield histopathological and translational insights in NSCLC and SCLC that can empower clinicians to make better therapeutic response predictions. Citation Format: Si Wu, Yujie Zhao, Hugo Luo, Kevin Kolahi, Thanh Bui, Xu Shi, Aditee Shrotre, Alexander Liede, Xi Zhao, Josue Samayoa, Weilong Zhao. Integrating real-world histopathological and clinicogenomic data from 1799 lung cancer patients by applying unsupervised deep learning [abstract]. In: Proceedings of the American Association for Cancer Research Annual Meeting 2024; Part 1 (Regular s); 2024 Apr 5-10; San Diego, CA. Philadelphia (PA): AACR; Cancer Res 2024;84(6_Suppl) nr 2310.

Integrative Analysis of Histological Textures and Lymphocyte Infiltration in Renal Cell Carcinoma using Deep Learning

Development of a Histopathology Informatics Pipeline for Classification and Prediction of Clinical Outcomes in Subtypes of Renal Cell Carcinoma

Abstract 5420: Deep learning-based mapping of tertiary lymphoid structure scores from H&E images of renal cell carcinoma trained by spatial transcriptomics data

Abstract 2310: Integrating real-world histopathological and clinicogenomic data from 1799 lung cancer patients by applying unsupervised deep learning

A comprehensive texture feature analysis framework of renal cell carcinoma: pathological, prognostic, and genomic evaluation based on CT images

Integrative Analysis Of Histopathological Images And Genomic Data Predicts Clear Cell Renal Cell Carcinoma Prognosis

A Deep Learning Approach for Tissue Spatial Quantification and Genomic Correlations of Histopathological Images

Renal Cancer Detection: Fusing Deep and Texture Features from Histopathology Images

Deep learning integrates histopathology and proteogenomics at a pan-cancer level

Deep learning-based breast cancer grading and survival analysis on whole-slide histopathology images

Deep learning-based assessment of tumor-associated stroma for diagnosing breast cancer in histopathology images

Using deep convolutional neural networks to identify and classify tumor-associated stroma in diagnostic breast biopsies

Development and evaluation of a deep neural network for histologic classification of renal cell carcinoma on biopsy and surgical resection slides

Computational Staining of Pathology Images to Study the Tumor Microenvironment in Lung Cancer

Deep learned tissue “fingerprints” classify breast cancers by ER/PR/Her2 status from H&E images

Deep learning-based tumor microenvironment analysis in colon adenocarcinoma histopathological whole-slide images

Classifying Subtypes and Predicting Survival of Renal Cell Carcinoma Using Histopathology Image-Based Deep Learning

Value renal CT volumetric texture analysis with machine learning radiomics in assessment of pathological grade of clear cell renal cell carcinoma

Computational pathology model to assess acute and chronic transformations of the tubulointerstitial compartment in renal allograft biopsies

Abstract 7399: Development of a reproducible AI-based spatial biomarker of the tumor immune infiltrate on H&E slides