Abstract:Objective: AI has demonstrated great promise in learning sophisticated features and relations in data that would otherwise remain hidden to the human eye. Here, we developed a proprietary AI-based multimodal approach to integrate clinical, digitized hematoxylin-eosin (H&E), and radiology bone scan (rBS) data for outcome prediction in ADT-treated nmCRPC patients. Identifying prostate cancer patients who may not benefit from ADT could improve the medical management of this disease beyond current definitive therapy. Methods: Patients in the ADT+placebo arm from SPARTAN clinical trial on nmCRPC with available clinical, H&E, and rBS were used (n=154). These patients were randomly divided into 70% (n=107) discovery and 30% (n=47) hold-out test datasets. Using the discovery set, we developed and trained a multimodal approach that combines survival convolutional neural networks (SCNNs1) and Cox proportional-hazards model (CPH) to learn ADT outcomes for overall survival (OS) and time to PSA progression (TTP) from the integration of imaging data and 11 traditional clinical features (e.g., tumor stage, Gleason score, PSA). The ability of the trained framework in predicting outcomes and risk stratification was evaluated on the hold-out set. Bootstrap analysis with Wilcoxon signed rank test was used to determine the significance of the multimodal framework's performance improvement compared to clinical CPH. Results: The multimodal framework was predictive of ADT outcomes for OS and TTP in nmCRPC patients. In SPARTAN's hold-out set, the multimodal framework significantly improved the predictive power of clinical CPH by 14%—16% across both outcomes (Wilcoxon signed rank P<0.0001). In particular, the multimodal framework's concordance index (c-index) was 0.72 for OS and 0.73 for TTP, while clinical CPH's c-index was 0.62 for OS, and 0.64 for TTP. Further, multimodal framework significantly stratified high- from low-risk nmCRPC patients for OS and TTP (log-rank P= 0.0049-0.0072), while clinical CPH failed to stratify risk for OS (log-rank P= 0.2891). Conclusion: AI-based framework that learns from the integration of different data types improves outcome prediction in ADT-treated nmCRPC. The multimodal approach demonstrates promise in treatment decision support for the early use of androgen receptor-directed therapy and patient selection for clinical trials with novel treatment combinations. Reference: 1. Mobadersany, Pooya, et al. "Predicting cancer outcomes from histology and genomics using convolutional networks." Proceedings of the National Academy of Sciences 115.13 (2018): E2970-E2979. Conflict of interest statement P.M., J.L., D.G., C.A., S.M., S.B., M.K.Y., K.T., N.H., J.Z., J.G., N.K., and S.S.F.Y., are employees of Janssen Pharmaceutical, LLC. Citation Format: Pooya Mobadersany, Justin Lucas, Darshana Govind, Clemente Aguilar-Bonavides, Sharon McCarthy, Sabine Brookman-May, Margaret K. Yu, Ken Tian, Natalie Hutnick, Jose Zamalloa, Joel Greshock, Najat Khan, Stephen S.F. Yip. Artificial intelligence (AI)-based multimodal framework predicts androgen-deprivation therapy (ADT) outcomes in non-metastatic castration resistant prostate cancer (nmCRPC) from SPARTAN [abstract]. In: Proceedings of the American Association for Cancer Research Annual Meeting 2022; 2022 Apr 8-13. Philadelphia (PA): AACR; Cancer Res 2022;82(12_Suppl) nr 5053.

MOSAIC: An Artificial Intelligence-Based Framework for Multimodal Analysis, Classification, and Personalized Prognostic Assessment in Rare Cancers

MOSAIC: Multi-Omic Spatial Atlas in Cancer, effect on precision oncology.

Classification and Personalized Prognostic Assessment on the Basis of Clinical and Genomic Features in Myelodysplastic Syndromes

Multimodal Survival Ensemble Network: Integrating Genomic and Histopathological Insights for Enhanced Cancer Prognosis.

Precision Oncology and AI: the ACMA System for Personalized Treatment Matching Using Real-World Data.

Multi-modal AI for comprehensive breast cancer prognostication

Abstract PO4-01-10: Multi-modal artificial intelligence models from baseline histopathology predict prognosis in HR+ HER2- early breast cancer

Non-Small Cell Lung Cancer Survival Estimation Through multi-omic two-layer SVM: a Multi-Omics and Multi-Sources Integrative Model

DeepMOCCA: A pan-cancer prognostic model identifies personalized prognostic markers through graph attention and multi-omics data integration

Personalized Prediction Model to Risk Stratify Patients With Myelodysplastic Syndromes

Abstract 7286: Optimizing prediction of metastatic castration-resistant prostate cancer: Multimodal fusion of clinicopathological and genomic features with machine learning

Abstract 5053: Artificial intelligence (AI)-based multimodal framework predicts androgen-deprivation therapy (ADT) outcomes in non-metastatic castration resistant prostate cancer (nmCRPC) from SPARTAN

A time-dependent explainable radiomic analysis from the multi-omic cohort of CPTAC-Pancreatic Ductal Adenocarcinoma

Using artificial intelligence-powered evidence-based molecular decision-making for improved outcomes in ovarian cancer.

Machine Learning Using Real-World and Translational Data to Improve Treatment Selection for NSCLC Patients Treated with Immunotherapy

Multi-task deep latent spaces for cancer survival and drug sensitivity prediction

DeepMMSA: A Novel Multimodal Deep Learning Method for Non-small Cell Lung Cancer Survival Analysis

Multimodal artificial intelligence-based pathogenomics improves survival prediction in oral squamous cell carcinoma

Lung cancer multi-omics digital human avatars for integrating precision medicine into clinical practice: the LANTERN study

Developing an ensemble machine learning study: Insights from a multi-center proof-of-concept study

A framework for validating AI in precision medicine: considerations from the European ITFoC consortium