Artificial Intelligence-Based Disease Activity Monitoring to Personalized Neovascular Age-Related Macular Degeneration Treatment: A Feasibility Study
Zufar Mulyukov,Pearse A Keane,Jayashree Sahni,Sandra Liakopoulos,Katja Hatz,Daniel Shu Wei Ting,Roberto Gallego-Pinazo,Tariq Aslam,Chui Ming Gemmy Cheung,Gabriella De Salvo,Oudy Semoun,Gábor Márk Somfai,Andreas Stahl,Brandon J Lujan,Daniel Lorand
DOI: https://doi.org/10.1016/j.xops.2024.100565
2024-06-17
Abstract:Purpose: To evaluate the performance of a disease activity (DA) model developed to detect DA in participants with neovascular age-related macular degeneration (nAMD). Design: Post hoc analysis. Participants: Patient dataset from the phase III HAWK and HARRIER (H&H) studies. Methods: An artificial intelligence (AI)-based DA model was developed to generate a DA score based on measurements of OCT images and other parameters collected from H&H study participants. Disease activity assessments were classified into 3 categories based on the extent of agreement between the DA model's scores and the H&H investigators' decisions: agreement ("easy"), disagreement ("noisy"), and close to the decision boundary ("difficult"). Then, a panel of 10 international retina specialists ("panelists") reviewed a sample of DA assessments of these 3 categories that contributed to the training of the final DA model. A panelists' majority vote on the reviewed cases was used to evaluate the accuracy, sensitivity, and specificity of the DA model. Main outcome measures: The DA model's performance in detecting DA compared with the DA assessments made by the investigators and panelists' majority vote. Results: A total of 4472 OCT DA assessments were used to develop the model; of these, panelists reviewed 425, categorized as "easy" (17.2%), "noisy" (20.5%), and "difficult" (62.4%). False-positive and false negative rates of the DA model's assessments decreased after changing the assessment in some cases reviewed by the panelists and retraining the DA model. Overall, the DA model achieved 80% accuracy. For "easy" cases, the DA model reached 96% accuracy and performed as well as the investigators (96% accuracy) and panelists (90% accuracy). For "noisy" cases, the DA model performed similarly to panelists and outperformed the investigators (84%, 86%, and 16% accuracies, respectively). The DA model also outperformed the investigators for "difficult" cases (74% and 53% accuracies, respectively) but underperformed the panelists (86% accuracy) owing to lower specificity. Subretinal and intraretinal fluids were the main clinical parameters driving the DA assessments made by the panelists. Conclusions: These results demonstrate the potential of using an AI-based DA model to optimize treatment decisions in the clinical setting and in detecting and monitoring DA in patients with nAMD. Financial disclosures: Proprietary or commercial disclosure may be found in the Footnotes and Disclosures at the end of this article.