Real-World evaluation of an AI triaging system for chest X-rays: A prospective clinical study
Srinath Sridharan,Alicia Seah Xin Hui,Narayan Venkataraman,Prasanna Sivanath Tirukonda,Ram Pratab Jeyaratnam,Sindhu John,Saraswathy Suresh Babu,Perry Liew,Joe Francis,Tsai Koh Tzan,Wong Kang Min,Goh Min Liong,Charlene Liew Jin Yee
DOI: https://doi.org/10.1016/j.ejrad.2024.111783
IF: 4.531
2024-10-13
European Journal of Radiology
Abstract:Chest X-rays (CXRs) are crucial for diagnosing and managing lung conditions. While CXR is a common and cost-effective diagnostic tool, interpreting the high volume of CXRs is challenging due to workforce limitations. Artificial intelligence (AI) offers promise in enhancing efficiency and accuracy. However, real-world applicability and generalizability across diverse patient cohorts remain areas of concerns. In our study, the LUNIT INSIGHT CXR Triage software was evaluated in a diverse patient cohort. Forty-three radiologists, blinded to AI results, assessed CXRs categorized into normal, non-urgent, and urgent using a 3-tier classification system. Performance metrics and turnaround times were analyzed.The AI system demonstrated sensitivity of 89% for normal CXRs, specificity of 93%, PPV of 83%, and NPV of 95%, with an F1 score of 0.86 and an AUC of 0.91. For non-urgent CXRs, sensitivity and specificity were 93% and 91%, with PPV and NPV at 94% and 89%, respectively, and an F1 score of 0.94 and an AUC of 0.92. In the urgent category, sensitivity was 82%, specificity 99%, PPV 90%, and NPV 98%. Subgroup analysis revealed consistently high accuracy across various age groups (Young, Adult, Senior), genders, and ethnicities (Chinese, Malay, Indian, Others), with sensitivity, specificity, and AUC consistently above 84%. The AI system also significantly reduced turnaround times across all subgroups, indicating its robust performance and generalizability in diverse healthcare settings.
radiology, nuclear medicine & medical imaging