Optic Disc Classification by Deep Learning versus Expert Neuro‐Ophthalmologists

Valérie Biousse,Nancy J. Newman,Raymond P. Najjar,Caroline Vasseneix,Xinxing Xu,Daniel S. Ting,Léonard B. Milea,Jeong‐Min Hwang,Dong Hyun Kim,Hee Kyung Yang,Steffen Hamann,John J. Chen,Yong Liu,Tien Yin Wong,Dan Milea,for the BONSAI Study Group,Barnabé Rondé‐Courbis,Philippe Gohier,Neil Miller,Tanyatuth Padungkiatsagul,Anuchit Poonyathalang,Yanin Suwan,Kavin Vanikieti,Leonard B Milea,Giulia Amore,Piero Barboni,Michele Carbonelli,Valerio Carelli,Chiara La Morgia,Martina Romagnoli,Marie‐Bénédicte Rougier,Selvakumar Ambika,Swetha Komma,Pedro Fonseca,Miguel Raimundo,Isabelle Karlesand,Wolf Alexander Lagrèze,Nicolae Sanda,Gabriele Thumann,Florent Aptel,Christophe Chiquet,Kaiqun Liu,Hui Yang,Carmen KM Chan,Noel CY Chan,Carol Y Cheung,Thi Ha Chau Tran,James Acheson,Maged S Habib,Neringa Jurkute,Patrick Yu‐Wai‐Man,Richard Kho,Jost B Jonas,Nouran Sabbagh,Catherine Vignal‐Clermont,Rabih Hage,Raoul K Khanna,Tin Aung,Ching‐Yu Cheng,Ecosse Lamoureux,Jing Liang Loo,Raymond P Najjar,Shweta Singhal,Daniel Ting,Sharon Tow,Zhubo Jiang,Clare L Fraser,Luis J. Mejico,Masoud Aghsaei Fard,
DOI: https://doi.org/10.1002/ana.25839
IF: 11.2
2020-08-07
Annals of Neurology
Abstract:<section class="article-section__content"><h3 class="article-section__sub-title section1"> Objective</h3><p>To compare the diagnostic performance of an artificial intelligence deep learning system with that of expert neuro‐ophthalmologists in classifying optic disc appearance.</p></section><section class="article-section__content"><h3 class="article-section__sub-title section1"> Methods</h3><p>The deep learning system was previously trained and validated on 14,341 ocular fundus photographs from 19 international centers. The performance of the system was evaluated on 800 new fundus photographs (400 normal optic discs, 201 papilledema [disc edema from elevated intracranial pressure], 199 other optic disc abnormalities) and compared with that of two expert neuro‐ophthalmologists who independently reviewed the same randomly‐presented images without clinical information. Area‐under‐the‐receiver‐operating‐characteristic‐curve, accuracy, sensitivity and specificity were calculated.</p></section><section class="article-section__content"><h3 class="article-section__sub-title section1"> Results</h3><p>The system correctly classified 678/800 (84.7%) photographs, compared with 675/800 (84.4%) for Expert 1 and 641/800 (80.1%) for Expert 2. The system yielded area‐under‐the‐receiver‐operating‐characteristic‐curves of 0.97 (CI 95%, 0.96 ‐ 0.98), 0.96 (CI 95%, 0.94 ‐ 0.97) and 0.89 (CI 95%, 0.87 ‐ 0.92) for the detection of normal discs, papilledema and other disc abnormalities, respectively. The accuracy, sensitivity and specificity of the system's classification of optic discs were similar or better than the two experts. Inter‐grader agreement at the eye level was 0.71 (CI 95%, 0.67‐0.76) between Expert 1 and Expert 2, 0.72 (CI 95%, 0.68‐0.76) between the system and Expert 1, and 0.65 (CI 95%, 0.61‐0.70) between the system and Expert 2.</p></section><section class="article-section__content"><h3 class="article-section__sub-title section1"> Interpretation</h3><p>The performance of this deep learning system at classifying optic disc abnormalities was at least as good as two expert neuro‐ophthalmologists. Future prospective studies are needed to validate this system as a diagnostic aid in relevant clinical settings.</p><p>This article is protected by copyright. All rights reserved.</p></section>
neurosciences,clinical neurology
What problem does this paper attempt to address?
The aim of this paper is to address the problem of comparing the diagnostic performance of an artificial intelligence deep learning system (BONSAI-DLS) with that of expert neuro-ophthalmologists in classifying optic disc appearances. Specifically, the study evaluates whether the deep learning system can match or even surpass the diagnostic level of human experts in distinguishing normal optic discs, papilledema (optic disc swelling due to increased intracranial pressure), and other optic disc abnormalities, without any clinical information. The results show that the deep learning system not only matches or exceeds human experts in classification accuracy but also processes data much faster than human experts. Additionally, the study explores the potential application value of AI-assisted diagnosis of optic disc abnormalities in settings such as emergency care and neurology outpatient clinics. Future research will further validate the effectiveness of this system as a diagnostic aid in relevant clinical environments.