Using artificial intelligence to improve human performance: efficient retinal disease detection training with synthetic images

Hitoshi Tabuchi,Justin Engelmann,Fumiatsu Maeda,Ryo Nishikawa,Toshihiko Nagasawa,Tomofusa Yamauchi,Mao Tanabe,Masahiro Akada,Keita Kihara,Yasuyuki Nakae,Yoshiaki Kiuchi,Miguel O Bernabeu

DOI: https://doi.org/10.1136/bjo-2023-324923

2024-03-14

British Journal of Ophthalmology

Abstract:Background Artificial intelligence (AI) in medical imaging diagnostics has huge potential, but human judgement is still indispensable. We propose an AI-aided teaching method that leverages generative AI to train students on many images while preserving patient privacy. Methods A web-based course was designed using 600 synthetic ultra-widefield (UWF) retinal images to teach students to detect disease in these images. The images were generated by stable diffusion, a large generative foundation model, which we fine-tuned with 6285 real UWF images from six categories: five retinal diseases (age-related macular degeneration, glaucoma, diabetic retinopathy, retinal detachment and retinal vein occlusion) and normal. 161 trainee orthoptists took the course. They were evaluated with two tests: one consisting of UWF images and another of standard field (SF) images, which the students had not encountered in the course. Both tests contained 120 real patient images, 20 per category. The students took both tests once before and after training, with a cool-off period in between. Results On average, students completed the course in 53 min, significantly improving their diagnostic accuracy. For UWF images, student accuracy increased from 43.6% to 74.1% (p<0.0001 by paired t-test), nearly matching the previously published state-of-the-art AI model’s accuracy of 73.3%. For SF images, student accuracy rose from 42.7% to 68.7% (p<0.0001), surpassing the state-of-the-art AI model’s 40%. Conclusion Synthetic images can be used effectively in medical education. We also found that humans are more robust to novel situations than AI models, thus showcasing human judgement’s essential role in medical diagnosis.

ophthalmology

What problem does this paper attempt to address?

The main problem that this paper attempts to solve is to improve human performance in medical image diagnosis, especially in retinal disease detection, by using synthetic images generated by artificial intelligence. Specifically, the researchers developed an artificial - intelligence - based teaching method. They used generative adversarial network (GAN) technology to generate a large number of synthetic ultra - wide - field (UWF) retinal images to train students to recognize diseases in these images while protecting patients' privacy. The goal of the study was to verify whether this method could effectively improve students' diagnostic accuracy and explore whether this teaching method could enable students to perform better than the existing state - of - the - art artificial intelligence models when facing unseen imaging modes.

The study found that after about 1 hour of training, the students' diagnostic accuracy improved significantly. For UWF images, the accuracy increased from 43.6% to 74.1%, almost reaching the level of the state - of - the - art AI model (73.3%). For standard - field - of - view (SF) images, the students' accuracy also increased from 42.7% to 68.7%, exceeding the 40% of the AI model. This indicates that humans have stronger adaptability when facing new imaging modes, emphasizing the importance of retaining human judgment in medical diagnosis. In addition, the study also demonstrated the potential of using AI - generated synthetic images for medical education. This method is not only cost - effective but also can avoid the ethical and data privacy issues that may be encountered when using real patient images.