Abstract:Background: Thyroid-associated ophthalmopathy (TAO) is the most prevalent autoimmune orbital condition, significantly impacting patients' appearance and quality of life. Early and accurate identification of active TAO along with timely treatment can enhance prognosis and reduce the occurrence of severe cases. Although the Clinical Activity Score (CAS) serves as an effective assessment system for TAO, it is susceptible to assessor experience bias. This study aimed to develop an ensemble deep learning system that combines anterior segment slit-lamp photographs of patients with facial images to simulate expert assessment of TAO. Method: The study included 156 patients with TAO who underwent detailed diagnosis and treatment at Shanxi Eye Hospital Affiliated to Shanxi Medical University from May 2020 to September 2023. Anterior segment slit-lamp photographs and facial images were used as different modalities and analyzed from multiple perspectives. Two ophthalmologists with more than 10 years of clinical experience independently determined the reference CAS for each image. An ensemble deep learning model based on the residual network was constructed under supervised learning to predict five key inflammatory signs (redness of the eyelids and conjunctiva, and swelling of the eyelids, conjunctiva, and caruncle or plica) associated with TAO, and to integrate these objective signs with two subjective symptoms (spontaneous retrobulbar pain and pain on attempted upward or downward gaze) in order to assess TAO activity. Results: The proposed model achieved 0.906 accuracy, 0.833 specificity, 0.906 precision, 0.906 recall, and 0.906 F1-score in active TAO diagnosis, demonstrating advanced performance in predicting CAS and TAO activity signs compared to conventional single-view unimodal approaches. The integration of multiple views and modalities, encompassing both anterior segment slit-lamp photographs and facial images, significantly improved the prediction accuracy of the model for TAO activity and CAS. Conclusion: The ensemble multi-view multimodal deep learning system developed in this study can more accurately assess the clinical activity of TAO than traditional methods that solely rely on facial images. This innovative approach is intended to enhance the efficiency of TAO activity assessment, providing a novel means for its comprehensive, early, and precise evaluation.

Advances in artificial intelligence in thyroid-associated ophthalmopathy

Research progress and application of artificial intelligence in thyroid associated ophthalmopathy

Artificial intelligence CT screening model for thyroid-associated ophthalmopathy and tests under clinical conditions

Diagnosing Systemic Disorders with AI Algorithms Based on Ocular Images

An Intelligent Diagnostic System for Thyroid-Associated Ophthalmopathy Based on Facial Images

An overview of artificial intelligence in diabetic retinopathy and other ocular diseases

Artificial intelligence in thyroid ultrasound

Artificial Intelligence in Ophthalmic Surgery: Current Applications and Expectations

Research Progress of Artificial Intelligence Image Analysis in Systemic Disease-Related Ophthalmopathy

Retinal Photograph-based Deep Learning System for Detection of Thyroid-Associated Ophthalmopathy

Artificial intelligence and deep learning in ophthalmology: Current status and future perspectives

Artificial intelligence in the anterior segment of eye diseases

Application of artificial intelligence in the assessment of thyroid eye disease (TED) - a scoping review

Thyroid-associated ophthalmopathy: cu-rrent concepts regarding etiopathogenesis and management

Advances in artificial intelligence applications for ocular surface diseases diagnosis

Applications of Artificial Intelligence in Ophthalmology: General Overview

Artificial intelligence in glaucoma: opportunities, challenges, and future directions

Application of Artificial Intelligence in Ophthalmology: An Updated Comprehensive Review

Artificial intelligence assisted diagnosis of early tc markers and its application

Use of extreme gradient boosting, light gradient boosting machine, and deep neural networks to evaluate the activity stage of extraocular muscles in thyroid-associated ophthalmopathy

An ensemble deep learning diagnostic system for determining Clinical Activity Scores in thyroid-associated ophthalmopathy: integrating multi-view multimodal images from anterior segment slit-lamp photographs and facial images