Abstract:Objective: Vision transformers (ViTs) have shown promising performance in various classification tasks previously dominated by convolutional neural networks (CNNs). However, the performance of ViTs in referable diabetic retinopathy (DR) detection is relatively underexplored. In this study, using retinal photographs, we evaluated the comparative performances of ViTs and CNNs on detection of referable DR. Design: Retrospective study. Participants: A total of 48 269 retinal images from the open-source Kaggle DR detection dataset, the Messidor-1 dataset and the Singapore Epidemiology of Eye Diseases (SEED) study were included. Methods: Using 41 614 retinal photographs from the Kaggle dataset, we developed 5 CNN (Visual Geometry Group 19, ResNet50, InceptionV3, DenseNet201, and EfficientNetV2S) and 4 ViTs models (VAN_small, CrossViT_small, ViT_small, and Hierarchical Vision transformer using Shifted Windows [SWIN]_tiny) for the detection of referable DR. We defined the presence of referable DR as eyes with moderate or worse DR. The comparative performance of all 9 models was evaluated in the Kaggle internal test dataset (with 1045 study eyes), and in 2 external test sets, the SEED study (5455 study eyes) and the Messidor-1 (1200 study eyes). Main outcome measures: Area under operating characteristics curve (AUC), specificity, and sensitivity. Results: Among all models, the SWIN transformer displayed the highest AUC of 95.7% on the internal test set, significantly outperforming the CNN models (all P < 0.001). The same observation was confirmed in the external test sets, with the SWIN transformer achieving AUC of 97.3% in SEED and 96.3% in Messidor-1. When specificity level was fixed at 80% for the internal test, the SWIN transformer achieved the highest sensitivity of 94.4%, significantly better than all the CNN models (sensitivity levels ranging between 76.3% and 83.8%; all P < 0.001). This trend was also consistently observed in both external test sets. Conclusions: Our findings demonstrate that ViTs provide superior performance over CNNs in detecting referable DR from retinal photographs. These results point to the potential of utilizing ViT models to improve and optimize retinal photo-based deep learning for referable DR detection. Financial disclosures: Proprietary or commercial disclosure may be found in the Footnotes and Disclosures at the end of this article.

ViT-DR: Vision Transformers in Diabetic Retinopathy Grading Using Fundus Images

Classification of Diabetic Retinopathy Severity in Fundus Images Using the Vision Transformer and Residual Attention

Detecting severity of Diabetic Retinopathy from fundus images: A transformer network-based review

Comparative Analysis of Vision Transformers and Conventional Convolutional Neural Networks in Detecting Referable Diabetic Retinopathy

Automated retinal disease classification using hybrid transformer model (SViT) using optical coherence tomography images

Automated classification of choroidal neovascularization, diabetic macular edema, and drusen from retinal OCT images using vision transformers: a comparative study

A Novel Transformer Model With Multiple Instance Learning for Diabetic Retinopathy Classification

Diabetic Retinopathy Detection with Enhanced Vision Transformers: the Twins-PCPVT Solution

Vision transformer with masked autoencoders for referable diabetic retinopathy classification based on large-size retina image

Diabetic Retinopathy Classification using Transfer learning

Convolutional Neural Networks for Automated Diagnosis of Diabetic Retinopathy in Fundus Images

IDRiD: Diabetic Retinopathy - Segmentation and Grading Challenge.

Improved ResNet_101 assisted attentional global transformer network for automated detection and classification of diabetic retinopathy disease

DRStageNet: Deep Learning for Diabetic Retinopathy Staging from Fundus Images

Conv-ViT: A Convolution and Vision Transformer-Based Hybrid Feature Extraction Method for Retinal Disease Detection

Diabetic Retinopathy (DR) Severity Level Classification Using Multimodel Convolutional Neural Networks

Conversion and Implementation of State-of-the-Art Deep Learning Algorithms for the Classification of Diabetic Retinopathy

Feature fusion and optimization integrated refined deep residual network for diabetic retinopathy severity classification using fundus image

Cross-Fundus Transformer for Multi-modal Diabetic Retinopathy Grading with Cataract

Convolutional Network With Twofold Feature Augmentation for Diabetic Retinopathy Recognition From Multi-Modal Images