Abstract:Myopia screening using cutting-edge ultra-widefield (UWF) fundus imaging is potentially significant for ophthalmic outcomes. Current multidisciplinary research between ophthalmology and deep learning (DL) concentrates primarily on disease classification and diagnosis using single-eye images, largely ignoring joint modeling and prediction for Oculus Uterque (OU, both eyes). Inspired by the complex relationships between OU and the high correlation between the (continuous) outcome labels (Spherical Equivalent and Axial Length), we propose a framework of copula-enhanced adapter convolutional neural network (CNN) learning with OU UWF fundus images (OUCopula) for joint prediction of multiple clinical scores. We design a novel bi-channel multi-label CNN that can (1) take bi-channel image inputs subject to both high correlation and heterogeneity (by sharing the same backbone network and employing adapters to parameterize the channel-wise discrepancy), and (2) incorporate correlation information between continuous output labels (using a copula). Solid experiments show that OUCopula achieves satisfactory performance in myopia score prediction compared to backbone models. Moreover, OUCopula can far exceed the performance of models constructed for single-eye inputs. Importantly, our study also hints at the potential extension of the bi-channel model to a multi-channel paradigm and the generalizability of OUCopula across various backbone CNNs.

What problem does this paper attempt to address?

### Problems Addressed by the Paper The paper aims to address several key issues in myopia screening using Ultra-Widefield (UWF) fundus images: 1. **Bilateral Joint Modeling and Prediction**: - Most current studies focus primarily on disease classification and diagnosis using single-eye images, neglecting the complex relationship and high correlation between both eyes (Oculus Uterque, OU). The paper proposes a joint prediction framework based on bilateral UWF fundus images to address this issue. 2. **Heterogeneity and Symmetry**: - There exists heterogeneity between the two eyes (i.e., "inter-eye asymmetry"), for example, a patient’s two eyes may have different myopia statuses. The paper introduces adapter modules to handle this heterogeneity while preserving the strong correlation between the two eyes. 3. **Multi-Label Prediction**: - Myopia assessment typically relies on Spherical Equivalent (SE), while Axial Length (AL), an important continuous output variable, is often overlooked in terms of its predictive capability. The paper proposes a multi-label learning method to predict both SE and AL simultaneously, leveraging their correlation to improve prediction performance. 4. **Dataset Quality**: - The lack of high-quality annotated data in existing studies limits model performance. The paper establishes a high-quality annotated dataset containing 5228 bilateral UWF fundus images, all de-identified, labeled, and graded by experienced doctors. ### Main Contributions 1. **High-Quality Dataset**: - Established a high-quality annotated dataset containing 5228 bilateral UWF fundus images. 2. **Dual-Channel Multi-Label CNN Architecture**: - Proposed a new dual-channel multi-label CNN architecture that handles inter-eye asymmetry and high correlation by sharing a backbone network and introducing adapter modules. 3. **Joint Prediction Framework**: - Proposed a joint prediction framework based on a Gaussian copula model (OUCopula), capturing the correlation between multiple continuous labels through a copula likelihood loss function to enhance model prediction capability. ### Experimental Results - **Quantitative Evaluation**: - Comparative experiments with a simplified ResNet backbone model validated the superior performance of OUCopula in predicting SE and AL. - OUCopula outperformed pure ResNet and ResNet + Adapters models in almost all sub-results. - **Improvement Effect**: - Compared to standalone ResNet, OUCopula improved the total loss by an average of 7.18%. - Compared to ResNet + Adapters, OUCopula showed more stable performance improvements in most sub-results. ### Conclusion Overall, the experimental results demonstrate that OUCopula excels in enhancing the prediction accuracy of the backbone model, particularly in handling the heterogeneity and correlation between both eyes. This framework not only improves the prediction performance of SE and AL but also showcases robustness and balance in multi-label prediction tasks.

OUCopula: Bi-Channel Multi-Label Copula-Enhanced Adapter-Based CNN for Myopia Screening Based on OU-UWF Images

OU-CoViT: Copula-Enhanced Bi-Channel Multi-Task Vision Transformers with Dual Adaptation for OU-UWF Images

CeCNN: Copula-enhanced convolutional neural networks in joint prediction of refraction error and axial length based on ultra-widefield fundus images

Simultaneous Alignment and Surface Regression Using Hybrid 2D-3D Networks for 3D Coherent Layer Segmentation of Retinal OCT Images with Full and Annotations

A Deep Learning Analysis Framework for Ophthalmic Diseases and Physical Health from Binocular Fundus Image Pairs

Geometric Correspondence-Based Multimodal Learning for Ophthalmic Image Analysis

Parallel Multi-Path Network for Ocular Disease Detection Inspired by Visual Cognition Mechanism

Multi-label ocular disease classification with a dense correlation deep neural network

Unifying Structure Analysis and Surrogate-driven Function Regression for Glaucoma OCT Image Screening

Multi-Context Deep Network for Angle-Closure Glaucoma Screening in Anterior Segment OCT

An Artificial Intelligent Risk Classification Method of High Myopia Based on Fundus Images

Automatic Screening and Identifying Myopic Maculopathy on Optical Coherence Tomography Images Using Deep Learning

DeepUWF: An Automated Ultra-Wide-Field Fundus Screening System via Deep Learning

A multimodal deep learning system to distinguish late stages of AMD and to compare expert vs. AI ocular biomarkers

Dual-correlate optimized coarse-fine strategy for monocular laparoscopic videos feature matching via multilevel sequential coupling feature descriptor

Multi-Scale Clinical-Guided Binocular Fusion Framework for Predicting New-Onset Hypertension over a Four-Year Period

Development and validation of a deep learning system to screen vision-threatening conditions in high myopia using optical coherence tomography images

Development and Validation of an Automatic Ultrawide-Field Fundus Imaging Enhancement System for Facilitating Clinical Diagnosis: A Cross-sectional Multicenter Study

Leveraging Multimodal Fusion for Enhanced Diagnosis of Multiple Retinal Diseases in Ultra-wide OCTA

A Deep Learning based Joint Segmentation and Classification Framework for Glaucoma Assesment in Retinal Color Fundus Images

Localizing Optic Disc And Cup For Glaucoma Screening Via Deep Object Detection Networks