Abstract:In the field of medical imaging, 3D deep learning models play a crucial role in building powerful predictive models of disease progression. However, the size of these models presents significant challenges, both in terms of computational resources and data requirements. Moreover, achieving high-quality pretraining of 3D models proves to be even more challenging. To address these issues, hybrid 2.5D approaches provide an effective solution for utilizing 3D volumetric data efficiently using 2D models. Combining 2D and 3D techniques offers a promising avenue for optimizing performance while minimizing memory requirements. In this paper, we explore 2.5D architectures based on a combination of convolutional neural networks (CNNs), long short-term memory (LSTM), and Transformers. In addition, leveraging the benefits of recent non-contrastive pretraining approaches in 2D, we enhanced the performance and data efficiency of 2.5D techniques even further. We demonstrate the effectiveness of architectures and associated pretraining on a task of predicting progression to wet age-related macular degeneration (AMD) within a six-month period on two large longitudinal OCT datasets.

What problem does this paper attempt to address?

The paper attempts to address the problem of efficiently predicting the progression of age-related macular degeneration (AMD) to wet AMD in retinal optical coherence tomography (OCT). Specifically, the authors aim to develop an effective 3D prognostic model that can predict whether patients with intermediate AMD (iAMD) will progress to late-stage wet AMD within 6 months. ### Background and Challenges 1. **Limitations of 3D Models**: - 3D deep learning models play a significant role in medical imaging, but they require high computational resources and a large amount of training data. - Pre-training 3D models is very challenging, especially in the medical field where labeled data is scarce. 2. **Advantages of 2.5D Methods**: - 2.5D methods combine 2D and 3D techniques, allowing the use of 3D volumetric data while reducing memory requirements. - 2D models can be more effectively pre-trained on large-scale natural image datasets or using self-supervised learning (SSL) on domain-specific datasets. ### Research Objectives - Develop an efficient 2.5D architecture that combines Convolutional Neural Networks (CNN), Long Short-Term Memory networks (LSTM), and Transformers to predict AMD progression from OCT volumetric data. - Improve model performance and data efficiency under limited data conditions through pre-training and fine-tuning. ### Main Contributions - **2.5D Architecture**: Explored two 2.5D architectures, namely CNN + BiLSTM and CNN + Transformer, to generate B-scan embeddings and perform volumetric-level predictions. - **Pre-training Strategy**: Utilized Time-sensitive Non-Contrastive Learning (TINC) to pre-train 2D CNNs, enhancing model performance and data efficiency. - **Experimental Validation**: Conducted experiments on two large longitudinal OCT datasets, validating the effectiveness and advantages of the 2.5D approach in predicting AMD progression. ### Conclusion - The 2.5D approach is not only more efficient in terms of memory and label usage but also outperforms larger 3D models when appropriately pre-trained. - The BiLSTM method remains advantageous under limited and imbalanced 3D data conditions, especially in terms of the PRAUC metric. - Domain-specific pre-training (TINC) outperforms the use of ImageNet pre-trained weights in all cases, highlighting the importance of domain information for prediction tasks. These findings provide valuable insights for developing accurate and efficient models for predicting AMD progression.

Pretrained Deep 2.5D Models for Efficient Predictive Modeling from Retinal OCT

Simultaneous Alignment and Surface Regression Using Hybrid 2D-3D Networks for 3D Coherent Layer Segmentation of Retinal OCT Images with Full and Annotations

Optimizing Ocular Pathology Classification with CNNs and OCT Imaging: A Systematic and Performance Review

Leveraging 2D Deep Learning ImageNet-trained models for Native 3D Medical Image Analysis

3DTINC: Time-Equivariant Non-Contrastive Learning for Predicting Disease Progression From Longitudinal OCTs

Forecasting Disease Progression with Parallel Hyperplanes in Longitudinal Retinal OCT

2.75D: Boosting learning by representing 3D Medical imaging to 2D features for small data

Accurate prediction of disease-risk factors from volumetric medical scans by a deep vision model pre-trained with 2D scans

A Deep-learning Approach for Prognosis of Age-Related Macular Degeneration Disease using SD-OCT Imaging Biomarkers

Performance of a Deep Neural Network Algorithm Based on a Small Medical Image Dataset: Incremental Impact of 3D-to-2D Reformation Combined with Novel Data Augmentation, Photometric Conversion, or Transfer Learning

Enhancing Retinal Disease Classification from OCTA Images via Active Learning Techniques

Deep-Learning–Aided Diagnosis of Diabetic Retinopathy, Age-Related Macular Degeneration, and Glaucoma Based on Structural and Angiographic OCT

Super Images -- A New 2D Perspective on 3D Medical Imaging Analysis

Video Pretraining Advances 3D Deep Learning on Chest CT Tasks

Time-to-Event Pretraining for 3D Medical Imaging

Interpretable 2D Vision Models for 3D Medical Images

DISCOVER: 2-D Multiview Summarization of Optical Coherence Tomography Angiography for Automatic Diabetic Retinopathy Diagnosis

An Efficient Investigation on Age-Related Macular Degeneration Using Deep Learning with Cloud-Based Teleophthalmology Architecture

Recurrent and Concurrent Prediction of Longitudinal Progression of Stargardt Atrophy and Geographic Atrophy

DeepAD: A Robust Deep Learning Model of Alzheimer's Disease Progression for Real-World Clinical Applications

Deep Learning to Predict Glaucoma Progression using Structural Changes in the Eye