SAM-OCTA2: Layer Sequence OCTA Segmentation with Fine-tuned Segment Anything Model 2

Xinrun Chen,Chengliang Wang,Haojian Ning,Mengzhan Zhang,Mei Shen,Shiying Li

2024-09-14

Abstract:Segmentation of indicated targets aids in the precise analysis of optical coherence tomography angiography (OCTA) samples. Existing segmentation methods typically perform on 2D projection targets, making it challenging to capture the variance of segmented objects through the 3D volume. To address this limitation, the low-rank adaptation technique is adopted to fine-tune the Segment Anything Model (SAM) version 2, enabling the tracking and segmentation of specified objects across the OCTA scanning layer sequence. To further this work, a prompt point generation strategy in frame sequence and a sparse annotation method to acquire retinal vessel (RV) layer masks are proposed. This method is named SAM-OCTA2 and has been experimented on the OCTA-500 dataset. It achieves state-of-the-art performance in segmenting the foveal avascular zone (FAZ) on regular 2D en-face and effectively tracks local vessels across scanning layer sequences. The code is available at: <a class="link-external link-https" href="https://github.com/ShellRedia/SAM-OCTA2" rel="external noopener nofollow">this https URL</a>.

Computer Vision and Pattern Recognition

What problem does this paper attempt to address?

This paper attempts to solve the problem of retinal vessel (RV) and foveal avascular zone (FAZ) segmentation in optical coherence tomography angiography (OCTA). Specifically, existing segmentation methods are usually carried out on 2D projection images and it is difficult to capture the changes of segmented objects in 3D volumes. To overcome this limitation, the authors propose a method of fine - tuning Segment Anything Model 2 (SAM 2) based on low - rank adaptation technology, named SAM - OCTA2.

### Main problems:
1. **Limitations of existing methods**: Most of the existing segmentation methods are based on 2D projection images and cannot effectively capture the changes of objects in 3D volumes.
2. **Accurate segmentation in 3D volumes**: A method that can track and segment specified objects in 3D volumes is required to improve the accurate analysis of retinal vessels and foveal avascular zones.
3. **Lack of annotation data**: The currently publicly available OCTA datasets lack hierarchical retinal vessel annotations, which limits the effectiveness of model training.

### Solutions:
- **Fine - tuning SAM 2 with low - rank adaptation technology**: Fine - tune SAM 2 through low - rank adaptation (LoRA) technology so that it can process OCTA scan layer sequences and achieve accurate segmentation of specified objects (such as retinal vessels and foveal avascular zones).
- **Prompt point generation strategy**: Propose a prompt point generation strategy for identifying and indicating local objects in the frame sequence.
- **Sparse annotation method**: Design a sparse annotation method to provide retinal vessel layer annotations for OCTA volume samples, solving the problem of lack of annotation data.

### Experimental results:
- **2D projection image segmentation**: Achieve accurate segmentation of the foveal avascular zone (FAZ) on conventional 2D projection images.
- **Layer sequence segmentation**: Effectively track and segment local vessels in the scan layer sequence, demonstrating its superior performance in 3D volumes.

Through these improvements, SAM - OCTA2 has achieved state - of - the - art performance in experiments on the OCTA - 500 dataset, especially in segmenting the foveal avascular zone (FAZ).

### Formula display:
The Dice coefficient and Jaccard index are used to evaluate the segmentation results. The formulas are as follows:

\[
\text{Dice}(\hat{Y}, Y)=\frac{2|\hat{Y} \cap Y|}{|\hat{Y}|+|Y|}
\]

\[
\text{Jaccard}(\hat{Y}, Y)=\frac{|\hat{Y} \cap Y|}{|\hat{Y} \cup Y|}
\]

where \(Y\) represents the ground - truth label and \(\hat{Y}\) represents the predicted value.

Through these methods, SAM - OCTA2 significantly improves the accuracy and robustness of OCTA image segmentation, providing strong support for the diagnosis of retinal diseases and 3D structure reconstruction.