Unpaired Optical Coherence Tomography Angiography Image Super-Resolution via Frequency-Aware Inverse-Consistency GAN

Weiwen Zhang,Dawei Yang,Haoxuan Che,An Ran Ran,Carol Y. Cheung,Hao Chen
2023-09-29
Abstract:For optical coherence tomography angiography (OCTA) images, a limited scanning rate leads to a trade-off between field-of-view (FOV) and imaging resolution. Although larger FOV images may reveal more parafoveal vascular lesions, their application is greatly hampered due to lower resolution. To increase the resolution, previous works only achieved satisfactory performance by using paired data for training, but real-world applications are limited by the challenge of collecting large-scale paired images. Thus, an unpaired approach is highly demanded. Generative Adversarial Network (GAN) has been commonly used in the unpaired setting, but it may struggle to accurately preserve fine-grained capillary details, which are critical biomarkers for OCTA. In this paper, our approach aspires to preserve these details by leveraging the frequency information, which represents details as high-frequencies ($\textbf{hf}$) and coarse-grained backgrounds as low-frequencies ($\textbf{lf}$). In general, we propose a GAN-based unpaired super-resolution method for OCTA images and exceptionally emphasize $\textbf{hf}$ fine capillaries through a dual-path generator. To facilitate a precise spectrum of the reconstructed image, we also propose a frequency-aware adversarial loss for the discriminator and introduce a frequency-aware focal consistency loss for end-to-end optimization. Experiments show that our method outperforms other state-of-the-art unpaired methods both quantitatively and visually.
Image and Video Processing,Computer Vision and Pattern Recognition
What problem does this paper attempt to address?
This paper attempts to address the challenges encountered in super - resolution reconstruction of optical coherence tomography angiography (OCTA) images, especially improving the resolution of 6mm×6mm OCTA images in the unpaired data setting to make their quality close to that of 3mm×3mm images. Specific problems include: 1. **Trade - off between field of view and resolution**: The scanning rate of commercial OCT instruments is limited, resulting in lower image resolution for a larger field of view (FOV), which restricts its clinical applications. 2. **Preservation of fine - grained capillary details**: Existing methods have difficulty in accurately preserving important fine - grained capillary details when using generative adversarial networks (GANs) in the unpaired setting, and these details are key biomarkers in OCTA images. 3. **Difficulty in collecting large - scale paired data**: In reality, it is very difficult to obtain large - scale paired high - resolution and low - resolution images, which limits the application of paired - data training methods. To solve these problems, the authors propose a method based on Frequency - Aware Inverse - Consistency GAN (FAIC - GAN) for unpaired OCTA image super - resolution reconstruction. The main contributions of this method include: - **Introduction of frequency information**: By separating high - frequency (hf) and low - frequency (lf) components and adopting a dual - path structure in the generator to process these components respectively, the fine - grained capillary details can be better preserved. - **Frequency - Aware Adversarial Loss (FAL)**: A frequency - aware adversarial loss is designed for the discriminator to improve its sensitivity to frequency information. - **Frequency - Aware Focal Consistency Loss (FFCL)**: A new loss function is introduced to ensure that the spectral distribution of the reconstructed image is consistent with that of the real high - resolution image, with particular emphasis on the accuracy of the high - frequency components. Through these technical means, the paper aims to improve the resolution of 6mm×6mm OCTA images while ensuring the accuracy and integrity of fine - grained capillary details, thereby providing ophthalmologists with better tools to assess microvascular lesions and develop personalized treatment plans. ### Formula Summary 1. **Fourier Transform Formula**: \[ X(u,v)=\sum_{m = 0}^{M - 1}\sum_{n = 0}^{N - 1}x(m,n)e^{-i2\pi(um/M+vn/N)}=R(u,v)+iI(u,v) \] where \(X\) is the frequency representation of the image \(x\), and \(R(u,v)\) and \(I(u,v)\) are the real part and the imaginary part respectively. 2. **Azimuthal Integration Formula**: \[ A(\omega_k)=\int_{0}^{2\pi}\|X(\omega_k\cdot\cos(\phi),\omega_k\cdot\sin(\phi))\|^2d\phi \] which is used to visualize the spectral power. 3. **Gaussian Kernel Definition**: \[ G(u,v)=\frac{1}{2\pi\sigma^2}e^{-(u^2 + v^2)/(2\sigma^2)} \] 4. **High - Frequency Enhancement Operation**: \[ hf^*=x+\alpha\cdot hf \] 5. **Inverse - Consistency Loss**: \[ L_{\text{Res}}^{\text{inv}}(G_{\text{Res}},G_{\text{Deg}},LR)=\mathbb{E}[\|LR^{\uparrow\downarrow}-LR\|_1] \] \[ L_{\text{Deg}}^{\text{inv}}(G_{\text{Deg}},G_{\text{Res}},HR)=\mathbb{E}[\|HR^{\downarrow\uparrow}-HR\|_1] \] 6. **Frequency - Aware Adversarial Loss**: