Abstract:Background and objective: Fundus fluorescein angiography (FFA) is widely used in clinical ophthalmic diagnosis and treatment with the requirement of adverse fluorescent dyes injection. Recently, many deep Convolutional Neural Network(CNN)-based methods have been proposed to estimate FFA from color fundus (CF) images to eliminate the use of adverse fluorescent dyes. However, the robustness of these methods is affected by pathological changes. Method: In this work, we present a CNN-based approach, lesion-aware generative adversarial networks (LA-GAN), to enhance the visual effect of lesion characteristics in the generated FFA images. First, we lead the generator notice lesion information by joint learning with lesion region segmentation. A new hierarchical correlation multi-task framework for high-resolution images is designed. Second, to enhance the visual contrast between normal regions and lesion regions, a newly designed region-level adversarial loss is used rather than the image-level adversarial loss. The code is publicly available at: https://github.com/nicetomeetu21/LA-GAN. Results: The effectiveness of LA-Net has been verified in data with branch retinal vein occlusion. The proposed model reported as measures of generation performance a mean structural similarity (SSIM) of 0.536, mean learned perceptual image patch similarity (LPIPS) 0.312, outperforming other FFA generation and general image generation methods. Further, due to the proposed multi-task learning framework, the lesion-region segmentation performance was further reported as the mean Dice increased from 0.714 to 0.797 and the mean accuracy increased from 0.873 to 0.905, outperforming general single-task image segmentation methods. Conclusions: The results show that the visual effect of lesion characteristics can be improved by employing the region-level adversarial loss and the hierarchical correlation multi-task framework respectively. Based on the results of comparison with the state-of-the-art methods, LA-GAN is not only effective for CF-to-FFA translation, but also effective for lesion-region segmentation. Thus, it may be used for various image translation and lesion segmentation tasks in future research.

Improved Cgan Based Linear Lesion Segmentation in High Myopia ICGA Images

SGCNet: a scale-aware and global context network for linear lesion segmentation in MCSL fundus images of high myopia

Attention-Guided Network with Densely Connected Convolution for Skin Lesion Segmentation

Multimodal Affine Registration for ICGA and MCSL Fundus Images of High Myopia

Applying Conditional Generative Adversarial Networks for Imaging Diagnosis

Skin Lesion Segmentation through Generative Adversarial Networks with Global and Local Semantic Feature Awareness

Lesion-aware generative adversarial networks for color fundus image to fundus fluorescein angiography translation

Simultaneous Classification and Segmentation of Subretinal Lesions on ICGA Images

Lesion Conditional Image Generation for Improved Segmentation of Intracranial Hemorrhage from CT Images

GAN-based Super-Resolution and Segmentation of Retinal Layers in Optical coherence tomography Scans

Interpretable Semantic-aware Unsupervised Segmentation of Skin Lesions

A generative adversarial neural network with multi-attention feature extraction for fundus lesion segmentation

A GCN-assisted Deep Learning Method for Peripapillary Retinal Layer Segmentation in OCT Images

Segmentation of skin lesion using superpixel guided generative adversarial network with dual-stream patch-based discriminators

Group-wise Context Selection Network for Choroid Segmentation in Optical Coherence Tomography

A Hybrid Approach for Segmenting Non-ideal Iris Images Using CGAN and Geometry Constraints.

Multi-scale GCN-assisted Two-Stage Network for Joint Segmentation of Retinal Layers and Discs in Peripapillary OCT Images.

Transfer Learning and Interpretable Analysis-Based Quality Assessment of Synthetic Optical Coherence Tomography Images by CGAN Model for Retinal Diseases

Improved generative adversarial network for retinal image super-resolution

Improving Lesion Segmentation for Diabetic Retinopathy using Adversarial Learning

Generative Adversarial Network Combined with SE-ResNet and Dilated Inception Block for Segmenting Retinal Vessels