Abstract:Background and objective: Fundus fluorescein angiography (FFA) is widely used in clinical ophthalmic diagnosis and treatment with the requirement of adverse fluorescent dyes injection. Recently, many deep Convolutional Neural Network(CNN)-based methods have been proposed to estimate FFA from color fundus (CF) images to eliminate the use of adverse fluorescent dyes. However, the robustness of these methods is affected by pathological changes. Method: In this work, we present a CNN-based approach, lesion-aware generative adversarial networks (LA-GAN), to enhance the visual effect of lesion characteristics in the generated FFA images. First, we lead the generator notice lesion information by joint learning with lesion region segmentation. A new hierarchical correlation multi-task framework for high-resolution images is designed. Second, to enhance the visual contrast between normal regions and lesion regions, a newly designed region-level adversarial loss is used rather than the image-level adversarial loss. The code is publicly available at: https://github.com/nicetomeetu21/LA-GAN. Results: The effectiveness of LA-Net has been verified in data with branch retinal vein occlusion. The proposed model reported as measures of generation performance a mean structural similarity (SSIM) of 0.536, mean learned perceptual image patch similarity (LPIPS) 0.312, outperforming other FFA generation and general image generation methods. Further, due to the proposed multi-task learning framework, the lesion-region segmentation performance was further reported as the mean Dice increased from 0.714 to 0.797 and the mean accuracy increased from 0.873 to 0.905, outperforming general single-task image segmentation methods. Conclusions: The results show that the visual effect of lesion characteristics can be improved by employing the region-level adversarial loss and the hierarchical correlation multi-task framework respectively. Based on the results of comparison with the state-of-the-art methods, LA-GAN is not only effective for CF-to-FFA translation, but also effective for lesion-region segmentation. Thus, it may be used for various image translation and lesion segmentation tasks in future research.

Real-time Coloring Method of Laser Surgery Video Based on Generative Adversarial Network

Controlled Synthesis of Dermoscopic Images via a New Color Labeled Generative Style Transfer Network to Enhance Melanoma Segmentation

RIRGAN: an End-to-end Lightweight Multi-Task Learning Method for Brain MRI Super-Resolution and Denoising

Local Style Preservation in Improved GAN-Driven Synthetic Image Generation for Endoscopic Tool Segmentation

Lesion-aware generative adversarial networks for color fundus image to fundus fluorescein angiography translation

Guided image generation for improved surgical image segmentation

SkrGAN: Sketching-rendering Unconditional Generative Adversarial Networks for Medical Image Synthesis

Progressive Frequency-Aware Network for Laparoscopic Image Desmoking

Augmentation of Virtual Endoscopic Images with Intra-operative Data using Content-Nets

High-resolution medical image synthesis using progressively grown generative adversarial networks

Endoscopic Image Colorization Using Convolutional Neural Network

LC-GAN: Image-to-image Translation Based on Generative Adversarial Network for Endoscopic Images

Speckle reduction in laser illuminated endoscopy using adversarial deep learning

Color Restoration Method for Endoscope Image Using Multiscale Discriminator Based Model Compression Strategy

Generative models for color normalization in digital pathology and dermatology: Advancing the learning paradigm

Video Compression Coding via Colorization: A Generative Adversarial Network (GAN)-Based Approach

ROP-GAN: an image synthesis method for retinopathy of prematurity based on generative adversarial network

High Accuracy Tumor Diagnoses and Benchmarking of Hematoxylin and Eosin Stained Prostate Core Biopsy Images Generated by Explainable Deep Neural Networks

StereoCNC: A Stereovision-guided Robotic Laser System

Generative Adversarial Networks in Medical Image Processing

A new generative adversarial network for medical images super resolution