Abstract:The lack of large datasets and high-quality annotated data often limits the development of accurate and robust machine-learning models within the medical and surgical domains. In the machine learning community, generative models have recently demonstrated that it is possible to produce novel and diverse synthetic images that closely resemble reality while controlling their content with various types of annotations. However, generative models have not been yet fully explored in the surgical domain, partially due to the lack of large datasets and due to specific challenges present in the surgical domain such as the large anatomical diversity. We propose Surgery-GAN, a novel generative model that produces synthetic images from segmentation maps. Our architecture produces surgical images with improved quality when compared to early generative models thanks to the combination of channel- and pixel-level normalization layers that boost image quality while granting adherence to the input segmentation map. While state-of-the-art generative models often generate overfitted images, lacking diversity, or containing unrealistic artefacts such as cartooning; experiments demonstrate that Surgery-GAN is able to generate novel, realistic, and diverse surgical images in three different surgical datasets: cholecystectomy, partial nephrectomy, and radical prostatectomy. In addition, we investigate whether the use of synthetic images together with real ones can be used to improve the performance of other machine-learning models. Specifically, we use Surgery-GAN to generate large synthetic datasets which we then use to train five different segmentation models. Results demonstrate that using our synthetic images always improves the mean segmentation performance with respect to only using real images. For example, when considering radical prostatectomy, we can boost the mean segmentation performance by up to 5.43%. More interestingly, experimental results indicate that the performance improvement is larger in the set of classes that are under-represented in the training sets, where the performance boost of specific classes reaches up to 61.6%.

Laparoscopy Surgery CO2 Removal via Generative Adversary Network and Dark Channel Prior

Desmoking laparoscopy surgery images using an image-to-image translation guided by an embedded dark channel

Progressive Frequency-Aware Network for Laparoscopic Image Desmoking

Generative Smoke Removal

MARS-GAN: Multilevel-Feature-Learning Attention-Aware Based Generative Adversarial Network for Removing Surgical Smoke

Attention-Aware Laparoscopic Image Desmoking Network with Lightness Embedding and Hybrid Guided Embedding

A Smoke Removal Method for Laparoscopic Images

A Self-Supervised Network-Based Smoke Removal and Depth Estimation for Monocular Endoscopic Videos

LSD3K: A Benchmark for Smoke Removal from Laparoscopic Surgery Images

Can Image Enhancement be Beneficial to Find Smoke Images in Laparoscopic Surgery?

A Survey on Deep Learning Assisted Video Quality Enhancement in Laparoscopic Videos

SUPR-GAN: SUrgical PRediction GAN for Event Anticipation in Laparoscopic and Robotic Surgery

A Temporal Learning Approach to Inpainting Endoscopic Specularities and Its effect on Image Correspondence

Automatic Smoke Analysis in Minimally Invasive Surgery by Image-based Machine Learning

Augmentation of Virtual Endoscopic Images with Intra-operative Data using Content-Nets

Single Image Desmoking via Attentive Generative Adversarial Network for Smoke Detection Process

Feasibility of a Generative Adversarial Network for Artifact Removal in Experimental Photoacoustic Imaging

Using Conditional Generative Adversarial Networks to Reduce the Effects of Latency in Robotic Telesurgery

Guided image generation for improved surgical image segmentation

Self-Supervised Video Desmoking for Laparoscopic Surgery