Abstract:The lack of large datasets and high-quality annotated data often limits the development of accurate and robust machine-learning models within the medical and surgical domains. In the machine learning community, generative models have recently demonstrated that it is possible to produce novel and diverse synthetic images that closely resemble reality while controlling their content with various types of annotations. However, generative models have not been yet fully explored in the surgical domain, partially due to the lack of large datasets and due to specific challenges present in the surgical domain such as the large anatomical diversity. We propose Surgery-GAN, a novel generative model that produces synthetic images from segmentation maps. Our architecture produces surgical images with improved quality when compared to early generative models thanks to the combination of channel- and pixel-level normalization layers that boost image quality while granting adherence to the input segmentation map. While state-of-the-art generative models often generate overfitted images, lacking diversity, or containing unrealistic artefacts such as cartooning; experiments demonstrate that Surgery-GAN is able to generate novel, realistic, and diverse surgical images in three different surgical datasets: cholecystectomy, partial nephrectomy, and radical prostatectomy. In addition, we investigate whether the use of synthetic images together with real ones can be used to improve the performance of other machine-learning models. Specifically, we use Surgery-GAN to generate large synthetic datasets which we then use to train five different segmentation models. Results demonstrate that using our synthetic images always improves the mean segmentation performance with respect to only using real images. For example, when considering radical prostatectomy, we can boost the mean segmentation performance by up to 5.43%. More interestingly, experimental results indicate that the performance improvement is larger in the set of classes that are under-represented in the training sets, where the performance boost of specific classes reaches up to 61.6%.

A Method of Generating Synthetic Medical Image for the Validation of Segmentation Algorithms

Improved watershed transform for medical image segmentation using prior information

Informing selection of performance metrics for medical image segmentation evaluation using configurable synthetic errors

Point-Based Geometric Deformable Models for Medical Image Segmentation

Guided image generation for improved surgical image segmentation

Synthetic Data for Robust Stroke Segmentation

A Novel Multi-stage 3D Medical Image Segmentation: Methodology and Validation

SSIS-Seg: Simulation-Supervised Image Synthesis for Surgical Instrument Segmentation

Local Style Preservation in Improved GAN-Driven Synthetic Image Generation for Endoscopic Tool Segmentation

Aligning Synthetic Medical Images with Clinical Knowledge using Human Feedback

Computing Synthetic Echocardiography Volumes for Automatic Validation of 3d Segmentation Results

Coarse-to-fine Tuning Knowledgeable System for Boundary Delineation in Medical Images

Segmentation quality assessment by automated detection of erroneous surface regions in medical images

Image translation for medical image generation: Ischemic stroke lesion segmentation

Less is More: Unsupervised Mask-guided Annotated CT Image Synthesis with Minimum Manual Segmentations

Using Diffusion Models to Generate Synthetic Labelled Data for Medical Image Segmentation

A statistical deformation model-based data augmentation method for volumetric medical image segmentation

Fast interactive medical image segmentation with weakly supervised deep learning method

Enhancing Diagnostic Images to Improve the Performance of the Segment Anything Model in Medical Image Segmentation

sTBI-GAN: An adversarial learning approach for data synthesis on traumatic brain segmentation

Simulation-Based Segmentation of Blood Vessels in Cerebral 3D OCTA Images