Abstract:An important challenge and limiting factor in deep learning methods for medical imaging segmentation is the lack of available of annotated data to properly train models. For the specific task of tumor segmentation, the process entails clinicians labeling every slice of volumetric scans for every patient, which becomes prohibitive at the scale of datasets required to train neural networks to optimal performance. To address this, we propose a novel semi-supervised framework that allows training any segmentation (encoder–decoder) model using only information readily available in radiological data, namely the presence of a tumor in the image, in addition to a few annotated images. Specifically, we conjecture that a generative model performing domain translation on this weak label — healthy vs diseased scans — helps achieve tumor segmentation. The proposed GenSeg method first disentangles tumoral tissue from healthy “background” tissue. The latent representation is separated into (1) the common background information across both domains, and (2) the unique tumoral information. GenSeg then achieves diseased-to-healthy image translation by decoding a healthy version of the image from just the common representation, as well as a residual image that allows adding back the tumors. The same decoder that produces this residual tumor image, also outputs a tumor segmentation. Implicit data augmentation is achieved by re-using the same framework for healthy-to-diseased image translation, where a residual tumor image is produced from a prior distribution. By performing both image translation and segmentation simultaneously, GenSeg allows training on only partially annotated datasets. To test the framework, we trained U-Net-like architectures using GenSeg and evaluated their performance on 3 variants of a synthetic task, as well as on 2 benchmark datasets: brain tumor segmentation in MRI (derived from BraTS) and liver metastasis segmentation in CT (derived from LiTS). Our method outperforms the baseline semi-supervised (autoencoder and mean teacher) and supervised segmentation methods, with improvements ranging between 8–14% Dice score on the brain task and 5–8% on the liver task, when only 1% of the training images were annotated. These results show the proposed framework is ideal at addressing the problem of training deep segmentation models when a large portion of the available data is unlabeled and unpaired, a common issue in tumor segmentation.

Parasitic GAN for Semi-Supervised Brain Tumor Segmentation.

Generative Adversarial Semi-Supervised Network for Medical Image Segmentation

Generative Adversarial Networks for Weakly Supervised Generation and Evaluation of Brain Tumor Segmentations on MR Images

Medical image synthesis via conditional GANs: Application to segmenting brain tumours

ToStaGAN: An end-to-end two-stage generative adversarial network for brain tumor segmentation

Medical image segmentation with generative adversarial semi-supervised network

Brain tumor image pixel segmentation and detection using an aggregation of GAN models with vision transformer

A Transformer-based Generative Adversarial Network for Brain Tumor Segmentation

Synthetic augmentation for semantic segmentation of class imbalanced biomedical images: A data pair generative adversarial network approach

Synthesizing brain tumor images and annotations by combining progressive growing GAN and SPADE

SegAN: Adversarial Network with Multi-scale $L_1$ Loss for Medical Image Segmentation

An Augmented Modulated Deep Learning Based Intelligent Predictive Model for Brain Tumor Detection Using GAN Ensemble

Cross-domain attention-guided generative data augmentation for medical image analysis with limited data

Generative adversarial networks in medical image segmentation: A review

A Two-Stage Generative Model with CycleGAN and Joint Diffusion for MRI-based Brain Tumor Detection

Annotation-Free Gliomas Segmentation Based on a Few Labeled General Brain Tumor Images

Brain Tumor Classification Using a Combination of Variational Autoencoders and Generative Adversarial Networks

Brain Tumor Synthetic Segmentation in 3D Multimodal MRI Scans

Improved brain metastases segmentation using generative adversarial network and conditional random field optimization mask RвҖҗCNN

Improvement of Multiparametric MR Image Segmentation by Augmenting the Data With Generative Adversarial Networks for Glioma Patients

Towards annotation-efficient segmentation via image-to-image translation