Abstract:An important challenge and limiting factor in deep learning methods for medical imaging segmentation is the lack of available of annotated data to properly train models. For the specific task of tumor segmentation, the process entails clinicians labeling every slice of volumetric scans for every patient, which becomes prohibitive at the scale of datasets required to train neural networks to optimal performance. To address this, we propose a novel semi-supervised framework that allows training any segmentation (encoder–decoder) model using only information readily available in radiological data, namely the presence of a tumor in the image, in addition to a few annotated images. Specifically, we conjecture that a generative model performing domain translation on this weak label — healthy vs diseased scans — helps achieve tumor segmentation. The proposed GenSeg method first disentangles tumoral tissue from healthy “background” tissue. The latent representation is separated into (1) the common background information across both domains, and (2) the unique tumoral information. GenSeg then achieves diseased-to-healthy image translation by decoding a healthy version of the image from just the common representation, as well as a residual image that allows adding back the tumors. The same decoder that produces this residual tumor image, also outputs a tumor segmentation. Implicit data augmentation is achieved by re-using the same framework for healthy-to-diseased image translation, where a residual tumor image is produced from a prior distribution. By performing both image translation and segmentation simultaneously, GenSeg allows training on only partially annotated datasets. To test the framework, we trained U-Net-like architectures using GenSeg and evaluated their performance on 3 variants of a synthetic task, as well as on 2 benchmark datasets: brain tumor segmentation in MRI (derived from BraTS) and liver metastasis segmentation in CT (derived from LiTS). Our method outperforms the baseline semi-supervised (autoencoder and mean teacher) and supervised segmentation methods, with improvements ranging between 8–14% Dice score on the brain task and 5–8% on the liver task, when only 1% of the training images were annotated. These results show the proposed framework is ideal at addressing the problem of training deep segmentation models when a large portion of the available data is unlabeled and unpaired, a common issue in tumor segmentation.

Instance Segmentation of Unlabeled Modalities via Cyclic Segmentation GAN

SynSeg-Net: Synthetic Segmentation Without Target Modality Ground Truth

CNS - CycleGAN-Assisted Neonatal Segmentation Model for Cross-Datasets.

Few-shot 3D Multi-modal Medical Image Segmentation using Generative Adversarial Learning

Two-stage Generative Models of Simulating Training Data at the Voxel Level for Large-Scale Microscopy Bioimage Segmentation

LC-GAN: Image-to-image Translation Based on Generative Adversarial Network for Endoscopic Images

Unsupervised domain adaptation for nuclei segmentation: Adapting from hematoxylin & eosin stained slides to immunohistochemistry stained slides using a curriculum approach

SUSAN: Segment Unannotated image Structure using Adversarial Network

Towards cross-modal organ translation and segmentation: A cycle- and shape-consistent generative adversarial network

ISGAN: Unsupervised Domain Adaptation with Improved Symmetric GAN for Cross-Modality Multi-organ Segmentation

CAM-CycleGAN: A Weakly Supervised Segmentation Method for Medical Images Based on Cycle Consistensy Generative Adversarial Networks

Towards annotation-efficient segmentation via image-to-image translation

Unified Generative Adversarial Networks for Multimodal Segmentation from Unpaired 3D Medical Images.

Diverse Data Augmentation for Learning Image Segmentation with Cross-Modality Annotations.

Image-to-image translation of label-free molecular vibrational images for a histopathological review using the UNet+/seg-cGAN model

SegAN: Adversarial Network with Multi-scale $L_1$ Loss for Medical Image Segmentation

Shape-Consistent Generative Adversarial Networks for Multi-Modal Medical Segmentation Maps

Deep Adversarial Training for Multi-Organ Nuclei Segmentation in Histopathology Images

The Domain Shift Problem of Medical Image Segmentation and Vendor-Adaptation by Unet-GAN

Semantic Segmentation with Generative Models: Semi-Supervised Learning and Strong Out-of-Domain Generalization

DualMMP-GAN: Dual-scale Multi-Modality Perceptual Generative Adversarial Network for Medical Image Segmentation