Abstract:Generative adversarial network (GAN) has been widely explored for cross-modality medical image synthesis. The existing GAN models usually adversarially learn a global sample space mapping from the source-modality to the target-modality and then indiscriminately apply this mapping to all samples in the whole space for prediction. However, due to the scarcity of training samples in contrast to the complicated nature of medical image synthesis, learning a single global sample space mapping that is “optimal” to all samples is very challenging, if not intractable. To address this issue, this paper proposes sample-adaptive GAN models, which not only cater for the global sample space mapping between the source- and the target-modalities but also explore the local space around each given sample to extract its unique characteristic. Specifically, the proposed sample-adaptive GANs decompose the entire learning model into two cooperative paths. The baseline path learns a common GAN model by fitting all the training samples as usual for the global sample space mapping. The new sample-adaptive path additionally models each sample by learning its relationship with its neighboring training samples and using the target-modality features of these training samples as auxiliary information for synthesis. Enhanced by this sample-adaptive path, the proposed sample-adaptive GANs are able to flexibly adjust themselves to different samples, and therefore optimize the synthesis performance. Our models have been verified on three cross-modality MR image synthesis tasks from two public datasets, and they significantly outperform the state-of-the-art methods in comparison. Moreover, the experiment also indicates that our sample-adaptive strategy could be utilized to improve various backbone GAN models. It complements the existing GANs models and can be readily integrated when needed.

GL-GAN: Adaptive Global and Local Bilevel Optimization for Generative Adversarial Network

GL-GAN: Adaptive Global and Local Bilevel Optimization model of Image Generation

Incremental Focal Loss GANs.

Two Birds with One Stone: Iteratively Learn Facial Attributes with GANs.

UGC: Unified GAN Compression for Efficient Image-to-Image Translation

Local and Global GANs with Semantic-Aware Upsampling for Image Generation

Progressive Growing of GANs for Improved Quality, Stability, and Variation

Two Birds with One Stone: Transforming and Generating Facial Images with Iterative GAN

Improving GAN Training via Feature Space Shrinkage

GIU-GANs: Global Information Utilization for Generative Adversarial Networks

Adaptive Consensus Optimization Method for GANs

Biphasic Learning of GANs for High-Resolution Image-to-Image Translation

Improving OPT-GAN by Smooth Scale Mapping and Adaptive Exploration

Sample-adaptive GANs: Linking Global and Local Mappings for Cross-modality MR Image Synthesis.

Improving Generative Adversarial Networks with Local Coordinate Coding

Improving the Speed and Quality of GAN by Adversarial Training

Image generation step by step: animation generation-image translation

CDE-GAN: Cooperative Dual Evolution Based Generative Adversarial Network

Generative Adversarial Networks with Learnable Auxiliary Module for Image Synthesis

InfoMax-GAN: Improved Adversarial Image Generation via Information Maximization and Contrastive Learning

Improving GAN with Neighbors Embedding and Gradient Matching