Abstract:<h3 class="u-h4 u-margin-m-top u-margin-xs-bottom">Objective</h3><p>This work aims to synthesize a real-like missing MRI modality using multiple modalities those already obtained, thus providing more abundant diagnostic information, and promoting the improvement of some downstream tasks, such as segmentation and diagnosis.</p><h3 class="u-h4 u-margin-m-top u-margin-xs-bottom">Methods</h3><p>With an adversarial network modelling the nonlinear mapping between the inputs and the output, our proposed LR-cGAN extracts the inherent latent representations from different MRI modalities with N collaboratively trained encoders, and fuses them by a latent space processing network (LSPN) composed of several residual blocks. Apart from L1 loss, the image gradient difference loss (GDL) is considered additionally as the objective function to alleviate the problem of insufficient image sharpening. To validate the effectiveness of LR-cGAN, corresponding experiments were evaluated by peak SNR (PSNR), structural similarity index (SSIM) and normalized root-mean-square error (NRMSE) on BRATS 2015 dataset.</p><h3 class="u-h4 u-margin-m-top u-margin-xs-bottom">Results</h3><p>Compared to single-modality input, two-modality input improves the synthesis results by 1.196 dB PSNR, 0.019 SSIM and 0.04 NRMSE. With more inputs added, the synthesis performance exhibits an increasing trend. Once any key component, that is, LSPN, GDL loss or adversarial loss, is removed, the quality of the results will reduce to a lower level, proving their contributions to our model. Meanwhile, the final performance of our LR-cGAN network outperforms REPLICA, M-GAN, MILR and sGAN in all metrics on different synthesis tasks, demonstrating its superiority.</p><h3 class="u-h4 u-margin-m-top u-margin-xs-bottom">Conclusion</h3><p>Our proposed LR-cGAN has the flexible ability of receiving multiple modalities and generating realistic images compared to real modality images, having the potential to supplement diagnostic information in clinical.</p>

Style transfer in conditional GANs for cross-modality synthesis of brain magnetic resonance images

Statistics Enhancement Generative Adversarial Networks for Diverse Conditional Image Synthesis

Synthesizing Contrast-enhanced Computed Tomography Images with an Improved Conditional Generative Adversarial Network

Style Fader Generative Adversarial Networks for Style Degree Controllable Artistic Style Transfer

Multi-Constraint Transferable Generative Adversarial Networks for Cross-Modal Brain Image Synthesis

LR-cGAN: Latent representation based conditional generative adversarial network for multi-modality MRI synthesis

Style transfer between Microscopy and Magnetic Resonance Imaging via Generative Adversarial Network in small sample size settings

Gated-GAN: Adversarial Gated Networks for Multi-Collection Style Transfer

Style Transformer for Image Inversion and Editing

MSGAN: Multi-Stage Generative Adversarial Networks for Cross-Modality Domain Adaptation.

Multimodality-guided Image Style Transfer using Cross-modal GAN Inversion

Image neural style transfer: A review

Multi-Modal MRI Image Synthesis via GAN With Multi-Scale Gate Mergence

Multimodal MR Image Synthesis Using Gradient Prior and Adversarial Learning

Image Neural Style Transfer with Preserving the Salient Regions.

ReeGAN: MRI image edge-preserving synthesis based on GANs trained with misaligned data

Cross-Modal Synthesis of Structural MRI and Functional Connectivity Networks via Conditional ViT-GANs

Ea-GANs: Edge-aware Generative Adversarial Networks for Cross-modality MR Image Synthesis.

Common feature learning for brain tumor MRI synthesis by context-aware generative adversarial network

Multi-Modality Generative Adversarial Networks with Tumor Consistency Loss for Brain MR Image Synthesis

A closer look at incidental findings on cardiac computed tomography.