Ambient-Pix2PixGAN for Translating Medical Images from Noisy Data

Wentao Chen,Xichen Xu,Jie Luo,Weimin Zhou

2024-02-02

Abstract:Image-to-image translation is a common task in computer vision and has been rapidly increasing the impact on the field of medical imaging. Deep learning-based methods that employ conditional generative adversarial networks (cGANs), such as Pix2PixGAN, have been extensively explored to perform image-to-image translation tasks. However, when noisy medical image data are considered, such methods cannot be directly applied to produce clean images. Recently, an augmented GAN architecture named AmbientGAN has been proposed that can be trained on noisy measurement data to synthesize high-quality clean medical images. Inspired by AmbientGAN, in this work, we propose a new cGAN architecture, Ambient-Pix2PixGAN, for performing medical image-to-image translation tasks by use of noisy measurement data. Numerical studies that consider MRI-to-PET translation are conducted. Both traditional image quality metrics and task-based image quality metrics are employed to assess the proposed Ambient-Pix2PixGAN. It is demonstrated that our proposed Ambient-Pix2PixGAN can be successfully trained on noisy measurement data to produce high-quality translated images in target imaging modality.

Image and Video Processing,Computer Vision and Pattern Recognition

What problem does this paper attempt to address?

The problem that this paper attempts to solve is how to use generative adversarial networks (GANs) for the conversion between medical images in the presence of noisy data. Specifically, traditional conditional generative adversarial networks (cGANs), such as Pix2PixGAN, cannot be directly applied to generate high - quality clean images when dealing with noisy medical image data. This is because the data obtained by medical imaging systems usually contains noise and/or is incomplete, and training cGAN directly on these data will lead to a decline in image quality.

To overcome this challenge, the authors propose a new cGAN architecture - Ambient - Pix2PixGAN, which can be trained on noisy measurement data to synthesize high - quality clean medical images. This method simulates the measurement process on the "fake" objects generated by the generator and trains the discriminator to distinguish between the simulated measurement data and the real measurement data, thereby learning the underlying distribution of the object to be imaged and then generating high - quality images.

In the paper, the authors verify the performance of Ambient - Pix2PixGAN in the MRI - to - PET image conversion task through numerical research, evaluate traditional image quality metrics and task - based image quality metrics, and show that the proposed Ambient - Pix2PixGAN method can be successfully trained on noisy medical image data to produce high - quality converted images in the target imaging modality.