DISGAN: Wavelet-informed Discriminator Guides GAN to MRI Super-resolution with Noise Cleaning

Qi Wang,Lucas Mahler,Julius Steiglechner,Florian Birk,Klaus Scheffler,Gabriele Lohmann

2023-08-23

Abstract:MRI super-resolution (SR) and denoising tasks are fundamental challenges in the field of deep learning, which have traditionally been treated as distinct tasks with separate paired training data. In this paper, we propose an innovative method that addresses both tasks simultaneously using a single deep learning model, eliminating the need for explicitly paired noisy and clean images during training. Our proposed model is primarily trained for SR, but also exhibits remarkable noise-cleaning capabilities in the super-resolved images. Instead of conventional approaches that introduce frequency-related operations into the generative process, our novel approach involves the use of a GAN model guided by a frequency-informed discriminator. To achieve this, we harness the power of the 3D Discrete Wavelet Transform (DWT) operation as a frequency constraint within the GAN framework for the SR task on magnetic resonance imaging (MRI) data. Specifically, our contributions include: 1) a 3D generator based on residual-in-residual connected blocks; 2) the integration of the 3D DWT with $1\times 1$ convolution into a DWT+conv unit within a 3D Unet for the discriminator; 3) the use of the trained model for high-quality image SR, accompanied by an intrinsic denoising process. We dub the model "Denoising Induced Super-resolution GAN (DISGAN)" due to its dual effects of SR image generation and simultaneous denoising. Departing from the traditional approach of training SR and denoising tasks as separate models, our proposed DISGAN is trained only on the SR task, but also achieves exceptional performance in denoising. The model is trained on 3D MRI data from dozens of subjects from the Human Connectome Project (HCP) and further evaluated on previously unseen MRI data from subjects with brain tumours and epilepsy to assess its denoising and SR performance.

Image and Video Processing,Computer Vision and Pattern Recognition

What problem does this paper attempt to address?

The problem that this paper attempts to solve is to simultaneously achieve super - resolution (SR) and denoising of MRI images in the field of medical imaging. Traditional deep - learning methods usually regard these two tasks as independent tasks and are trained with paired training data sets respectively. However, this method not only requires a large amount of paired data, but also lacks robustness when dealing with noisy images. The paper proposes a new method - DISGAN (Denoising Induced Super - resolution GAN), aiming to complete these two tasks simultaneously through a single deep - learning model without explicitly providing paired noisy and clean images as training data.

Specifically, the DISGAN model is mainly trained for super - resolution, but also shows significant denoising ability in the generated super - resolution images. To achieve this goal, the paper introduces a discriminator guided by frequency information, using 3D discrete wavelet transform (3D DWT) operations as frequency constraints in the GAN framework to handle the super - resolution task of MRI data. The main contributions of the paper include:

1. **3D Generator**: A 3D generator based on Residual - in - Residual Connected Blocks.
2. **Integration of 3D DWT and 1x1 Convolution**: The integration of 3D DWT and 1x1 convolution in the discriminator of the 3D U - Net structure to form DWT + conv units.
3. **High - Quality Image Super - Resolution and Intrinsic Denoising**: The trained model can generate high - quality super - resolution images, accompanied by an intrinsic denoising process.

Through these innovations, the DISGAN model not only performs excellently in the super - resolution task, but also achieves remarkable results in denoising, especially when dealing with simulated noise data and real - world images. The paper is evaluated on multiple data sets, including data from the Human Connectome Project (HCP) and data from brain tumor and epilepsy patients, demonstrating its superior performance in super - resolution and denoising.