Inter-slice Super-resolution of Magnetic Resonance Images by Pre-training and Self-supervised Fine-tuning

Xin Wang,Zhiyun Song,Yitao Zhu,Sheng Wang,Lichi Zhang,Dinggang Shen,Qian Wang
2024-06-10
Abstract:In clinical practice, 2D magnetic resonance (MR) sequences are widely adopted. While individual 2D slices can be stacked to form a 3D volume, the relatively large slice spacing can pose challenges for both image visualization and subsequent analysis tasks, which often require isotropic voxel spacing. To reduce slice spacing, deep-learning-based super-resolution techniques are widely investigated. However, most current solutions require a substantial number of paired high-resolution and low-resolution images for supervised training, which are typically unavailable in real-world scenarios. In this work, we propose a self-supervised super-resolution framework for inter-slice super-resolution of MR images. Our framework is first featured by pre-training on video dataset, as temporal correlation of videos is found beneficial for modeling the spatial relation among MR slices. Then, we use public high-quality MR dataset to fine-tune our pre-trained model, for enhancing awareness of our model to medical data. Finally, given a target dataset at hand, we utilize self-supervised fine-tuning to further ensure our model works well with user-specific super-resolution tasks. The proposed method demonstrates superior performance compared to other self-supervised methods and also holds the potential to benefit various downstream applications.
Image and Video Processing,Computer Vision and Pattern Recognition
What problem does this paper attempt to address?
The problem that this paper attempts to solve is: in clinical practice, although 2D magnetic resonance (MR) image sequences are widely used, their large slice spacing will pose challenges to image visualization and subsequent analysis tasks. These tasks usually require isotropic voxel spacing. To solve this problem, the existing deep - learning super - resolution (SR) techniques require a large number of high - resolution (HR) and low - resolution (LR) image pairs for supervised training, but in actual scenarios, such data are usually unavailable. Therefore, this paper proposes a self - supervised super - resolution framework for improving the inter - slice resolution of MR images. Specifically, this method is achieved through the following three stages: 1. **Pre - training**: Use a video data set for pre - training, because the video frame interpolation task is similar to the inter - slice super - resolution task of MR images, and both can capture the spatial relationship between adjacent frames / slices. \[ L=\| \hat{I}_{i + k}-I_{i + k}\|=\| F(I_i, I_{i + n}, C_{i + k})-I_{i + k}\| \] 2. **Supervised Fine - tuning**: Use a high - quality public MR data set to fine - tune the pre - trained model to enhance the model's understanding and performance of medical images. 3. **Self - supervised Fine - tuning**: Perform self - supervised fine - tuning for the specific data set provided by the user to ensure that the model can adapt to specific user needs and data characteristics. Through this three - stage method, the paper aims to overcome the problems of limited training data and indirect mapping in the existing self - supervised SR methods, and demonstrates superior performance on knee joint MR images.