Super-resolution imaging through a multimode fiber: the physical upsampling of speckle-driven

Chuncheng Zhang,Tingting Liu,Zhihua Xie,Yu Wang,Tong Liu,Qian Chen,Xiubao Sui

2023-07-11

Abstract:Following recent advancements in multimode fiber (MMF), miniaturization of imaging endoscopes has proven crucial for minimally invasive surgery in vivo. Recent progress enabled by super-resolution imaging methods with a data-driven deep learning (DL) framework has balanced the relationship between the core size and resolution. However, most of the DL approaches lack attention to the physical properties of the speckle, which is crucial for reconciling the relationship between the magnification of super-resolution imaging and the quality of reconstruction quality. In the paper, we find that the interferometric process of speckle formation is an essential basis for creating DL models with super-resolution imaging. It physically realizes the upsampling of low-resolution (LR) images and enhances the perceptual capabilities of the models. The finding experimentally validates the role played by the physical upsampling of speckle-driven, effectively complementing the lack of information in data-driven. Experimentally, we break the restriction of the poor reconstruction quality at great magnification by inputting the same size of the speckle with the size of the high-resolution (HR) image to the model. The guidance of our research for endoscopic imaging may accelerate the further development of minimally invasive surgery.

Optics,Image and Video Processing,Medical Physics

What problem does this paper attempt to address?

The problem that this paper attempts to solve is how to achieve high - magnification while maintaining image quality during super - resolution imaging through multimode fiber (MMF). Specifically, most existing deep - learning (DL) methods lack attention to the physical characteristics of speckle when dealing with super - resolution imaging, which leads to poor image reconstruction quality at high - magnification. The paper proposes a speckle - driven physical up - sampling method, which utilizes the interference characteristics of speckle to achieve physical pixel expansion of low - resolution (LR) images, thereby improving the perceptual ability and reconstruction quality of the image.

### Main problems:
1. **Image quality at high - magnification**: Existing methods have poor image reconstruction quality at high - magnification, especially in terms of detail preservation.
2. **Utilization of speckle physical characteristics**: Most deep - learning methods lack attention to the physical characteristics of speckle, resulting in an inability to fully utilize speckle information redundancy at high - magnification.
3. **Limitations of data - driven methods**: Data - driven methods rely on the correlation between pixel values, but at high - magnification, this correlation is not sufficient to provide enough information, resulting in poor reconstruction effects.

### Solutions:
The paper proposes a speckle - driven physical up - sampling method, which specifically includes the following aspects:
- **Interference characteristics of speckle**: Use the interference process of speckle to achieve physical pixel expansion and compensate for the perceptual loss in data - driven methods.
- **Experimental verification**: Generate speckle patterns of the same size as high - resolution (HR) images through experiments for training the model, effectively solving the information - missing problem of data - driven methods during up - sampling.
- **Deep - learning model**: Use the U - Net model combined with techniques such as convolutional layers, dense blocks, skip connections, and PixelShuffle to improve the quality and generalization ability of image reconstruction.

### Experimental results:
- **Subjective evaluation**: At different magnifications, whether for known or unknown objects, the visual effects of the reconstructed images are very good. Especially at 8 - fold magnification, the topological structure and local details of the image are nearly perfect.
- **Quantitative evaluation**: Evaluation is carried out through indicators such as Pearson correlation coefficient (PCC), peak signal - to - noise ratio (PSNR), and structural similarity index (SSIM), and the results show that this method has significant advantages at high - magnification.

### Conclusion:
The speckle - driven physical up - sampling method proposed in the paper can effectively overcome the problem of image quality degradation of existing methods at high - magnification and improve the quality and generalization ability of super - resolution imaging. This method is expected to play an important role in minimally invasive surgery.