HSD-PAM: High Speed Super Resolution Deep Penetration Photoacoustic Microscopy Imaging Boosted by Dual Branch Fusion Network

Zhengyuan Zhang,Haoran Jin,Zesheng Zheng,Wenwen Zhang,Wenhao Lu,Feng Qin,Arunima Sharma,Manojit Pramanik,Yuanjin Zheng
2023-08-09
Abstract:Photoacoustic microscopy (PAM) is a novel implementation of photoacoustic imaging (PAI) for visualizing the 3D bio-structure, which is realized by raster scanning of the tissue. However, as three involved critical imaging parameters, imaging speed, lateral resolution, and penetration depth have mutual effect to one the other. The improvement of one parameter results in the degradation of other two parameters, which constrains the overall performance of the PAM system. Here, we propose to break these limitations by hardware and software co-design. Starting with low lateral resolution, low sampling rate AR-PAM imaging which possesses the deep penetration capability, we aim to enhance the lateral resolution and up sampling the images, so that high speed, super resolution, and deep penetration for the PAM system (HSD-PAM) can be achieved. Data-driven based algorithm is a promising approach to solve this issue, thereby a dedicated novel dual branch fusion network is proposed, which includes a high resolution branch and a high speed branch. Since the availability of switchable AR-OR-PAM imaging system, the corresponding low resolution, undersample AR-PAM and high resolution, full sampled OR-PAM image pairs are utilized for training the network. Extensive simulation and in vivo experiments have been conducted to validate the trained model, enhancement results have proved the proposed algorithm achieved the best perceptual and quantitative image quality. As a result, the imaging speed is increased 16 times and the imaging lateral resolution is improved 5 times, while the deep penetration merit of AR-PAM modality is still reserved.
Image and Video Processing,Medical Physics,Optics
What problem does this paper attempt to address?
The problem that this paper attempts to solve is the challenge of simultaneously improving the imaging speed, lateral resolution and penetration depth in photoacoustic microscopy (PAM). Specifically, in traditional PAM systems, when one parameter is improved, the performance of the other two parameters usually deteriorates, which limits the overall performance of the system. For example, optical - resolution PAM (OR - PAM) can provide high - resolution imaging, but its penetration depth is shallow and the imaging speed is slow; while acoustic - resolution PAM (AR - PAM) has a deeper penetration ability and a faster imaging speed, but has a lower resolution. To overcome these limitations, the paper proposes a method of co - design of hardware and software. By using AR - PAM imaging with low lateral resolution and low sampling rate as input, it aims to enhance the lateral resolution of the image and increase the sampling rate, so as to realize a high - speed, super - resolution and deep - penetration PAM system (HSD - PAM). To this end, the researchers developed a two - branch fusion network, which includes a high - resolution branch and a high - speed branch, for processing AR - PAM images, and improve the image quality and imaging speed by training the network. The paper verifies the effectiveness of the proposed algorithm and system through extensive simulations and in - vivo experiments. The results show that the imaging speed is increased by 16 times, the lateral resolution is improved by 5 times, while retaining the deep - penetration advantage of the AR - PAM mode.