Abstract:Photoacoustic imaging (PAI) is an emerging modality that has generated increasing interest for its uses in clinical research and translation. To fully exploit its potential for various preclinical and clinical applications, it is necessary to develop systems that offer high imaging speed, reasonable cost, and manageable data flow. Currently, a significant challenge lies in the fabrication of ultrasound arrays, as many of them are not densely populated enough to fully sample the signals. Ideally, the pitch of the arrays should be half of the center ultrasonic wavelength to prevent spatial undersampling and the subsequent reconstruction artifacts, such as aliasing artifacts and structural deformation. Here, a novel photoacoustic sparse sampling Transformer-CNN coupling network (passFormer) is proposed to decouple target details and spatial under-sampling artifacts from high-frequency image information in a heterogeneous feature-aware manner. To be specific, we first decompose the sparse sampling (SS) photoacoustic (PA) images into 2 parts: high-frequency(HF) and low-frequency(LF) compositions. Our methodology incorporates two bridging modules, the LF modules and the HF modules. The LF coupling module extracts content features (X-c) and latent texture feature (X-tex), and the HF coupling module extracts high-frequency embedding (X-emb) containing the target details features (X-detail) and under-sampling artifacts (X-art). We feed X-t and X-emb into a modified transformer with three encoders and decoders to obtain well-refined HF texture features. At last, we combine the refined HF texture features with pre-extracted X-c by pixel-wise summation reconstruction. Experimental results on publicly available full and sparse reconstruction datasets of mouse and phantom PA images highlight the superior performance of our method, particularly in live mouse imaging. This new approach enables accelerated data acquisition and image reconstruction, facilitating the development of practical and cost-effective imaging systems.

PSCAT: a lightweight transformer for simultaneous denoising and super-resolution of OCT images

Improved Sparse Representation Algorithm for Optical Coherence Tomography Images

A generative adversarial network with multi-scale convolution and dilated convolution res-network for OCT retinal image despeckling

Hybrid attention structure preserving network for reconstruction of under-sampled OCT images

From Patch to Pixel: A Transformer-Based Hierarchical Framework for Compressive Image Sensing.

OCT image denoising algorithm based on discrete wavelet transform and spatial domain feature fusion

Texture Preservation and Speckle Reduction in Poor Optical Coherence Tomography Using the Convolutional Neural Network

Activated Sparsely Sub-Pixel Transformer for Remote Sensing Image Super-Resolution

Passformer: Heterogeneous Feature-Aware CNN-transformer Coupling Network for Sparse Sampling Photoacoustic Tomography

A hand-held optical coherence tomography angiography scanner based on angiography reconstruction transformer networks

Semi-Supervised Speckle Noise Reduction in OCT Images With UNet and Swin-Uformer

O-PRESS: Boosting OCT axial resolution with Prior guidance, Recurrence, and Equivariant Self-Supervision

An efficient parallel fusion structure of distilled and transformer-enhanced modules for lightweight image super-resolution

A Joint Architecture of Mixed-Attention Transformer and Octave Module for Hyperspectral Image Denoising

Simultaneous Denoising and Super-Resolution of Optical Coherence Tomography Images Based on Generative Adversarial Network

Unsupervised despeckling of optical coherence tomography images by combining cross-scale CNN with an intra-patch and inter-patch based transformer.

SPECAT: SPatial-spEctral Cumulative-Attention Transformer for High-Resolution Hyperspectral Image Reconstruction

VET: Vasculature Extraction Transformer for Single-Scan Optical Coherence Tomography Angiography

Dilated-residual U-Net for Optical Coherence Tomography Noise Reduction and Resolution Improvement

SCTANet: A Spatial Attention-Guided CNN-Transformer Aggregation Network for Deep Face Image Super-Resolution