Abstract:The second near-infrared (NIR-II) fluorescence imaging has attracted widespread attention from researchers due to its powerful advantage of assisting doctors in real-time tumor removal during surgery. Light has different penetration capabilities in different bands. Although NIR-IIb window (1,500-1,700 nm) imaging is clearer than NIR-IIa window (1,000-1,300 nm), there are no effective molecular probes for NIR-IIb clinical transformation. NIR-II images are rich in variety, including animals, tissues, etc., and the huge differences in structure also bring difficulties to image enhancement. In order to solve the above problems, we propose a universal NIR-II fluorescence image enhancement model based on arbitrary style transfer, and simultaneously achieve the quality enhancement of multiple types of NIR-IIa fluorescence images. The goal of this model design is to completely preserve the important information of the original image to prevent artifacts and meet the high demand for real-time imaging during NIR-II surgery. To this end, we design a fusion network based on the calculation method of square-square root-sign assignment, in which the content information of NIR-IIa is integrated into the style representation of NIR-IIb to increase the maintenance of the original content structure, so that the stylized feature map tends to show the content. In addition, we propose an attention matching loss, which extracts key structural information through the attention mechanism and maintains the structure of the enhanced image consistent with the original image. Extensive experiments demonstrate that our proposed model has a great advantage in maintaining the original content information, with the SSIM of 0.625 and the content loss of 0.279 compared to the original images. In addition, the quality of enhanced images is satisfactory, with the PSNR of 31.20 and the SSIM of 0.501 compared to the real images.

Universal NIR-II fluorescence image enhancement via covariance weighted attention network

Universal NIR-II fluorescence image enhancement via square and square root network

Fusion of Low-Illuminance Visible and Near-Infrared Images Based on Convolutional Neural Networks

Deep Networks with Detail Enhancement for Infrared Image Super-Resolution

Single-image-based Nonuniformity Correction of Uncooled Long-Wave Infrared Detectors: a Deep-Learning Approach.

Multispectral Fusion of RGB and NIR Images Using Weighted Least Squares and Convolution Neural Networks

Infrared and Visible Image Fusion Based on a Two-Stage Class Conditioned Auto-Encoder Network.

Donor Engineering for NIR-II Molecular Fluorophores with Enhanced Fluorescent Performance

Multi-scale attention-based lightweight network with dilated convolutions for infrared and visible image fusion

Attention-Guided NIR Image Colorization via Adaptive Fusion of Semantic and Texture Clues

Fusion of Infrared and Visible Light Images Based on Improved Adaptive Dual-Channel Pulse Coupled Neural Network

Self-Confocal NIR-II Fluorescence Microscopy for Multifunctional in Vivo Imaging

Multimodal fusion via a series of transfers for noise removal

RDCa-Net: Residual Dense Channel Attention Symmetric Network for Infrared and Visible Image Fusion

Deep Learning Fluorescence Imaging of Visible to NIR‐II Based on Modulated Multimode Emissions Lanthanide Nanocrystals

Near-Infrared Image Guided Neural Networks For Color Image Denoising

NIR-II Fluorescence in Vivo Functional Bioimaging

CoCoNet: Coupled Contrastive Learning Network with Multi-level Feature Ensemble for Multi-modality Image Fusion

Luminance-Preserving Visible and Near-Infrared Image Fusion Network with Edge Guidance

Convolutional Neural Network Transformer (CNNT) for Fluorescence Microscopy image Denoising with Improved Generalization and Fast Adaptation

Hierarchical Fusion of Infrared and Visible Images Based on Channel Attention Mechanism and Generative Adversarial Networks