Abstract:Abstract Infrared and visible image fusion aims to generate synthetic images including salient targets and abundant texture details. However, traditional techniques and recent deep learning-based approaches have faced challenges in preserving prominent structures and fine-grained features. In this study, we propose a lightweight infrared and visible image fusion network utilizing multi-scale attention modules and hybrid dilated convolutional blocks to preserve significant structural features and fine-grained textural details. First, we design a hybrid dilated convolutional block with different dilation rates that enable the extraction of prominent structure features by enlarging the receptive field in the fusion network. Compared with other deep learning methods, our method can obtain more high-level semantic information without piling up a large number of convolutional blocks, effectively improving the ability of feature representation. Second, distinct attention modules are designed to integrate into different layers of the network to fully exploit contextual information of the source images, and we leverage the total loss to guide the fusion process to focus on vital regions and compensate for missing information. Extensive qualitative and quantitative experiments demonstrate the superiority of our proposed method over state-of-the-art methods in both visual effects and evaluation metrics. The experimental results on public datasets show that our method can improve the entropy (EN) by 4.80%, standard deviation (SD) by 3.97%, correlation coefficient (CC) by 1.86%, correlations of differences (SCD) by 9.98%, and multi-scale structural similarity (MS_SSIM) by 5.64%, respectively. In addition, experiments with the VIFB dataset further indicate that our approach outperforms other comparable models.

A super-resolution based method to synthesize visual images from near infrared

Deep Networks with Detail Enhancement for Infrared Image Super-Resolution

Fusion Of Infrared And Visible Light Images Based On Nonsubsampled Shearlet Transform

A Fusion Framework For Face Recognition Under Varying Illumination Based On Multi-Scale Analysis

The face image super-resolution algorithm based on combined representation learning

Boosting Image Super-Resolution Via Fusion of Complementary Information Captured by Multi-Modal Sensors

Fusion of Infrared and Visible Images Via Multi-Layer Convolutional Sparse Representation

Fusion of Low-Quality Visible and Infrared Images Based on Multi-Level Latent Low-Rank Representation Joint with Retinex Enhancement and Multi-Visual Weight Information

Cascaded Deep Networks with Multiple Receptive Fields for Infrared Image Super-Resolution

Improved Thermal Infrared Image Super-Resolution Reconstruction Method Base on Multimodal Sensor Fusion

Face recognition using both visible light image and near-infrared image and a deep network

A robust infrared and visible image fusion framework via multi-receptive-field attention and color visual perception

A Novel Infrared and Visible Image Fusion Approach Based on Adversarial Neural Network

Hypergraph-Guided Disentangled Spectrum Transformer Networks for Near-Infrared Facial Expression Recognition

FSADFuse: A Novel Fusion Approach to Infrared and Visible Images

Infrared and visible light dual-camera super-resolution imaging with texture transfer network

Implicit Multi-Spectral Transformer: An Lightweight and Effective Visible to Infrared Image Translation Model

Near-infrared and visible light image fusion algorithm for face recognition

EV-Fusion: A Novel Infrared and Low-Light Color Visible Image Fusion Network Integrating Unsupervised Visible Image Enhancement

Multi-scale attention-based lightweight network with dilated convolutions for infrared and visible image fusion