Abstract:Abstract Infrared and visible image fusion aims to generate synthetic images including salient targets and abundant texture details. However, traditional techniques and recent deep learning-based approaches have faced challenges in preserving prominent structures and fine-grained features. In this study, we propose a lightweight infrared and visible image fusion network utilizing multi-scale attention modules and hybrid dilated convolutional blocks to preserve significant structural features and fine-grained textural details. First, we design a hybrid dilated convolutional block with different dilation rates that enable the extraction of prominent structure features by enlarging the receptive field in the fusion network. Compared with other deep learning methods, our method can obtain more high-level semantic information without piling up a large number of convolutional blocks, effectively improving the ability of feature representation. Second, distinct attention modules are designed to integrate into different layers of the network to fully exploit contextual information of the source images, and we leverage the total loss to guide the fusion process to focus on vital regions and compensate for missing information. Extensive qualitative and quantitative experiments demonstrate the superiority of our proposed method over state-of-the-art methods in both visual effects and evaluation metrics. The experimental results on public datasets show that our method can improve the entropy (EN) by 4.80%, standard deviation (SD) by 3.97%, correlation coefficient (CC) by 1.86%, correlations of differences (SCD) by 9.98%, and multi-scale structural similarity (MS_SSIM) by 5.64%, respectively. In addition, experiments with the VIFB dataset further indicate that our approach outperforms other comparable models.

Attention based dual UNET network for infrared and visible image fusion

Hyperspectral and Multispectral Image Fusion Based on Deep Attention Network.

Cross-UNet: dual-branch infrared and visible image fusion framework based on cross-convolution and attention mechanism

Multi-scale unsupervised network for infrared and visible image fusion based on joint attention mechanism

RDCa-Net: Residual Dense Channel Attention Symmetric Network for Infrared and Visible Image Fusion

Attention-based F-UNet for Remote Sensing Image Fusion.

A Dual Cross Attention Transformer Network for Infrared and Visible Image Fusion

IVJDN: an End-to-End Network for Joint Infrared and Visible Image Fusion and Detection

Visible and Infrared Image Fusion Based on Attention and Multiscale Residuals

A Multi-Stage Visible and Infrared Image Fusion Network Based on Attention Mechanism

DDFNet-A: Attention-Based Dual-Branch Feature Decomposition Fusion Network for Infrared and Visible Image Fusion

An infrared and visible image fusion network based on multi‐scale feature cascades and non‐local attention

UFFusion: an Unified Feature Space for Infrared-Visible Image Fusion Network Based on Dynamic Domain Transformation

Weakly Supervised Infrared and Visible Image Fusion Based on Attention Mechanism

Dual Attention Mechanisms with Perceptual Loss Ensemble for Infrared and Visible Image Fusion

Multiscale Channel Attention Network for Infrared and Visible Image Fusion.

Deep Selective Fusion of Visible and Near-Infrared Images Using Unsupervised U-Net.

Dual-Attention-Based Feature Aggregation Network for Infrared and Visible Image Fusion

Multi-scale attention-based lightweight network with dilated convolutions for infrared and visible image fusion

DANT-GAN: A dual attention-based of nested training network for infrared and visible image fusion

ADF‐Net: Attention‐guided deep feature decomposition network for infrared and visible image fusion