An Infrared and Visible Image Fusion Framework Based on Dual Scale Decomposition and Learnable Attention Fusion Strategy

Guanzheng Cheng,Lizuo Jin,Lin Chai

DOI: https://doi.org/10.1109/ccdc58219.2023.10326978

2023-01-01

Abstract:The fusion of infrared and visible images is a hot field in image processing, aiming to preserve the prominent targets in infrared images and the clear background texture in visible images. This paper proposes a novel auto-encoder framework for infrared and visible image fusion based on dual-scale decomposition and a learnable attention fusion strategy. The core idea is that the encoder decomposes the image into low-level multi-scale features, deep-level difference features, and common features. And we use a two-stage training strategy. In the first stage, the auto-encoder network is trained to decompose, extract features, and reconstruct images. In the second stage, the learnable attention-based fusion network is trained using the proposed loss function, which enables the learnable fusion network to learn different appropriate fusion strategies for different levels of feature layers. The results show that our fusion framework has achieved better performance than the state-of-the-art methods in both subjective and objective evaluation. And our proposed method achieves better values on 6 out of 8 common quality metrics.