MDAN: Multilevel dual-branch attention network for infrared and visible image fusion

Jiawei Wang,Min Jiang,Jun Kong

DOI: https://doi.org/10.1016/j.optlaseng.2024.108042

IF: 5.666

2024-01-31

Optics and Lasers in Engineering

Abstract:Infrared and visible image fusion (IVIF) aims to integrate information captured by optical sensors operating in two different modalities, generating a fused image with both salient targets and texture details. Despite significant advancements in IVIF algorithms, the challenge of preserving complete information, especially regarding texture details, still persists. To alleviate this problem, we propose a multilevel dual-branch attention network (MDAN) which comprises an encoder-decoder network and a fusion strategy layer composed of dual-branch fusion block (DBFB). Firstly, the encoder-decoder network is designed to extract multilevel image features and reconstruct the fused images. Secondly, a novel loss function based on singular value decomposition is proposed to constrain the reconstructed images to preserve abundant algebra features which reflect the structure and texture information of the source images. Thirdly, a fusion strategy layer based on spatial-channel attention and feature aggregation block, which consists of DBFB, is proposed to integrate the extracted features. Finally, we evaluate our method through qualitative and quantitative experiments, the results demonstrate that our method exhibits superiority in performance and achieves a remarkable balance between visual perception and objective evaluation metrics when compared to the state-of-the-art (SOTA) methods.

optics