IAIFNet: An Illumination-Aware Infrared and Visible Image Fusion Network

Qiao Yang,Yu Zhang,Jian Zhang,Zijing Zhao,Shunli Zhang,Jinqiao Wang,Junzhe Chen

2024-05-21

Abstract:Infrared and visible image fusion (IVIF) is used to generate fusion images with comprehensive features of both images, which is beneficial for downstream vision tasks. However, current methods rarely consider the illumination condition in low-light environments, and the targets in the fused images are often not prominent. To address the above issues, we propose an Illumination-Aware Infrared and Visible Image Fusion Network, named as IAIFNet. In our framework, an illumination enhancement network first estimates the incident illumination maps of input images. Afterwards, with the help of proposed adaptive differential fusion module (ADFM) and salient target aware module (STAM), an image fusion network effectively integrates the salient features of the illumination-enhanced infrared and visible images into a fusion image of high visual quality. Extensive experimental results verify that our method outperforms five state-of-the-art methods of fusing infrared and visible images.

Computer Vision and Pattern Recognition

What problem does this paper attempt to address?

This paper aims to address the issue of infrared and visible image fusion (IVIF) in low-light environments. Specifically, existing methods often overlook the illumination conditions under low-light scenarios, resulting in fused images where the targets are not prominent. To solve these problems, the authors propose a new framework—Illumination-Aware Infrared and Visible Image Fusion Network (IAIFNet). This framework first estimates the incident illumination map of the input images through an illumination enhancement network and enhances the texture details of the input images under low-light conditions based on this map. Subsequently, the image fusion network merges the salient features of the enhanced infrared and visible images to generate a fused image with higher visual quality.

Moreover, to further improve the fusion effect, the framework introduces two new modules: the Adaptive Differential Fusion Module (ADFM) and the Salient Target-Aware Module (STAM), which are used to enhance brightness sensitivity and contrast, respectively. Experimental results show that this method outperforms seven state-of-the-art infrared and visible image fusion methods on the public LLVIP dataset, and its lightweight design makes the image fusion process very efficient. The results of downstream multi-object detection tasks also demonstrate that this method significantly improves target detection performance in low-light environments.