DCFusion: Difference correlation-driven fusion mechanism of infrared and visible images

Min Li,Enguang Zuo,Feng Li,Cheng Chen,Chaoxun Guo,Pei Liu,Yunling Wang,Xiaoyi Lv,Chen Chen

DOI: https://doi.org/10.1016/j.patcog.2024.111002

IF: 8

2024-09-19

Pattern Recognition

Abstract:In end-to-end image fusion models, the loss function significantly impacts performance. However, most loss functions treat salient and background regions in source images equally, failing to distinguish complementary areas in multimodal images. This limits the model's ability to effectively integrate information from these regions. Therefore, we propose difference correlation-driven fusion mechanism of infrared and visible images, which called DCFusion. Specifically, the model utilizes a dual-branch interactive network that dynamically fuses cross-modal multi-scale complementary information through element-wise multiplication, effectively integrating region-specific information. We introduce a two-stage method for generating salient target masks that adaptively focus on high-contrast regions in infrared images by analyzing pixel contrasts in local areas. Furthermore, we utilize the salient target masks to create heterogeneous images and design the LSCD loss function to minimize the information gap between the heterogeneous images and the fused image, thereby enhancing the model's interpretability. Experiments on the RoadScene and TNO datasets show that DCFusion surpasses with existing representativity fusion approaches, achieving state-of-the-art performance in both subjective visual and objective evaluations. Our code will be publicly available at https://github.com/MinLila/DCFusion .

computer science, artificial intelligence,engineering, electrical & electronic