Correlation-Guided Discriminative Cross-Modality Features Network for Infrared and Visible Image Fusion
Zhao Cai,Yong Ma,Jun Huang,Xiaoguang Mei,Fan Fan
DOI: https://doi.org/10.1109/tim.2023.3341137
IF: 5.6
2024-01-01
IEEE Transactions on Instrumentation and Measurement
Abstract:Infrared and visible image fusion, which focuses on generating a new synthetic image, seeks to integrate the complementary properties of both modalities to contain abundant information. Many existing image fusion methods adopt artificially designed fusion rules. However, they often lack sufficient consideration for the interaction of long-range context information. As a result, important information may be lost, which limits the application of subsequent vision tasks. To address these limitations, we propose a novel unsupervised image fusion network named DCSFuse, which is guided by the correlation of image features. Our method can adaptively integrate complementary and long-range context information. More specifically, the proposed method first learns the modal-specific features of the two modalities. Then it calculates the correlation between each feature and integrates the cross-modal features guided by the correlation. Finally, these integrated features are reconstructed into a fused image that contains supplementary information from both modalities and captures long-range context dependencies. Extensive experiments on three mainstream datasets demonstrate the superiority of our method over other methods in terms of quantitative metrics and qualitative effects. Furthermore, we demonstrate experimentally that our fused images enhance the accuracy of visual object detection tasks. In particular, our network benefits from its superior computational efficiency, enabling it to generate fused images in real time. The source code of DCSFuse has been released at: https://github.com/zc617/DCSFuse.
engineering, electrical & electronic,instruments & instrumentation