S2F-Net: Shared-Specific Fusion Network for Infrared and Visible Image Fusion

Yijing Zhao,Yuchao Xia,Yi Ding,Yumeng Liu,Shuai Liu,Hongan Wang

DOI: https://doi.org/10.1145/3652583.3658100

2024-01-01

Abstract:A modality gap exists between infrared and visible images, presenting challenges for image fusion. Despite the modality heterogeneity, both types of images inherently capture the same scene, suggesting the presence of common information. Effectively extracting shared features while distinguishing modality-specific ones is pivotal for bridging this gap and achieving superior fusion outcomes. To address this, we propose the S hared-S pecific F usion Net work (S2F-Net). The S2F-Net introduces a three-branch feature extractor, which retains two branches for extracting features from each modality, innovatively creating an additional branch dedicated to facilitating the separation of shared features from modality-specific ones. This facilitates guiding the fusion of cross-modal information to generate efficient fusion features, ensuring the effective integration of complementary information from different modalities. To achieve feature fusion and image reconstruction, we propose two fusion modules: the Cross-modality Attention-Guided Fusion Module (CAGFM) and the Multi-Level Fusion Module (MLFM). The former utilizes shared and specific features by employing cross-modality channel attention, enabling effective integration of information across modalities. The latter facilitates feature interaction across different levels. Additionally, to effectively disentangle shared and specific features, we introduce the shared-specific learning module. Extensive experiments conducted on open-source datasets validate the superior performance of our proposed method.