AFIRE: Adaptive FusionNet for Illumination-Robust Feature Extraction in Heterogeneous Imaging Environments

Mingxin Yu,Xufan Miao,Yichen Sun,Yuchen Bai,Lianqing Zhu

DOI: https://doi.org/10.1016/j.infrared.2024.105557

IF: 2.997

2024-01-01

Infrared Physics & Technology

Abstract:The fusion of infrared and visible images aims to synthesize a fused image that incorporates richer information by leveraging the distinct characteristics of each modality. However, the disparate quality of input images in terms of infrared and visible light significantly impacts fusion performance. To address this issue, we propose a novel deep adaptive fusion method called Adaptive FusionNet for Illumination-Robust Feature Extraction (AFIRE). This method involves the interactive processing of two input features and dynamically adjusts the fusion weights based on varying illumination conditions. Specifically, we introduce a novel interactive extraction structure during the feature extraction stage for both infrared and visible light, enabling the capture of more complementary information. Additionally, we design a Deep Adaptive Fusion module to assess the quality of input features and perform weighted fusion through a channel attention mechanism. Finally, a new loss function is formulated by incorporating the entropy and median of input images to guide the training of the fusion network. Extensive experiments demonstrate that AFIRE outperforms state-of-the-art methods in preserving pixel intensity distribution and texture details. Source code is available at: https://www.github.com/ISCLab-Bistu/AFIRE.