Pctlfusion: A Progressive Fusion Network Via Contextual Texture Learning for Infrared and Visible Image

Yixiang Lu,Daiwei Gong,Dawei Zhao,Davydau Maksim,Qingwei Gao

DOI: https://doi.org/10.2139/ssrn.4824665

2024-01-01

Abstract:The objective of infrared and visible image fusion is to generate a fused image that contains rich texture details and salient targets. However, most of the existing fusion methods tend to focus on preserving texture details in visible image and salient targets in infrared image, ignoring the fact that infrared image have richer texture details and better visual effects than visible image under low-light conditions. In other words, differences in information sources under the condition of illumination imbalance is ignored in setting the task goal. In addition, existing network designs usually ignore multimodal information interactions during the coding phase, which may result in loss of information. In this work, we propose a progressive fusion network via contextual texture learning for infrared and visible image, termed as PCTLFusion. To enhance the limited texture information, we design a content and contextual texture learning module (CCTLM) to model the texture and content of both the source images, respectively. Then, an affine transform-based multimodal pre-fusion module (ATMPM) is designed to learn and pre-fuse the complementary and similar enhanced information, which achieves information interaction during encoding. To learn fully feature information using multilayers, we iterate CCTLM and ATMPM three times in the process of model forming. Finally, the fused image is reconstructed by decoding the output of the last ATMPM. Numerous experiments show that our proposed fusion algorithm is superior to the state-of-the-art existing methods in both subjective and objective evaluations.