An infrared and visible image fusion network based on multi‐scale feature cascades and non‐local attention
Jing Xu,Zhenjin Liu,Ming Fang
DOI: https://doi.org/10.1049/ipr2.13088
IF: 2.3
2024-03-29
IET Image Processing
Abstract:This paper introduces a novel infrared and visible image fusion network to address the limitations of auto‐encoder fusion networks. In the designed network, the encoder employs a multi‐branch cascade structure with convolution kernels of different sizes to extract multi‐scale features, and the fusion layer incorporates a non‐local attention module alongside a spatial feature fusion strategy for both global and local feature fusion. Comparative experiments on the TNO and MSRS datasets demonstrate that the proposed method outperforms other state‐of‐the‐art fusion approaches. In recent years, research on infrared and visible image fusion has mainly focused on deep learning‐based approaches, particularly deep neural networks with auto‐encoder architectures. However, these approaches suffer from problems such as insufficient feature extraction capability and inefficient fusion strategies. Therefore, this paper introduces a novel image fusion network to address the limitations of infrared and visible image fusion networks with auto‐encoder architectures. In the designed network, the encoder employs a multi‐branch cascade structure, and these convolution branches with different kernel sizes provide the encoder with an adaptive receptive field to extract multi‐scale features. In addition, the fusion layer incorporates a non‐local attention module that is inspired by the self‐attention mechanism. With its global receptive field, this module is used to build a non‐local attention fusion network, which works together with the l1 ‐norm spatial fusion strategy to extract, split, filter, and fuse global and local features. Comparative experiments on the TNO and MSRS datasets demonstrate that the proposed method outperforms other state‐of‐the‐art fusion approaches.
computer science, artificial intelligence,engineering, electrical & electronic,imaging science & photographic technology