Abstract:The intention of infrared and visible image fusion is to combine the images captured by different modal sensors in the same scene to enhance its understanding. Deep learning has been proven its powerful application in image fusion due to its fine generalization, robustness, and representability of deep features. However, the performance of these deep learning-based methods heavily depends on the illumination condition. Especially in dark or exposed scenes, the fused results are over-smoothness and low-contrast, resulting in inaccuracy of object detection. To address these issues, this paper develops a multi-stage feature learning approach with channel-spatial attention mechanism, namely MSCS, for infrared and visible image fusion. The MSCS is composed of the following four key procedures: Firstly, the infrared and visible images are decomposed into illumination and reflectance components by a proposed network called as Retinex_Net. Then, the components are transported to an encoder for features coding. Next, we propose an adaptive fusion module with attention mechanisms to fuse the features. Finally, the fused image is generated by the decoder for decoding the fused features. Meanwhile, a novel fusion loss function and a multi-stage training strategy are proposed to train the above modules. The subjective and objective results of experiments on TNO, LLVIP and MSRS datasets illustrate that the proposed method is effective and performs better than the state-of-the-art fusion methods on achieving enjoyable results in dark or over-exposure scenes. And the results of further experiments on the fused images for object detection demonstrate that the fusion outputs produced by our MSCS are more beneficial for detection tasks.

Multi-Source Fusion User Authentication Based on Channel and Image Features

Source Cell-phone Identification Based on Multi-feature Fusion.

Fusion of Infrared and Visible Images Via Multi-Layer Convolutional Sparse Representation

Multi-Sensor Image Fusion Using Optimized Support Vector Machine and Multiscale Weighted Principal Component Analysis

A Semantic-Aware and Multi-Guided Network for Infrared-Visible Image Fusion

MFST: Multi-Modal Feature Self-Adaptive Transformer for Infrared and Visible Image Fusion

Multi-focus Image Fusion Using Fully Convolutional Two-stream Network for Visual Sensors.

Subspace-based Multi-View Fusion for Instance-Level Image Retrieval

An improved Multi-channels Information Fusion Model with multi-scale signals for fault diagnosis

Target recognition with fusion of visible and infrared images based on mutual learning

Multi-source Remote Sensing Image Fusion Based on Support Vector Machine

Multi-scale unsupervised network for infrared and visible image fusion based on joint attention mechanism

DCFusion: Dual-Headed Fusion Strategy and Contextual Information Awareness for Infrared and Visible Remote Sensing Image

Unsupervised Image Fusion Method based on Feature Mutual Mapping

Source Camera Identification with Multi-Scale Feature Fusion Network

MSCS: Multi-stage Feature Learning with Channel-Spatial Attention Mechanism for Infrared and Visible Image Fusion

Multi-Focus Image Fusion Using U-Shaped Networks with a Hybrid Objective

Multi-Modal Image Fusion Via Sparse Representation and Multi-Scale Anisotropic Guided Measure

SADFusion: A multi-scale infrared and visible image fusion method based on salient-aware and domain-specific

A Multi-Stage Visible and Infrared Image Fusion Network Based on Attention Mechanism

Multi-Stream Multi-Class Fusion of Deep Networks for Video Classification.