Abstract:Infrared and visible image fusion aims at generating a fused image containing the intensity and detail information of source images, and the key issue is effectively measuring and integrating the complementary information of multi-modality images from the same scene. Existing methods mostly adopt a simple weight in the loss function to decide the information retention of each modality rather than adaptively measuring complementary information for different image pairs. In this study, we propose a multi-scale dual attention (MDA) framework for infrared and visible image fusion, which is designed to measure and integrate complementary information in both structure and loss function at the image and patch level. In our method, the residual downsample block decomposes source images into three scales first. Then, dual attention fusion block integrates complementary information and generates a spatial and channel attention map at each scale for feature fusion. Finally, the output image is reconstructed by the residual reconstruction block. Loss function consists of image-level, feature-level and patch-level three parts, of which the calculation of the image-level and patch-level two parts are based on the weights generated by the complementary information measurement. Indeed, to constrain the pixel intensity distribution between the output and infrared image, a style loss is added. Our fusion results perform robust and informative across different scenarios. Qualitative and quantitative results on two datasets illustrate that our method is able to preserve both thermal radiation and detailed information from two modalities and achieve comparable results compared with the other state-of-the-art methods. Ablation experiments show the effectiveness of our information integration architecture and adaptively measure complementary information retention in the loss function.

Real-time Implementation of Visible and Infrared Image Fusion and New Measure Based on Spectral Information

Fusion of infrared and visual images through multiscale hybrid unidirectional total variation

Infrared and Visible Image Fusion Via Variational Bayesian Approximation Method

Infrared and visible image fusion method based on saliency detection and target-enhancement

Fusion Of Infrared And Visible Light Images Based On Nonsubsampled Shearlet Transform

The Realistic Fusion of Multi-spectral Images

Infrared and Visible Image Fusion with Hybrid Image Filtering

Infrared Target Detection And Location For Visual Surveillance Using Fusion Scheme Of Visible And Infrared Images

A Visible/Infrared Gray Image Fusion Algorithm Based on the Yuv Color Transformation

Real time fusion system of multispectral imaging

Low-light Infrared and Visible Image Fusion with Imbalanced Thermal Radiation Distribution

A method for real-time implementation of image fusion

Adaptive low light visual enhancement and high-significant target detection for infrared and visible image fusion

Infrared and Visible Image Fusion Using Threshold Segmentation and Weight Optimization

Infrared and Visible Image Fusion Technology and Application: A Review

Infrared and Visual Image Fusion Based on a Local-Extrema-Driven Image Filter

A Multi-scale Information Integration Framework for Infrared and Visible Image Fusion

Infrared and visible image fusion algorithm based on spatial domain and image features

An image fusion algorithm based on image clustering theory

Infrared and Visible Image Fusion Method Based on Hierarchical Attention Mechanism

Infrared and Visual Image Fusion Via Iterative Quadtree Decomposition and Bézier Interpolation