Abstract:Abstract For the past few years, image fusion technology has made great progress, especially in infrared and visible light image infusion. However, the fusion methods, based on traditional or deep learning technology, have some disadvantages such as unobvious structure or texture detail loss. In this regard, a novel generative adversarial network named MSAt-GAN is proposed in this paper. It is based on multi-scale feature transfer and deep attention mechanism feature fusion, and used for infrared and visible image fusion. First, this paper employs three different receptive fields to extract the multi-scale and multi-level deep features of multi-modality images in three channels rather than artificially setting a single receptive field. In this way, the important features of the source image can be better obtained from different receptive fields and angles, and the extracted feature representation is also more flexible and diverse. Second, a multi-scale deep attention fusion mechanism is designed in this essay. It describes the important representation of multi-level receptive field extraction features through both spatial and channel attention and merges them according to the level of attention. Doing so can lay more emphasis on the attention feature map and extract significant features of multi-modality images, which eliminates noise to some extent. Third, the concatenate operation of the multi-level deep features in the encoder and the deep features in the decoder are cascaded to enhance the feature transmission while making better use of the previous features. Finally, this paper adopts a dual-discriminator generative adversarial network on the network structure, which can force the generated image to retain the intensity of the infrared image and the texture detail information of the visible image at the same time. Substantial qualitative and quantitative experimental analysis of infrared and visible image pairs on three public datasets show that compared with state-of-the-art fusion methods, the proposed MSAt-GAN network has comparable outstanding fusion performance in subjective perception and objective quantitative measurement.

Multi-Focus Image Fusion Based on Multi-Scale Generative Adversarial Network

StackMFF: End-to-end Multi-Focus Image Stack Fusion Network

Multi-Scale Cross-Attention Fusion Network Based on Image Super-Resolution

The Fusion of Multi-Focus Images Based on the Complex Shearlet Features-Motivated Generative Adversarial Network

MFF-GAN: An unsupervised generative adversarial network with adaptive and gradient joint constraints for multi-focus image fusion

Multi-scale Convolutional Neural Network for Multi-Focus Image Fusion.

MSAt-GAN: a generative adversarial network based on multi-scale and deep attention mechanism for infrared and visible light image fusion

MFIF-GAN: A New Generative Adversarial Network for Multi-Focus Image Fusion

Multi-focused image fusion algorithm based on multi-scale hybrid attention residual network

DDF-GAN: A Generative Adversarial Network with Dual-Discriminator for Multi-Focus Image Fusion

Multi-focus Image Fusion Using Fully Convolutional Two-stream Network for Visual Sensors.

FuseGAN: Learning to fuse Multi-focus Image via Conditional Generative Adversarial Network

Fusion-UDCGAN: Multifocus Image Fusion via a U-Type Densely Connected Generation Adversarial Network

Multi-Focus Image Fusion via PAPCNN and Fractal Dimension in NSST Domain

Multi-Focus Image Fusion Method Based on Multi-Scale Decomposition of Information Complementary

MEF-GAN: Multi-Exposure Image Fusion via Generative Adversarial Networks

Multi-Modal Image Fusion Via Deep Laplacian Pyramid Hybrid Network

Multi-Focus Image Fusion Using U-Shaped Networks with a Hybrid Objective

Siamese conditional generative adversarial network for multi-focus image fusion

Multi-Focus Image Fusion Based on Multi-Scale Gradients and Image Matting