Abstract:Abstract For the past few years, image fusion technology has made great progress, especially in infrared and visible light image infusion. However, the fusion methods, based on traditional or deep learning technology, have some disadvantages such as unobvious structure or texture detail loss. In this regard, a novel generative adversarial network named MSAt-GAN is proposed in this paper. It is based on multi-scale feature transfer and deep attention mechanism feature fusion, and used for infrared and visible image fusion. First, this paper employs three different receptive fields to extract the multi-scale and multi-level deep features of multi-modality images in three channels rather than artificially setting a single receptive field. In this way, the important features of the source image can be better obtained from different receptive fields and angles, and the extracted feature representation is also more flexible and diverse. Second, a multi-scale deep attention fusion mechanism is designed in this essay. It describes the important representation of multi-level receptive field extraction features through both spatial and channel attention and merges them according to the level of attention. Doing so can lay more emphasis on the attention feature map and extract significant features of multi-modality images, which eliminates noise to some extent. Third, the concatenate operation of the multi-level deep features in the encoder and the deep features in the decoder are cascaded to enhance the feature transmission while making better use of the previous features. Finally, this paper adopts a dual-discriminator generative adversarial network on the network structure, which can force the generated image to retain the intensity of the infrared image and the texture detail information of the visible image at the same time. Substantial qualitative and quantitative experimental analysis of infrared and visible image pairs on three public datasets show that compared with state-of-the-art fusion methods, the proposed MSAt-GAN network has comparable outstanding fusion performance in subjective perception and objective quantitative measurement.

MDRDA-GAN: A Multi-discriminator Generative Adversarial Network Based on Residual Dense Attention for Remote Sensing Image Fusion

Multi-Scale Cross-Attention Fusion Network Based on Image Super-Resolution

MFFAGAN: Generative Adversarial Network With Multilevel Feature Fusion Attention Mechanism for Remote Sensing Image Super-Resolution

Multiattention Generative Adversarial Network for Remote Sensing Image Super-Resolution

DDF-GAN: A Generative Adversarial Network with Dual-Discriminator for Multi-Focus Image Fusion

Single Remote Sensing Image Super-Resolution Via a Generative Adversarial Network with Stratified Dense Sampling and Chain Training

MLFF-GAN: A Multilevel Feature Fusion With GAN for Spatiotemporal Remote Sensing Images

Mun-GAN: A Multiscale Unsupervised Network for Remote Sensing Image Pansharpening

DDcGAN: A Dual-Discriminator Conditional Generative Adversarial Network for Multi-Resolution Image Fusion

MJ-GAN: Generative Adversarial Network with Multi-Grained Feature Extraction and Joint Attention Fusion for Infrared and Visible Image Fusion

Triple disentangled network with dual attention for remote sensing image fusion

A Generative Adversarial Network for infrared and visible image fusion using adaptive dense generator and Markovian discriminator

A Unified Generative Adversarial Network With Convolution and Transformer for Remote Sensing Image Fusion

Multispectral Image Fusion Using Super-Resolution Conditional Generative Adversarial Networks

Attention-based dual residual network-based for multi-spectral pan-sharpening

DCDR-GAN: A Densely Connected Disentangled Representation Generative Adversarial Network for Infrared and Visible Image Fusion

MSAt-GAN: a generative adversarial network based on multi-scale and deep attention mechanism for infrared and visible light image fusion

Deep-Learning-Based Spatio-Temporal-Spectral Integrated Fusion of Heterogeneous Remote Sensing Images

Multigrained Attention Network for Infrared and Visible Image Fusion

DADR: Dual Attention Based Dual Regression Network for Remote Sensing Image Pan-Sharpening

Learning a Generative Model for Fusing Infrared and Visible Images Via Conditional Generative Adversarial Network with Dual Discriminators