Abstract:Infrared and visible image fusion is an essential task for multi-sensor image fusion. Generative adversarial networks (GAN) have achieved remarkable performance in the fusion of infrared and visible image. Existing GAN based fusion methods merely using infrared and visible image as input for the fusion, while we found that differential images obtained by subtraction between two image sources could provide contrast information for the fusion. To this end, a novel dual fusion path generative adversarial network (DFPGAN) is proposed in this paper for infrared and visible image fusion. We divided the generator of generative adversarial network into two fusion paths namely infrared-visible path and differential path. The input of infrared-visible path concatenated two image sources to make infrared intensity and texture details keep balance fusion in this path. The input of differential path concatenated differential images obtained by subtraction between two image sources to make contrast information fusion in this path. The features extracted by two fusion paths are concatenated at the end of the generator to generate fused images with contrast effect and balanced information distribution. Meanwhile, we have implemented dual self-attention feature refine module (DSAM) on two fusion paths to refine feature maps in two fusion paths. We adopted switchable normalization layer (SN) substitute for batch normalization layer (BN) in the generator and discriminator to avoid fusion artifact. Furthermore, a mixed content loss is integrated in the generator loss functions to guide the generated image keep balanced information distribution and preserving contrast simultaneously. The adversarial training employed dual adversarial architecture to balance the distribution of infrared intensity and texture details. To verifying the improvement effect of fusion image on target detection, we introduce the Scaled-YOLOv4 target detection framework as evaluation framework, and use the proposed network to fuse RGB images and infrared images for target detection. The results of qualitative and quantitative experiments conducted on public datasets demonstrated the superiority of proposed network over other state-of-the-art methods and could generate fused images with distinctly contrast.

FIRe-GAN: a novel deep learning-based infrared-visible fusion method for wildfire imagery

Infrared/Visible Light Fire Image Fusion Method Based on Generative Adversarial Network of Wavelet-Guided Pooling Vision Transformer

FusionGAN: A Generative Adversarial Network for Infrared and Visible Image Fusion.

Shortwave Infrared and Visible Light Image Fusion Method Based on Dual Discriminator GAN

A Study on Flame Detection Method Combining Visible Light and Thermal Infrared Multimodal Images

Fire Image Analysis Based on Spatio-Temporal Fusion Algorithms

GAN-HA: A generative adversarial network with a novel heterogeneous dual-discriminator network and a new attention-based fusion strategy for infrared and visible image fusion

FSADFuse: A Novel Fusion Approach to Infrared and Visible Images

DFPGAN: Dual Fusion Path Generative Adversarial Network for Infrared and Visible Image Fusion

A Generative Adversarial Network for infrared and visible image fusion using adaptive dense generator and Markovian discriminator

Hierarchical Fusion of Infrared and Visible Images Based on Channel Attention Mechanism and Generative Adversarial Networks

FIgLib & SmokeyNet: Dataset and Deep Learning Model for Real-Time Wildland Fire Smoke Detection

Coupled GAN With Relativistic Discriminators for Infrared and Visible Images Fusion

FUELVISION: A Multimodal Data Fusion and Multimodel Ensemble Algorithm for Wildfire Fuels Mapping

A Novel Infrared and Visible Image Fusion Approach Based on Adversarial Neural Network

Adversarial Fusion Network for Forest Fire Smoke Detection

An attention-guided and wavelet-constrained generative adversarial network for infrared and visible image fusion

Forest Fire Monitoring Method Based on UAV Visual and Infrared Image Fusion

Deep Learning-Based Wildfire Image Detection and Classification Systems for Controlling Biomass

A RGB-Thermal Based Adaptive Modality Learning Network for Day–night Wildfire Identification

Fire-Net: A Deep Learning Framework for Active Forest Fire Detection