Abstract:With high sensitivity to capture rich details, visible imaging equipment can take images containing more textures and contours which are important to visual perception. Unlike visible cameras, infrared imaging devices can detect targets invisible in visible images, because the imaging principle of infrared sensors derives from differences of thermal radiation. Thus, the purpose of image fusion is to merge as much meaningful feature information from the infrared and visible images into the fused image as possible, such as contours as well as textures of the visible image and thermal targets of the infrared image. In this paper, we propose an image fusion network based on variational auto-encoder (VAE), which performs the image fusion process in deep hidden layers. We divide the proposed network into image fusion network and infrared feature compensation network. Firstly, in the image fusion network, the encoder of the image fusion network is created to generate the latent vectors in hidden layers from the input visible image and infrared image. Secondly, two different latent vectors merge into one based on the product of Gaussian probability density; accordingly, the decoder begins to reconstruct the fused image with the descent of the loss function value. Meanwhile, Residual block and symmetric skip connection methods are added to the network to enhance the efficiency of network training. Finally, due to the defect of the loss function setting in the fusion network, an infrared feature compensation network is designed to compensate critical radiation features of the infrared image. Experimental results on public available datasets demonstrate that the proposed method is superior to other traditional and deep learning methods in both objective metrics and subjective visual perception.

RAN: Infrared and Visible Image Fusion Network Based on Residual Attention Decomposition

IVJDN: an End-to-End Network for Joint Infrared and Visible Image Fusion and Detection

Fusion of Infrared and Visible Images based on Spatial-Channel Attentional Mechanism

A Fusion Algorithm for Infrared and Visible Images Based on RDU-PCNN and ICA-bases in NSST Domain

IAIFNet: An Illumination-Aware Infrared and Visible Image Fusion Network

Visible and Infrared Image Fusion Based on Attention and Multiscale Residuals

RITFusion: Reinforced Interactive Transformer Network for Infrared and Visible Image Fusion

A Self-Supervised Method for Infrared and Visible Image Fusion

Infrared and Visible Image Fusion Via Decoupling Network

MDAN: Multilevel dual-branch attention network for infrared and visible image fusion

Integrating Parallel Attention Mechanisms and Multi-Scale Features for Infrared and Visible Image Fusion

RDCa-Net: Residual Dense Channel Attention Symmetric Network for Infrared and Visible Image Fusion

Infrared and visible image fusion based on residual dense network and gradient loss

DDFNet-A: Attention-Based Dual-Branch Feature Decomposition Fusion Network for Infrared and Visible Image Fusion

Multi-Modality Image Fusion and Object Detection Based on Semantic Information

Weakly Supervised Infrared and Visible Image Fusion Based on Attention Mechanism

RADFNet: an Infrared and Visible Image Fusion Framework Based on Distributed Network.

Rethinking Cross-Attention for Infrared and Visible Image Fusion

Infrared and Visible Image Fusion Based on Variational Auto-Encoder and Infrared Feature Compensation

A Novel Infrared and Visible Image Fusion Algorithm Based on Global Information-Enhanced Attention Network