Abstract:With high sensitivity to capture rich details, visible imaging equipment can take images containing more textures and contours which are important to visual perception. Unlike visible cameras, infrared imaging devices can detect targets invisible in visible images, because the imaging principle of infrared sensors derives from differences of thermal radiation. Thus, the purpose of image fusion is to merge as much meaningful feature information from the infrared and visible images into the fused image as possible, such as contours as well as textures of the visible image and thermal targets of the infrared image. In this paper, we propose an image fusion network based on variational auto-encoder (VAE), which performs the image fusion process in deep hidden layers. We divide the proposed network into image fusion network and infrared feature compensation network. Firstly, in the image fusion network, the encoder of the image fusion network is created to generate the latent vectors in hidden layers from the input visible image and infrared image. Secondly, two different latent vectors merge into one based on the product of Gaussian probability density; accordingly, the decoder begins to reconstruct the fused image with the descent of the loss function value. Meanwhile, Residual block and symmetric skip connection methods are added to the network to enhance the efficiency of network training. Finally, due to the defect of the loss function setting in the fusion network, an infrared feature compensation network is designed to compensate critical radiation features of the infrared image. Experimental results on public available datasets demonstrate that the proposed method is superior to other traditional and deep learning methods in both objective metrics and subjective visual perception.

Infrared and visible image fusion based on weighted variance guided filter and image contrast enhancement

Fusion of infrared and visual images through multiscale hybrid unidirectional total variation

Fusion of Visible and Infrared Images Using Saliency Analysis and Detail Preserving Based Image Decomposition

Infrared and visible image fusion method based on saliency detection and target-enhancement

Fusion of Low-Quality Visible and Infrared Images Based on Multi-Level Latent Low-Rank Representation Joint with Retinex Enhancement and Multi-Visual Weight Information

Fusion Of Infrared And Visible Light Images Based On Nonsubsampled Shearlet Transform

Infrared and Visible Image Fusion Via Variational Bayesian Approximation Method

Detail preserved fusion of visible and infrared images using regional saliency extraction and multi-scale image decomposition

Infrared and visible image fusion based on edge-preserving filter and weighted least square optimization

Infrared and Visual Image Fusion Based on a Local-Extrema-Driven Image Filter

Infrared and Visible Image Fusion Based on Filtering Enhancement

Infrared and Visible Image Fusion with Hybrid Image Filtering

An infrared and visible image fusion method based on deep learning

Infrared and Visible Image Fusion Based on Feature Energy

Infrared–Visible Image Fusion through Feature-Based Decomposition and Domain Normalization

Infrared and visible image fusion based on edge-preserving guided filter and infrared feature decomposition

Fusion for visible and infrared images using visual weight analysis and bilateral filter-based multi scale decomposition

Two-scale Image Fusion of Visible and Infrared Images Using Guided Filter

Infrared and Visible Image Fusion Algorithm Based on Double-Domain Transform Filter and Contrast Transform Feature Extraction

Infrared and Visible Image Fusion Using Threshold Segmentation and Weight Optimization

Infrared and Visible Image Fusion Based on Variational Auto-Encoder and Infrared Feature Compensation