Abstract:Color plays an important role in human visual perception, reflecting the spectrum of objects. However, the existing infrared and visible image fusion methods rarely explore how to handle multi-spectral/channel data directly and achieve high color fidelity. This paper addresses the above issue by proposing a novel method with diffusion models, termed as Dif-Fusion, to generate the distribution of the multi-channel input data, which increases the ability of multi-source information aggregation and the fidelity of colors. In specific, instead of converting multi-channel images into single-channel data in existing fusion methods, we create the multi-channel data distribution with a denoising network in a latent space with forward and reverse diffusion process. Then, we use the the denoising network to extract the multi-channel diffusion features with both visible and infrared information. Finally, we feed the multi-channel diffusion features to the multi-channel fusion module to directly generate the three-channel fused image. To retain the texture and intensity information, we propose multi-channel gradient loss and intensity loss. Along with the current evaluation metrics for measuring texture and intensity fidelity, we introduce Delta E as a new evaluation metric to quantify color fidelity. Extensive experiments indicate that our method is more effective than other state-of-the-art image fusion methods, especially in color fidelity. The source code is available at https://github.com/GeoVectorMatrix/Dif-Fusion.

Color Fusion Algorithm for Visible and Infrared Images Based on Color Transfer in YUV Color Space

Region-aware RGB and Near-Infrared Image Fusion

Infrared and visible image fusion method based on saliency detection and target-enhancement

Fusion of infrared and visual images through multiscale hybrid unidirectional total variation

Fusion Of Infrared And Visible Light Images Based On Nonsubsampled Shearlet Transform

One Color Contrast Enhanced Infrared and Visible Image Fusion Method

Infrared and Visible Image Fusion Via Variational Bayesian Approximation Method

Fusion of Visible and Infrared Images Using Saliency Analysis and Detail Preserving Based Image Decomposition

COLOR CONTRAST ENHANCEMENT METHOD TO IMPROVE TARGET DETECTABILITY IN NIGHT VISION FUSION

A robust infrared and visible image fusion framework via multi-receptive-field attention and color visual perception

Infrared and Visible Image Fusion Based on NSCT and Fuzzy Logic

Infrared and Visible Image Fusion Algorithm Based on Double-Domain Transform Filter and Contrast Transform Feature Extraction

Fusion of Low-Quality Visible and Infrared Images Based on Multi-Level Latent Low-Rank Representation Joint with Retinex Enhancement and Multi-Visual Weight Information

Infrared and Visible Image Fusion Using Threshold Segmentation and Weight Optimization

Adaptive low light visual enhancement and high-significant target detection for infrared and visible image fusion

Airborne Infrared and Visible Image Fusion Combined with Region Segmentation

EV-Fusion: A Novel Infrared and Low-Light Color Visible Image Fusion Network Integrating Unsupervised Visible Image Enhancement

Infrared and visible image fusion method based on visual saliency objects and fuzzy region attributes

Infrared and Visual Image Fusion Through Infrared Feature Extraction and Visual Information Preservation

Infrared and Visible Image Fusion with Hybrid Image Filtering

Dif-Fusion: Toward High Color Fidelity in Infrared and Visible Image Fusion With Diffusion Models