Abstract:Aiming to address the issues of missing detailed information, the blurring of significant target information, and poor visual effects in current image fusion algorithms, this paper proposes an infrared and visible-light image fusion algorithm based on discrete wavelet transform and convolutional neural networks. Our backbone network is an autoencoder. A DWT layer is embedded in the encoder to optimize frequency-domain feature extraction and prevent information loss, and a bottleneck residual block and a coordinate attention mechanism are introduced to enhance the ability to capture and characterize the low- and high-frequency feature information; an IDWT layer is embedded in the decoder to achieve the feature reconstruction of the fused frequencies; the fusion strategy adopts the l1−norm fusion strategy to integrate the encoder's output frequency mapping features; a weighted loss containing pixel loss, gradient loss, and structural loss is constructed for optimizing network training. DWT decomposes the image into sub-bands at different scales, including low-frequency sub-bands and high-frequency sub-bands. The low-frequency sub-bands contain the structural information of the image, which corresponds to the important target information, while the high-frequency sub-bands contain the detail information, such as edge and texture information. Through IDWT, the low-frequency sub-bands that contain important target information are synthesized with the high-frequency sub-bands that enhance the details, ensuring that the important target information and texture details are clearly visible in the reconstructed image. The whole process is able to reconstruct the information of different frequency sub-bands back into the image non-destructively, so that the fused image appears natural and harmonious visually. Experimental results on public datasets show that the fusion algorithm performs well according to both subjective and objective evaluation criteria and that the fused image is clearer and contains more scene information, which verifies the effectiveness of the algorithm, and the results of the generalization experiments also show that our network has good generalization ability.

Domain Transform Model Driven by Deep Learning for Anti-Noise Hyperspectral and Multispectral Image Fusion

Hyperspectral and Multispectral Image Fusion Based on Deep Attention Network.

A Self-Supervised Spaceborne Multispectral and Hyperspectral Image Fusion Unrolling Network

Hyperspectral and Multispectral Image Fusion by Deep Neural Network in a Self-Supervised Manner

Unsupervised Hyperspectral and Multispectral Image Blind Fusion Based on Deep Tucker Decomposition Network with Spatial-Spectral Manifold Learning

Hyperspectral and Multispectral Image Fusion Using the Conditional Denoising Diffusion Probabilistic Model

MLR-DBPFN: A Multi-scale Low Rank Deep Back Projection Fusion Network for Anti-noise Hyperspectral and Multispectral Image Fusion

Enhanced Deep Blind Hyperspectral Image Fusion

Edge-Conditioned Feature Transform Network for Hyperspectral and Multispectral Image Fusion

Coupled Matrix Factorization Constrained Deep Hyperspectral and Multispectral Image Fusion

Deep Hyperspectral Image Fusion Network With Iterative Spatio-Spectral Regularization

ADMM-HFNet: A Matrix Decomposition-Based Deep Approach for Hyperspectral Image Fusion

Hyperspectral Image Fusion Based on Dual-Resolution Fusion Feature Mutual Guidance Network

DM-Fusion: Deep Model-Driven Network for Heterogeneous Image Fusion.

Domain adaptation hyperspectral image fusion based on spatial-spectral domain separation

Deep-Learning-Based Spatio-Temporal-Spectral Integrated Fusion of Heterogeneous Remote Sensing Images

Unsupervised Deep Tensor Network for Hyperspectral–Multispectral Image Fusion

Multispectral and Hyperspectral Image Fusion by MS/HS Fusion Net.

HDCCT: Hybrid Densely Connected CNN and Transformer for Infrared and Visible Image Fusion

DCFNet: Infrared and Visible Image Fusion Network Based on Discrete Wavelet Transform and Convolutional Neural Network

Unified Cloud-Based Framework for Hyperspectral and Multispectral Image Fusion Incorporating Nonlocal Principles and Tensor Decomposition