Abstract:Fusion of images acquired using different sensors generates a single output with enhanced information for high-level visual perception applications. The transformer architecture has demonstrated its powerful ability to obtain important global contextual dependencies for multi-modal image fusion tasks. However, transformer-based image fusion methods face many critical issues, such as incurring huge computational burdens, limited ability to learn local features, and the difficulty of handling images of arbitrary sizes. To address the above limits, we proposed a novel Laplacian Pyramid Hybrid (LapH) network to combine the advantages of CNN and transformer architectures for multi-modal image fusion tasks. With the divide-and-conquer philosophy, we first build a light-weight CNN-based branch, performing effective extraction and fusion of texture/edge features via central difference convolutions, to process the high-resolution components with abundant details encoded in the lower pyramid levels of the Laplacian pyramid. Then, we design a transformer-based branch to process the low-resolution base components, learning long-range dependencies of global-contextual features without incurring extensive computational loads. Here, we design a multi-scale recurrent modulation mechanism to integrate the edge/texture features from the CNN branch as guidance to progressively refine the feature extraction and fusion on low-frequency components. Finally, we propose a new multi-scale spatial consistency loss term based on the neighbor contrast in source images, generating fused images with more natural and realistic appearances. Extensive experiments on two different multi-modal image fusion tasks verify the superiority of our method. The source codes are made publicly available at https://github.com/rgttadv/LapH .

An Image Fusion Algorithm Based on Local Laplasse Filter

Image Fusion Using Multiscale Edge-Preserving Decomposition Based on Weighted Least Squares Filter

Image Fusion Method Considering Region&texture Feature

A Multi-Scale Image Fusion Scheme with Texture and Edge Information

Image Fusion Algorithm at Pixel Level Based on Edge Detection

Detail enhanced multi-source fusion using visual weight map extraction based on multi scale edge preserving decomposition

Multi-Modal Image Fusion Via Deep Laplacian Pyramid Hybrid Network

An Edge-Preserving Image Fusion Algorithm Based on Wavelet Transform

Image edge detection based on multi-scale fusion technology

Image Fusion Using LEP Filtering and Bilinear Interpolation

Multi-focus Image Fusion Based on L1 Image Transform

Multi-modal remote sensing image fusion method guided by local extremum maps-guided image filter

Multi-modal Image Fusion with the Hybrid ℓ0ℓ1 Layer Decomposing and Multi-Directional Filter Banks

Image Fusion Algorithm Based on Statistical Signal Processing and Fuzzy Theory

A NOVEL IMAGE FUSION ALGORITHM USING MULTISCALE EDGES INFORMATION

Support value based fusing images with different focuses

Image Fusion Method Based on Structure-Based Saliency Map and FDST-PCNN Framework.

Multi-scale fusion algorithm of intensity and polarization-difference images based on edge information enhancement

Infrared and visible image fusion based on edge-preserving filter and weighted least square optimization

An Image Fusion Algorithm Based on Polyharmonic Local Sine Transform (Phlst)

A New Method of Multi-Focus Image Fusion Using Laplacian Operator and Region Optimization