Abstract:Multimodal image fusion (MMIF) can provide more comprehensive scene characteristics by synthesizing a single image from multi-sensor images of the same scene, which works out the limitation of single-type hardware. To handle MMIF tasks, current deep learning (DL)-based methods usually use convolutional neural networks (CNNs) or combine transformer to extract local and global contextual information of source images. However, none of the existing works fully explores contextual information both across modalities and within single modalities, leading to limited fusion results. To this end, we propose a new MMIF method via locally enhanced global learning, termed as LeGFusion. Specifically, the network of LeGFusion is devised based on locally enhanced transformer block (LETB), which can capture long-range dependencies benefiting from nonoverlapping window-based self-attention while capturing useful local context with the utilization of the convolution operator into transformer. On one hand, several LETBs are deployed to extract global contexts from each modality while emphasizing its local information. On the other hand, the fusion module that also consists of LETBs is designed to integrate multimodal features by perceiving cross-modal local and global interactions. Powered by these intramodal and intermodal contextual information exploration, the proposed LeGFusion enjoys a high capability in capturing significant complementary information for image fusion. Extensive experiments are conducted on two types of MMIF tasks, including infrared-visible image fusion (IVF) and medical image fusion. The qualitative and quantitative evaluation results demonstrate the superiority of our LeGFusion over state-of-the-art methods. Furthermore, we validate the generalization ability of LeGFusion without fine-tuning and achieve fantastic results.

LGABL: UHD Multi-Exposure Image Fusion Via Local and Global Aware Bilateral Learning

GALFusion: Multi-Exposure Image Fusion via a Global–Local Aggregation Learning Network

StackMFF: End-to-end Multi-Focus Image Stack Fusion Network

An Ultra-High-definition Multi-Exposure Image Fusion Method Based on Multi-Scale Feature Extraction

Multi-Exposure Image Fusion via Multi-Scale and Context-Aware Feature Learning

Multi Exposure Fusion for High Dynamic Range Imaging Via Multi-Channel Gradient Tensor

Ref-MEF: Reference-Guided Flexible Gated Image Reconstruction Network for Multi-Exposure Image Fusion

Efficient Multi-exposure Image Fusion Via Filter-dominated Fusion and Gradient-driven Unsupervised Learning.

Deep Guided Learning for Fast Multi-Exposure Image Fusion

Multi-Exposure Image Fusion via Deformable Self-Attention

A Dual Domain Multi-exposure Image Fusion Network based on the Spatial-Frequency Integration

Multi-exposure image fusion via deep perceptual enhancement

LeGFusion: Locally-enhanced Global Learning for Multi-Modal Image Fusion

MEFLUT: Unsupervised 1D Lookup Tables for Multi-exposure Image Fusion

Multi-Scale Exposure Fusion Via Content Adaptive Edge-Preserving Smoothing Pyramids.

LeGFusion: Locally Enhanced Global Learning for Multimodal Image Fusion

Attention-guided Global-local Adversarial Learning for Detail-preserving Multi-exposure Image Fusion

Rethinking Multi-Exposure Image Fusion with Extreme and Diverse Exposure Levels: A Robust Framework Based on Fourier Transform and Contrastive Learning.

MEF-GAN: Multi-Exposure Image Fusion via Generative Adversarial Networks