Abstract:Multi-modality image fusion refers to integrating series of images acquired from different sensors and obtaining a fused image which is expected to provide more comprehensive information. It plays a pivotal role in many computer vision tasks and promotes the performance of subsequent applications. Most existing approaches attempted to design appropriate fusion rules for specific image fusion task, which have a bad generalization to different fusion tasks. Apart from that, the texture details in the fused images are common blurred due to undesirable artifacts introduced from the different modalities. In this paper, we propose a generic multimodal image fusion framework by combing the visual saliency model and flexible bilevel paradigm. Specifically, we decompose input images into an intensity layer, representing large-scale intensity variations, and a detail layer, containing small textures changes. Then we fuse the intensity layer through visual saliency map to improve the contrast of an image under consideration, and design a bilevel paradigm for fusing the detail layer to obtain fine details. Furthermore, to make the fused result visual friendly, a deep prior is built in the bilevel paradigm. Besides, an elastic target-guided hyper-parameter is introduced to dominate the proportion of the textural details from the source images, which can be further adjusted in accordance with different fusion tasks. We conducted the experiments on three available datasets to demonstrate the superiority of our framework against the state-of-the-art methods quantitatively and qualitatively in a variety of fusion tasks, including infrared and visible image, near-infrared and visible image fusion and multimodal medical image fusion.

A Multi-Modal Image Fusion Framework Based on Guided Filter and Sparse Representation

Multi-Frame Image Fusion Method Combining Spatial-Temporal Saliency Detection and Nsct

A Novel Multi-Modality Image Fusion Method Based on Image Decomposition and Sparse Representation

A Unified Image Fusion Framework with Flexible Bilevel Paradigm Integration

Multi-Modal Image Fusion Via Sparse Representation and Multi-Scale Anisotropic Guided Measure

Multi-Focus Image Fusion Based on Cartoon-Texture Image Decomposition

Tri-Modal Medical Image Fusion Based on Cartoon-Texture Decomposition and Sparse Representation

Multi-focus Image Fusion Based on Sparse Decomposition and Background Detection.

Image Fusion Based on Multi-Morphological Component Analysis

Fusion of Infrared and Visible Images Via Multi-Layer Convolutional Sparse Representation

Multi-Modal Image Fusion Via Convolutional Morphological Component Analysis And Guided Filter

Multi-focus Image Fusion Based on Sparse Features

Multi-focus Image Fusion with Joint Guided Image Filtering

Tri-modal Medical Image Fusion Based on Adaptive Energy Choosing Scheme and Sparse Representation

Infrared and Visible Image Fusion with the Use of Multi-Scale Edge-Preserving Decomposition and Guided Image Filter

Multi-scale infrared and visible image fusion framework based on dual partial differential equations

Bridging the Gap between Multi-focus and Multi-modal: A Focused Integration Framework for Multi-modal Image Fusion

Multimodal Image Fusion with Joint Sparsity Model

Image Fusion with Morphological Component Analysis

Multi-scale Guided Image and Video Fusion: A Fast and Efficient Approach

Multi-scale Image Fusion Through Rolling Guidance Filter.