Abstract:As a result of several successful applications in computer vision and image processing, sparse representation (SR) has attracted significant attention in multi-sensor image fusion. Unlike the traditional multiscale transforms (MSTs) that presume the basis functions, SR learns an over-complete dictionary from a set of training images for image fusion, and it achieves more stable and meaningful representations of the source images. By doing so, the SR-based fusion methods generally outperform the traditional MST-based image fusion methods in both subjective and objective tests. In addition, they are less susceptible to mis-registration among the source images, thus facilitating the practical applications. This survey paper proposes a systematic review of the SR-based multi-sensor image fusion literature, highlighting the pros and cons of each category of approaches. Specifically, we start by performing a theoretical investigation of the entire system from three key algorithmic aspects, (1) sparse representation models; (2) dictionary learning methods; and (3) activity levels and fusion rules. Subsequently, we show how the existing works address these scientific problems and design the appropriate fusion rules for each application, such as multi-focus image fusion and multi-modality (e.g., infrared and visible) image fusion. At last, we carry out some experiments to evaluate the impact of these three algorithmic components on the fusion performance when dealing with different applications. This article is expected to serve as a tutorial and source of reference for researchers preparing to enter the field or who desire to employ the sparse representation theory in other fields.

A Novel Geometric Dictionary Construction Approach for Sparse Representation Based Image Fusion

Image Fusion Algorithms Based on Sparse Representation of Classified Image Patches

Joint sparse model with coupled dictionary for medical image fusion

Remote Sensing Image Fusion via Sparse Representations Over Learned Dictionaries

A Novel Fusion Strategy and Convolutional Sparse Coding for Robot Multisource Image Fusion

An Image Fusion Method Based on Sparse Representation and Sum Modified-Laplacian in NSCT Domain

Laplacian Sparse Coding Dictionary for Image Set Based Collaborative Representation

Deep Convolutional Sparse Coding Networks for Interpretable Image Fusion.

Multi-focus Image Fusion using dictionary learning and Low-Rank Representation

Multi-focus image fusion based on non-negative sparse representation and patch-level consistency rectification

Multi-Focus Image Fusion Using Sparse Representation and Coupled Dictionary Learning

Joint medical image fusion, denoising and enhancement via discriminative low-rank sparse dictionaries learning

Image Decomposition Fusion Method Based on Sparse Representation and Neural Network

Sparse Representation based Multi-sensor Image Fusion: A Review

Multi-Modal Image Fusion Via Sparse Representation and Multi-Scale Anisotropic Guided Measure

Fusion of Infrared and Visible Images Via Multi-Layer Convolutional Sparse Representation

Multimodal image fusion with adaptive joint sparsity model

Joint Sparse Representations and Coupled Dictionary Learning in Multisource Heterogeneous Image Pseudo-Color Fusion

An Integrated Dictionary-Learning Entropy-Based Medical Image Fusion Framework

Image Fusion with Convolutional Sparse Representation

Hyperspectral and Multispectral Image Fusion based on a Sparse Representation