Abstract:As a result of several successful applications in computer vision and image processing, sparse representation (SR) has attracted significant attention in multi-sensor image fusion. Unlike the traditional multiscale transforms (MSTs) that presume the basis functions, SR learns an over-complete dictionary from a set of training images for image fusion, and it achieves more stable and meaningful representations of the source images. By doing so, the SR-based fusion methods generally outperform the traditional MST-based image fusion methods in both subjective and objective tests. In addition, they are less susceptible to mis-registration among the source images, thus facilitating the practical applications. This survey paper proposes a systematic review of the SR-based multi-sensor image fusion literature, highlighting the pros and cons of each category of approaches. Specifically, we start by performing a theoretical investigation of the entire system from three key algorithmic aspects, (1) sparse representation models; (2) dictionary learning methods; and (3) activity levels and fusion rules. Subsequently, we show how the existing works address these scientific problems and design the appropriate fusion rules for each application, such as multi-focus image fusion and multi-modality (e.g., infrared and visible) image fusion. At last, we carry out some experiments to evaluate the impact of these three algorithmic components on the fusion performance when dealing with different applications. This article is expected to serve as a tutorial and source of reference for researchers preparing to enter the field or who desire to employ the sparse representation theory in other fields.

Image Fusion Method Based on Spatially Masked Convolutional Sparse Representation

Image Fusion of CT and MR with Sparse Representation in NSST Domain

Image Fusion with Convolutional Sparse Representation

Fusion of Infrared and Visible Images Via Multi-Layer Convolutional Sparse Representation

A multi-source image fusion algorithm based on gradient regularized convolution sparse representation

An Infrared and Visible Image Fusion Method Based on Non-Subsampled Contourlet Transform and Joint Sparse Representation.

Image Fusion with Nonsubsampled Contourlet Transform and Sparse Representation.

Infrared and Visible Image Fusion Via Joint Convolutional Sparse Representation

An Image Fusion Method Based on Sparse Representation and Sum Modified-Laplacian in NSCT Domain

Fusion of Hyperspectral and Multispectral Images by Convolutional Sparse Representation

Remote Sensing Image Fusion Based on Sparse Representation and Guided Filtering

Medical Image Fusion Via Convolutional Sparsity Based Morphological Component Analysis

Sparse Representation based Multi-sensor Image Fusion: A Review

The Multi-Focus-Image-Fusion Method Based on Convolutional Neural Network and Sparse Representation.

CCSR-Net: Unfolding Coupled Convolutional Sparse Representation for Multi-focus Image Fusion.

Image Fusion Method Based on Simultaneous Sparse Representation with Non-Subsampled Contourlet Transform

Using Taylor Expansion and Convolutional Sparse Representation for Image Fusion

Boosting Image Super-Resolution Via Fusion of Complementary Information Captured by Multi-Modal Sensors

Multi-source Remote Sensing Image Fusion Method Based on Sparse Representation

Multi-focus image fusion based on multi-scale sparse representation

Multi-focus image fusion based on non-negative sparse representation and patch-level consistency rectification