Abstract:As a result of several successful applications in computer vision and image processing, sparse representation (SR) has attracted significant attention in multi-sensor image fusion. Unlike the traditional multiscale transforms (MSTs) that presume the basis functions, SR learns an over-complete dictionary from a set of training images for image fusion, and it achieves more stable and meaningful representations of the source images. By doing so, the SR-based fusion methods generally outperform the traditional MST-based image fusion methods in both subjective and objective tests. In addition, they are less susceptible to mis-registration among the source images, thus facilitating the practical applications. This survey paper proposes a systematic review of the SR-based multi-sensor image fusion literature, highlighting the pros and cons of each category of approaches. Specifically, we start by performing a theoretical investigation of the entire system from three key algorithmic aspects, (1) sparse representation models; (2) dictionary learning methods; and (3) activity levels and fusion rules. Subsequently, we show how the existing works address these scientific problems and design the appropriate fusion rules for each application, such as multi-focus image fusion and multi-modality (e.g., infrared and visible) image fusion. At last, we carry out some experiments to evaluate the impact of these three algorithmic components on the fusion performance when dealing with different applications. This article is expected to serve as a tutorial and source of reference for researchers preparing to enter the field or who desire to employ the sparse representation theory in other fields.

Multi-source Remote Sensing Image Fusion Method Based on Sparse Representation

Remote Sensing Image Fusion via Sparse Representations Over Learned Dictionaries

Remote sensing image fusion via wavelet transform and sparse representation

Sparse Representation and Pca Method for Image Fusion in Remote Sensing

A new method for multi-source remote sensing image fusion

The Realistic Fusion of Multi-spectral Images

Fusion of Remote Sensing Image with Compressed Sensing Based on Wavelet Sparse Basis

A pixel-level fusion method for multi-source optical remote sensing image combining the principal component analysis and curvelet transform

Fusion of Infrared and Visible Images Via Multi-Layer Convolutional Sparse Representation

Multi-Modal Image Fusion Via Sparse Representation and Multi-Scale Anisotropic Guided Measure

The Study on Image Fusion for Medium and High Spatial Resolution Remote Sensing Images

Multi-focus Image Fusion Based on Adaptive Sparse Representation

Multi-Spectral and Panchromatic Image Fusion Based on His-Wavelet Transform and Mopso Algorithm

An Image Fusion Method Based on Sparse Representation and Sum Modified-Laplacian in NSCT Domain

Sparse Representation based Multi-sensor Image Fusion: A Review

New Insights into Multi-focus Image Fusion: A Fusion Method Based on Multi-dictionary Linear Sparse Representation and Region Fusion Model

Sparse Representation-Based Multi-Focus Image Fusion Method via Local Energy in Shearlet Domain

Hyperspectral and Multispectral Image Fusion based on a Sparse Representation

Multi-focus image fusion based on non-negative sparse representation and patch-level consistency rectification

A Novel Fusion Strategy and Convolutional Sparse Coding for Robot Multisource Image Fusion

Fusion of Hyperspectral and Multispectral Images by Convolutional Sparse Representation