Abstract:Hyperspectral and multispectral image fusion plays an important role in the fields such as image restoration, space robotics, remote sensing, computer vision, etc. The fused images take advantages of the high spectral resolutions of hyperspectral images and the high spatial resolutions of multispectral images. So there has been a recent outgrowth of interests of research in this field. However, the local structural image regularity and non-local image self-similarity are not yet explored in existing fusion methods. This paper develops a novel method for fusing hyperspectral and multispectral images with non-local structure tensor total variation regularization. Specifically, the weight function to measure self-similarity in non-local domain is introduced on the basis of the definition of structure tensor. Therefore, the proposed method can exploit the regularity and self-similarity of spatial-spectral image’s non-local structures simultaneously. In addition to the general formulation of non-local structure tensor variation, this paper presents the discrete non-local structure tensor variation for hyperspectral and multispectral image fusion whose measurements are discrete. This method can provide an alternative way to measure the image variation of multi-band image. The resulting problem is solved by alternating direction method of multipliers (ADMM), a simple but powerful algorithm for convex optimization problems. The optimization problem is handled by a splitting variables strategy, which is implemented by introducing four auxiliary variables regarding to the required fused image. The iteration related to the proposed non-local structure tensor regularization term is solved by the proximal mapping method. The solutions of other iterations are given through fast Fourier transforms. Extensive computational experiments are performed on various datasets. The results of index evaluation demonstrate the effectiveness of the proposed method. These results also indicate non-local structure tensor total variation regularization has obvious advantages over the state-of-the-art fusion methods.

Multisensor Image Fusion and Enhancement in Spectral Total Variation Domain

Fusion of infrared and visual images through multiscale hybrid unidirectional total variation

Image Fusion Method Based on Total Variation and Àtrous Wavelet

The Realistic Fusion of Multi-spectral Images

Hyperspectral and Multispectral Image Fusion Via Regularization on Non-local Structure Tensor Total Variation

A Novel Approach for Image Fusion Using Total Variation and Markov Random Field

A Total Variation with Joint Norms for Infrared and Visible Image Fusion

Multisource Image Fusion Method Using Support Value Transform

Medical Image Fusion and Denoising with Alternating Sequential Filter and Adaptive Fractional Order Total Variation.

The fusion of infrared and visible images via decomposition-based structure transfer and local saliency detection

Multi-scale Decomposition Based Fusion of Infrared and Visible Image Via Total Variation and Saliency Analysis

Nonsubsampied contourlet transform in multisensor images fusion

Image Fusion Via Dynamic Gradient Sparsity and Anisotropic Spectral-Spatial Total Variation

The Nonsubsampled Contourlet Transform in Multisensor Images Fusion

Multisensor video fusion based on spatial-temporal salience detection

Multiband Image Fusion Using Total Generalized Variation Regularization

Image Fusion Method Based On Multi-Directional Support Value Transform

Multispectral and Hyperspectral Image Fusion Based on Regularized Coupled Non-Negative Block-Term Tensor Decomposition

Multi-Level Image Fusion and Enhancement for Target Detection

Multimodal Medical Image Fusion Based on Multichannel Coupled Neural P Systems and Max-Cloud Models in Spectral Total Variation Domain

Vector Fusion of Multispectral Images Based on Multiwavelet Decomposition