Abstract:Hyperspectral and multispectral image fusion plays an important role in the fields such as image restoration, space robotics, remote sensing, computer vision, etc. The fused images take advantages of the high spectral resolutions of hyperspectral images and the high spatial resolutions of multispectral images. So there has been a recent outgrowth of interests of research in this field. However, the local structural image regularity and non-local image self-similarity are not yet explored in existing fusion methods. This paper develops a novel method for fusing hyperspectral and multispectral images with non-local structure tensor total variation regularization. Specifically, the weight function to measure self-similarity in non-local domain is introduced on the basis of the definition of structure tensor. Therefore, the proposed method can exploit the regularity and self-similarity of spatial-spectral image’s non-local structures simultaneously. In addition to the general formulation of non-local structure tensor variation, this paper presents the discrete non-local structure tensor variation for hyperspectral and multispectral image fusion whose measurements are discrete. This method can provide an alternative way to measure the image variation of multi-band image. The resulting problem is solved by alternating direction method of multipliers (ADMM), a simple but powerful algorithm for convex optimization problems. The optimization problem is handled by a splitting variables strategy, which is implemented by introducing four auxiliary variables regarding to the required fused image. The iteration related to the proposed non-local structure tensor regularization term is solved by the proximal mapping method. The solutions of other iterations are given through fast Fourier transforms. Extensive computational experiments are performed on various datasets. The results of index evaluation demonstrate the effectiveness of the proposed method. These results also indicate non-local structure tensor total variation regularization has obvious advantages over the state-of-the-art fusion methods.

Joint Image Pansharpening and Registration Via Structure Tensor Total Variation Regularization

SSETPAN: Spatial-Spectral Enhanced Transformer Based Network for Pansharpening

Structure Tensor-Driven Block-Based Adaptive Variational Pansharpening

Pan-sharpening: a Fast Variational Fusion Approach

Multiband Image Fusion Using Total Generalized Variation Regularization

Improved Generalized IHS Based on Total Variation for Pansharpening

Joint Ihs And Variational Methods For Pan-Sharpening Of Very High Resolution Imagery

A Variational Pan-Sharpening with Local Gradient Constraints.

Hyperspectral and Multispectral Image Fusion Via Regularization on Non-local Structure Tensor Total Variation

The Fusion of Panchromatic and Multispectral Remote Sensing Images Via Tensor-Based Sparse Modeling and Hyper-Laplacian Prior

A Total Variation Based Nonrigid Image Registration by Combining Parametric and Non-Parametric Transformation Models

LRTCFPan: Low-Rank Tensor Completion Based Framework for Pansharpening

VO+Net: an Adaptive Approach Using Variational Optimization and Deep Learning for Panchromatic Sharpening

Variational Pansharpening Based on Coefficient Estimation With Nonlocal Regression

Pansharpening via Double Nonconvex Tensor Low-Tubal-Rank Priors

Pan-sharpening of multi-spectral images using a new variational model

A Framelet-Based Iterative Pan-Sharpening Approach

Variational PCA Fusion for Pan-sharpening Very High Resolution Imagery

A New Variational Approach Based on Proximal Deep Injection and Gradient Intensity Similarity for Spatio-Spectral Image Fusion

Framelet based pan-sharpening via a variational method

Cartoon-Texture Decomposition-Based Variational Pansharpening