Abstract:Pansharpening aims to generate a high-resolution multispectral (HRMS) image by merging a low-resolution multispectral (LRMS) image with a high-resolution panchromatic (PAN) image. While traditional model-based pansharpening algorithms have strong theoretical foundations, their performance, and generalizability are limited by handcrafted formulations. In contrast, recent deep learning (DL) approaches outperform model-based algorithms but do not effectively consider the physical properties of multispectral (MS) images, such as their spatial and spectral dependencies. These physical properties facilitate the exploitation of the actual imaging process, leading to enhanced spatial and spectral fidelities. In this work, we propose a deep unfolded tensor rank minimization framework with generalized detail injection for pansharpening to overcome the weaknesses of both model- and learning-based approaches while leveraging their advantages. Specifically, we first formulate the pansharpening task as a tensor rank minimization problem to exploit the low-rankness of MS images, providing a robust theoretical foundation on the physical properties of MS data. We also develop a generalized detail injection component, which effectively exploits the information in the PAN images, and incorporates it into the optimization to improve generalizability and representation capability. Then, we define a data-driven regularizer to compensate for modeling inaccuracies in the low-rank model and solve the optimization problem using an iterative technique. Finally, the iterative algorithm is unfolded into a multistage deep network, in which the optimization variables are solved by closed-form solutions and a data-driven regularizer in each stage. Experimental results on various MS image datasets demonstrate that the proposed algorithm achieves better pansharpening performance and interpretability than state-of-the-art algorithms.

Pansharpening Based on Low-Rank and Sparse Decomposition

Low-rank and Sparse Matrix Decomposition-Based Pan Sharpening

Pansharpening by Exploiting Sharpness of the Spatial Structure

A Joint Optimization Based Pansharpening via Subpixel-Shift Decomposition

Pansharpening by Combining Enhanced Spatial-Spectral Fidelity and Gradient-Domain Guided Filtering

Sparse Representation over Shared Coefficients in Multispectral Pansharpening

LRTCFPan: Low-Rank Tensor Completion Based Framework for Pansharpening

Pansharpening via semi-framelet-guided sparse reconstruction

A Variational Method with Kernel Estimation and Low Rank Prior for Pansharpening.

Convolutional Sparse Representation of Injected Details for Pansharpening

Sparse Representation-Based Detail-Injection Method for Pan-Sharpening

Rethinking Pan-Sharpening Via Spectral-Band Modulation

Variational Pansharpening Based on Coefficient Estimation With Nonlocal Regression

A FRAMELET SPARSE RECONSTRUCTION METHOD FOR PANSHARPENING WITH GUARANTEED CONVERGENCE

Deep Unfolding Tensor Rank Minimization With Generalized Detail Injection for Pansharpening

Pansharpening Based on Fuzzy Logic and Edge Activity

Fast and High-Quality Blind Multi-Spectral Image Pansharpening

Sparse representation based pan-sharpening

Locally Linear Detail Injection for Pansharpening

Memory-augmented Deep Conditional Unfolding Network for Pansharpening

The Fusion of Panchromatic and Multispectral Remote Sensing Images Via Tensor-Based Sparse Modeling and Hyper-Laplacian Prior