Abstract:Pansharpening aims to generate a high-resolution multispectral (HRMS) image by merging a low-resolution multispectral (LRMS) image with a high-resolution panchromatic (PAN) image. While traditional model-based pansharpening algorithms have strong theoretical foundations, their performance, and generalizability are limited by handcrafted formulations. In contrast, recent deep learning (DL) approaches outperform model-based algorithms but do not effectively consider the physical properties of multispectral (MS) images, such as their spatial and spectral dependencies. These physical properties facilitate the exploitation of the actual imaging process, leading to enhanced spatial and spectral fidelities. In this work, we propose a deep unfolded tensor rank minimization framework with generalized detail injection for pansharpening to overcome the weaknesses of both model- and learning-based approaches while leveraging their advantages. Specifically, we first formulate the pansharpening task as a tensor rank minimization problem to exploit the low-rankness of MS images, providing a robust theoretical foundation on the physical properties of MS data. We also develop a generalized detail injection component, which effectively exploits the information in the PAN images, and incorporates it into the optimization to improve generalizability and representation capability. Then, we define a data-driven regularizer to compensate for modeling inaccuracies in the low-rank model and solve the optimization problem using an iterative technique. Finally, the iterative algorithm is unfolded into a multistage deep network, in which the optimization variables are solved by closed-form solutions and a data-driven regularizer in each stage. Experimental results on various MS image datasets demonstrate that the proposed algorithm achieves better pansharpening performance and interpretability than state-of-the-art algorithms.

A deep unrolling pansharpening method based on spectral consistency and double spatial priors

Efficient and Accurate Hyperspectral Pansharpening Using 3D VolumeNet and 2.5D Texture Transfer

DuCoFPan: Dual-Condition Flow-based Network for Pan-sharpening

DPDU-Net: Double Prior Deep Unrolling Network for Pansharpening

Deep Algorithm Unrolling with Registration Embedding for Pansharpening

Deep Unfolding Tensor Rank Minimization With Generalized Detail Injection for Pansharpening

Optimization Algorithm Unfolding Deep Networks of Detail Injection Model for Pansharpening

A Multi-Stage Progressive Pansharpening Network Based on Detail Injection with Redundancy Reduction

Detail-Injection-Based Multiscale Asymmetric Residual Network for Pansharpening

PanCSC-Net: A Model-Driven Deep Unfolding Method for Pansharpening

A New Pansharpening Method Using Multistage Joint Bilateral Filtering

Pansharpening Based on Joint Gaussian Guided Upsampling.

DiTBN: Detail Injection-Based Two-Branch Network for Pansharpening of Remote Sensing Images

VO+Net: an Adaptive Approach Using Variational Optimization and Deep Learning for Panchromatic Sharpening

Detail Injection-Based Deep Convolutional Neural Networks for Pansharpening

Detail Information Prior Net for Remote Sensing Image Pansharpening

Memory-augmented Deep Conditional Unfolding Network for Pansharpening

A Deep Unfolding Network Based on Intrinsic Image Decomposition for Pansharpening

Attention-based dual residual network-based for multi-spectral pan-sharpening

Rethinking Pan-Sharpening Via Spectral-Band Modulation

Hyperspectral Pansharpening via Deep Detail Injection Network