Abstract:Pansharpening is normally utilized to take full advantage of all the available spectral and spatial information that are derived from a low-spatial-resolution multispectral (MS) image and its associated high-spatial-resolution (HR) panchromatic (PAN) image, respectively, producing a fused MS image with high spectral and spatial resolutions. Many methods have been recently developed based on convolutional neural networks (CNNs) for the pansharpening task, but most of them still have some drawbacks: 1) The information cannot efficiently flow in their simple stacked convolutional architectures, thereby hindering the representation ability of the networks. 2) They are commonly trained using supervised learning, which does not only require an extra effort to produce the simulated training data, but can also lead to scale-related problems in the fusion results. In this article, we propose a novel unsupervised CNN-based pansharpening method to overcome these limitations. Specifically, we design an iterative network architecture, in which a PAN-guided strategy and a set of skip connections are adopted to continuously extract and fuse the features from the input, thus enhancing the information reuse and transmission. Besides, we propose a new loss function for unsupervised training in which the relationships between the input MS and PAN images and the fused MS image are used to design the spatial constrains and spectral consistency, respectively. The typical quality index with no-reference is also added to this function to further adjust the spectral and spatial qualities. The designed loss function allows the network to be learned only on input images, without any hand-crafted labels (reference HR MS image). We evaluated the effectiveness of our designed network architecture and the combined loss function, and the experiments testify that our unsupervised strategy can also obtain promising results with minor spectral and spatial distortions compared with other traditional and supervised methods.

SSconv

SSETPAN: Spatial-Spectral Enhanced Transformer Based Network for Pansharpening

SSCAConv: Self-Guided Spatial-Channel Adaptive Convolution for Image Fusion

Efficient and Accurate Hyperspectral Pansharpening Using 3D VolumeNet and 2.5D Texture Transfer

Convolutional Neural Network for Pansharpening with Spatial Structure Enhancement Operator

Multispectral Pan-sharpening via Dual-Channel Convolutional Network with Convolutional LSTM Based Hierarchical Spatial-Spectral Feature Fusion

Pansharpening via Unsupervised Convolutional Neural Networks

MDSCNN: Remote Sensing Image Spatial–Spectral Fusion Method via Multi-Scale Dual-Stream Convolutional Neural Network

Pansharpening Based on Joint Gaussian Guided Upsampling.

Progressive Band-Separated Convolutional Neural Network for Multispectral Pansharpening.

Pan-Sharpening via Multiscale Dynamic Convolutional Neural Network

Multispectral Pansharpening Based on Multisequence Convolutional Recurrent Neural Network

STCP: Synergistic Transformer and Convolutional Neural Network for Pansharpening

Going Deeper with Densely Connected Convolutional Neural Networks for Multispectral Pansharpening

Cross Spectral and Spatial Scale Non-local Attention-Based Unsupervised Pansharpening Network

CCC-SSA-UNet: U-Shaped Pansharpening Network with Channel Cross-Concatenation and Spatial-Spectral Attention Mechanism for Hyperspectral Image Super-Resolution

Local-Global Based High-Resolution Spatial-Spectral Representation Network for Pansharpening

MSCSCformer: Multiscale Convolutional Sparse Coding-Based Transformer for Pansharpening

Convolutional LSTM-Based Hierarchical Feature Fusion for Multispectral Pan-Sharpening

MSCSCformer: Multi-scale Convolutional Sparse Coding-based Transformer for Pansharpening

Multibranch Cnn-Based Pansharpening with Skip Connection