Abstract:We introduce a new deep detail network architecture with grouped multiscale dilated convolutions to sharpen images contain multiband spectral information. Specifically, our end-to-end network directly fuses low-resolution multispectral and panchromatic inputs to produce high-resolution multispectral results, which is the same goal of the pansharpening in remote sensing. The proposed network architecture is designed by utilizing our domain knowledge and considering the two aims of the pansharpening: spectral and spatial preservations. For spectral preservation, the up-sampled multispectral images are directly added to the output for lossless spectral information propagation. For spatial preservation, we train the proposed network in the high-frequency domain instead of the commonly used image domain. Different from conventional network structures, we remove pooling and batch normalization layers to preserve spatial information and improve generalization to new satellites, respectively. To effectively and efficiently obtain multiscale contextual features at a fine-grained level, we propose a grouped multiscale dilated network structure to enlarge the receptive fields for each network layer. This structure allows the network to capture multiscale representations without increasing the parameter burden and network complexity. These representations are finally utilized to reconstruct the residual images which contain spatial details of PAN. Our trained network is able to generalize different satellite images without the need for parameter tuning. Moreover, our model is a general framework, which can be directly used for other kinds of multiband spectral image sharpening, e.g., hyperspectral image sharpening. Experiments show that our model performs favorably against compared methods in terms of both qualitative and quantitative qualities.

Pannet: A Deep Network Architecture For Pan-Sharpening

SSETPAN: Spatial-Spectral Enhanced Transformer Based Network for Pansharpening

DuCoFPan: Dual-Condition Flow-based Network for Pan-sharpening

Pan-Sharpening Based On Parallel Pyramid Convolutional Neural Network

Pan-Sharpening via Multiscale Dynamic Convolutional Neural Network

Pan-Sharpening with Customized Transformer and Invertible Neural Network

PanFlowNet: A Flow-Based Deep Network for Pan-sharpening

Deep Gradient Projection Networks for Pan-sharpening

Detail Information Prior Net for Remote Sensing Image Pansharpening

PAN-Guided Band-Aware Multi-Spectral Feature Enhancement for Pan-Sharpening

PanCSC-Net: A Model-Driven Deep Unfolding Method for Pansharpening

Deep Multiscale Detail Networks for Multiband Spectral Image Sharpening

Memory-augmented Deep Conditional Unfolding Network for Pansharpening

Effective Pan-Sharpening with Transformer and Invertible Neural Network

Progressive Pan-Sharpening Via Cross-Scale Collaboration Networks

Domain-specific knowledge-driven pan-sharpening algorithm

Spatial and Spectral Extraction Network With Adaptive Feature Fusion for Pansharpening

Pan-GAN: An unsupervised pan-sharpening method for remote sensing image fusion

A Triple-Double Convolutional Neural Network for Panchromatic Sharpening

PSMD-Net: A Novel Pan-Sharpening Method Based on a Multiscale Dense Network

Towards Robust Pansharpening: A Large-Scale High-Resolution Multi-Scene Dataset and Novel Approach