Abstract:Achieving a balance between spectral resolution and spatial resolution in multi-spectral remote sensing images is challenging due to physical constraints. Consequently, pan-sharpening technology was developed to address this challenge. While significant progress was recently achieved in deep-learning-based pan-sharpening techniques, most existing deep learning approaches face two primary limitations: (1) convolutional neural networks (CNNs) struggle with long-range dependency issues, and (2) significant detail loss during deep network training. Moreover, despite these methods' pan-sharpening capabilities, their generalization to full-sized raw images remains problematic due to scaling disparities, rendering them less practical. To tackle these issues, we introduce in this study a multi-spectral remote sensing image fusion network, termed TAMINet, which leverages a two-stream coordinate attention mechanism and multi-detail injection. Initially, a two-stream feature extractor augmented with the coordinate attention (CA) block is employed to derive modal-specific features from low-resolution multi-spectral (LRMS) images and panchromatic (PAN) images. This is followed by feature-domain fusion and pan-sharpening image reconstruction. Crucially, a multi-detail injection approach is incorporated during fusion and reconstruction, ensuring the reintroduction of details lost earlier in the process, which minimizes high-frequency detail loss. Finally, a novel hybrid loss function is proposed that incorporates spatial loss, spectral loss, and an additional loss component to enhance performance. The proposed methodology's effectiveness was validated through experiments on WorldView-2 satellite images, IKONOS, and QuickBird, benchmarked against current state-of-the-art techniques. Experimental findings reveal that TAMINet significantly elevates the pan-sharpening performance for large-scale images, underscoring its potential to enhance multi-spectral remote sensing image quality.

Hyperspectral Pansharpening via Deep Detail Injection Network

A Deep-Shallow Fusion Network With Multidetail Extractor and Spectral Attention for Hyperspectral Pansharpening

SSETPAN: Spatial-Spectral Enhanced Transformer Based Network for Pansharpening

Efficient and Accurate Hyperspectral Pansharpening Using 3D VolumeNet and 2.5D Texture Transfer

Pansharpening via Detail Injection Based Convolutional Neural Networks

Detail Information Prior Net for Remote Sensing Image Pansharpening

A Multi-Stage Progressive Pansharpening Network Based on Detail Injection with Redundancy Reduction

Sparse Representation-Based Detail-Injection Method for Pan-Sharpening

Detail-Injection-Based Multiscale Asymmetric Residual Network for Pansharpening

CNN-Based Hyperspectral Pansharpening With Arbitrary Resolution

A deep unrolling pansharpening method based on spectral consistency and double spatial priors

Deep Multiscale Detail Networks for Multiband Spectral Image Sharpening

A pansharpening method combining iterative filtering and NSST-NSML-PAPCNN to optimize spatial detail extraction and injection

Domain-specific knowledge-driven pan-sharpening algorithm

Dynamic Hyperspectral Pansharpening CNNs

Pannet: A Deep Network Architecture For Pan-Sharpening

Hipandas: Hyperspectral Image Joint Denoising and Super-Resolution by Image Fusion with the Panchromatic Image

Pan-Sharpening Network of Multi-Spectral Remote Sensing Images Using Two-Stream Attention Feature Extractor and Multi-Detail Injection (TAMINet)

Deep dual stream residual network with contextual attention for pansharpening of remote sensing images

Deep Parameterized Neural Networks for Hyperspectral Image Denoising

Towards Robust Pansharpening: A Large-Scale High-Resolution Multi-Scene Dataset and Novel Approach