Abstract:Achieving a balance between spectral resolution and spatial resolution in multi-spectral remote sensing images is challenging due to physical constraints. Consequently, pan-sharpening technology was developed to address this challenge. While significant progress was recently achieved in deep-learning-based pan-sharpening techniques, most existing deep learning approaches face two primary limitations: (1) convolutional neural networks (CNNs) struggle with long-range dependency issues, and (2) significant detail loss during deep network training. Moreover, despite these methods' pan-sharpening capabilities, their generalization to full-sized raw images remains problematic due to scaling disparities, rendering them less practical. To tackle these issues, we introduce in this study a multi-spectral remote sensing image fusion network, termed TAMINet, which leverages a two-stream coordinate attention mechanism and multi-detail injection. Initially, a two-stream feature extractor augmented with the coordinate attention (CA) block is employed to derive modal-specific features from low-resolution multi-spectral (LRMS) images and panchromatic (PAN) images. This is followed by feature-domain fusion and pan-sharpening image reconstruction. Crucially, a multi-detail injection approach is incorporated during fusion and reconstruction, ensuring the reintroduction of details lost earlier in the process, which minimizes high-frequency detail loss. Finally, a novel hybrid loss function is proposed that incorporates spatial loss, spectral loss, and an additional loss component to enhance performance. The proposed methodology's effectiveness was validated through experiments on WorldView-2 satellite images, IKONOS, and QuickBird, benchmarked against current state-of-the-art techniques. Experimental findings reveal that TAMINet significantly elevates the pan-sharpening performance for large-scale images, underscoring its potential to enhance multi-spectral remote sensing image quality.

GF-CSTNet: A Method for Pan-Sharpening Remote Sensing Images by Integrating CSPNet and Transformer

A novel pansharpening method based on cross stage partial network and transformer

SSETPAN: Spatial-Spectral Enhanced Transformer Based Network for Pansharpening

Efficient and Accurate Hyperspectral Pansharpening Using 3D VolumeNet and 2.5D Texture Transfer

STCP: Synergistic Transformer and Convolutional Neural Network for Pansharpening

Pan-Sharpening with Customized Transformer and Invertible Neural Network

Effective Pan-Sharpening with Transformer and Invertible Neural Network

Remote Sensing Image Fusion Based on Two-stream Fusion Network

Transformer-based dual path cross fusion for pansharpening remote sensing images

A Cnn-Based Pan-Sharpening Method For Integrating Panchromatic And Multispectral Images Using Landsat 8

Transformer-Based Dual-Branch Multiscale Fusion Network for Pan-Sharpening Remote Sensing Images

Pan-Sharpening via Multiscale Dynamic Convolutional Neural Network

Local-Global Based High-Resolution Spatial-Spectral Representation Network for Pansharpening

PSMD-Net: A Novel Pan-Sharpening Method Based on a Multiscale Dense Network

MDSCNN: Remote Sensing Image Spatial–Spectral Fusion Method via Multi-Scale Dual-Stream Convolutional Neural Network

Pan-GAN: An unsupervised pan-sharpening method for remote sensing image fusion

A Remote-Sensing Image Pan-Sharpening Method Based on Multi-Scale Channel Attention Residual Network.

Pan-Sharpening Network of Multi-Spectral Remote Sensing Images Using Two-Stream Attention Feature Extractor and Multi-Detail Injection (TAMINet)

Transformer-based adaptive 3D residual CNN with sparse representation for PAN-sharpening of multispectral images

Multihead Global Attention and Spatial Spectral Information Fusion for Remote Sensing Image Compression

Pan-GAN: An unsupervised learning method for pan-sharpening in remote sensing image fusion using a generative adversarial network