Abstract:Spatiotemporal fusion (STF) is considered an effective way to address the mutual constraints of the spatiotemporal resolutions of the remote sensing images from a single satellite sensor. Although the deep learning (DL)-based STF methods have shown great potential so far, there are still some deficiencies. Supervised DL-based methods usually train the network on the original scale data with richer spatial information, but they utilize only the data at auxiliary dates to generate the fusion data at prediction dates, which may perform poorly when significant spatiotemporal changes occur between the auxiliary and prediction dates. For self-supervised DL-based methods, the training can focus more on the data at prediction dates, but the network is generally trained on the down-sampled data with less spatial information, causing insufficient spatial structures of the fusion results. Based on the above, we innovatively combine supervised and self-supervised learning and propose a dual-stage cascade STF framework. In the first stage, a model based on supervised learning is trained on the data at auxiliary dates to extract abundant spatial features and obtain the initial fusion results. In the second stage, we utilize a self-supervised strategy to excavate the spatiotemporal features of prediction dates based on the initial fusion results and the observed data at prediction dates. In addition, to alleviate the insufficient consideration of the temporal correlation in existing STF methods, we design a temporal consistency loss function to fully utilize the temporal correlation information between multiple prediction dates to generate more accurate fusion results. The proposed framework can not only realize the STF of remote sensing images but also be well applied to the STF of remote sensing products such as land surface temperature (LST). Based on the quantitative results of three datasets, the proposed method improves the quantitative indices of root mean square error (RMSE), structural similarity (SSIM), erreur relative global adimensionnelle de synthese (ERGAS), peak signal to noise ratio (PSNR), and spectral angle mapper (SAM) by up to 0.2014, 0.0279, 0.6797, 0.7629, and 0.2006 at most compared with the best comparison method, which fully shows the superiority of our method.

PSSTFN: A Progressive Spatial–Temporal–Spectral Fusion Network for Remote Sensing Images

Enhanced Spatiotemporal Fusion via MODIS-Like Images

FTDN: Multispectral and Hyperspectral Image Fusion With Diverse Temporal Difference Spans

Deep Learning-Based Spatiotemporal Fusion Architecture of Landsat 8 and Sentinel-2 Data for 10 m Series Imagery

A Spatial and Temporal Non-Local Filter Based Data Fusion

STINet: Vegetation Changes Reconstruction Through a Transformer-Based Spatiotemporal Fusion Approach in Remote Sensing

SOSSF: Landsat-8 Image Synthesis on the Blending of Sentinel-1 and MODIS Data

Adaptive Channel Attention and Feature Super-Resolution for Remote Sensing Images Spatiotemporal Fusion.

MSFusion: Multistage for Remote Sensing Image Spatiotemporal Fusion Based on Texture Transformer and Convolutional Neural Network

A spatio-temporal-spectral fusion framework for downscaling Sentinel-2 images using UAV images

Remote Sensing Image Spatiotemporal Fusion Using a Generative Adversarial Network

Supervised and self-supervised learning-based cascade spatiotemporal fusion framework and its application

Fusing Landsat-7, Landsat-8 and Sentinel-2 Surface Reflectance to Generate Dense Time Series Images with 10m Spatial Resolution

Improving agricultural field parcel delineation with a dual branch spatiotemporal fusion network by integrating multimodal satellite data

Spatiotemporal Image Fusion With Spectrally Preserved Pre-Prediction: Tackling Complex Land-Cover Changes

Fusing Landsat 8 and Sentinel-2 data for 10-m dense time-series imagery using a degradation-term constrained deep network

An Adaptive Image Fusion Method for Sentinel-2 Images and High-Resolution Images with Long-Time Intervals.

A Two-Layers Super-Resolution Based Generation Adversarial Spatiotemporal Fusion Model.

Pansharpening-Based Spatio-Temporal Fusion for Predicting Intense Surface Changes

An Effective Multi-model Fusion Method for SAR and Optical Remote Sensing Images

Optimization-Based Hyperspectral Spatiotemporal Super-Resolution