Abstract:Spatiotemporal fusion (STF) is considered an effective way to address the mutual constraints of the spatiotemporal resolutions of the remote sensing images from a single satellite sensor. Although the deep learning (DL)-based STF methods have shown great potential so far, there are still some deficiencies. Supervised DL-based methods usually train the network on the original scale data with richer spatial information, but they utilize only the data at auxiliary dates to generate the fusion data at prediction dates, which may perform poorly when significant spatiotemporal changes occur between the auxiliary and prediction dates. For self-supervised DL-based methods, the training can focus more on the data at prediction dates, but the network is generally trained on the down-sampled data with less spatial information, causing insufficient spatial structures of the fusion results. Based on the above, we innovatively combine supervised and self-supervised learning and propose a dual-stage cascade STF framework. In the first stage, a model based on supervised learning is trained on the data at auxiliary dates to extract abundant spatial features and obtain the initial fusion results. In the second stage, we utilize a self-supervised strategy to excavate the spatiotemporal features of prediction dates based on the initial fusion results and the observed data at prediction dates. In addition, to alleviate the insufficient consideration of the temporal correlation in existing STF methods, we design a temporal consistency loss function to fully utilize the temporal correlation information between multiple prediction dates to generate more accurate fusion results. The proposed framework can not only realize the STF of remote sensing images but also be well applied to the STF of remote sensing products such as land surface temperature (LST). Based on the quantitative results of three datasets, the proposed method improves the quantitative indices of root mean square error (RMSE), structural similarity (SSIM), erreur relative global adimensionnelle de synthese (ERGAS), peak signal to noise ratio (PSNR), and spectral angle mapper (SAM) by up to 0.2014, 0.0279, 0.6797, 0.7629, and 0.2006 at most compared with the best comparison method, which fully shows the superiority of our method.

Fusing Landsat 8 and Sentinel-2 data for 10-m dense time-series imagery using a degradation-term constrained deep network

Deep Learning-Based Spatiotemporal Fusion Architecture of Landsat 8 and Sentinel-2 Data for 10 m Series Imagery

Fusing Landsat-7, Landsat-8 and Sentinel-2 Surface Reflectance to Generate Dense Time Series Images with 10m Spatial Resolution

STINet: Vegetation Changes Reconstruction Through a Transformer-Based Spatiotemporal Fusion Approach in Remote Sensing

Deep Learning-Based Spatiotemporal Data Fusion Using a Patch-to-Pixel Mapping Strategy and Model Comparisons

Generating Sentinel-2 all-band 10-m data by sharpening 20/60-m bands: A hierarchical fusion network

STFRDN: a residual dense network for remote sensing image spatiotemporal fusion

An Enhanced Deep Convolutional Model for Spatiotemporal Image Fusion

Enhanced Spatiotemporal Fusion via MODIS-Like Images

Generating 10-Meter Resolution Land Use and Land Cover Products Using Historical Landsat Archive Based on Super Resolution Guided Semantic Segmentation Network

A Deep Learning-Based Spatio-Temporal NDVI Data Fusion Model

Time-Series-Based Spatiotemporal Fusion Network for Improving Crop Type Mapping

A Robust Hybrid Deep Learning Model for Spatiotemporal Image Fusion

Developing a Multi-Scale Convolutional Neural Network for Spatiotemporal Fusion to Generate MODIS-like Data Using AVHRR and Landsat Images

The Improved U-STFM: A Deep Learning-Based Nonlinear Spatial-Temporal Fusion Model for Land Surface Temperature Downscaling

Improving agricultural field parcel delineation with a dual branch spatiotemporal fusion network by integrating multimodal satellite data

Supervised and self-supervised learning-based cascade spatiotemporal fusion framework and its application

Deep Learning for Feature-Level Data Fusion: Higher Resolution Reconstruction of Historical Landsat Archive

STAIR 2.0: A Generic and Automatic Algorithm to Fuse Modis, Landsat, and Sentinel-2 to Generate 10 m, Daily, and Cloud-/Gap-Free Surface Reflectance Product

A Land Cover Classification Method for High-Resolution Remote Sensing Images Based on NDVI Deep Learning Fusion Network

A Comprehensive and Automated Fusion Method: The Enhanced Flexible Spatiotemporal DAta Fusion Model for Monitoring Dynamic Changes of Land Surface