Abstract:Agricultural applications of remote sensing data typically require high spatial resolution and frequent observations. The increasing availability of high spatial resolution imagery meets the spatial resolution requirement well. However, the long revisit period and frequent cloud contamination severely compromise their ability to monitor crop growth, which is characterized by high temporal heterogeneity. Many spatiotemporal fusion methods have been developed to produce synthetic images with high spatial and temporal resolutions. However, these existing methods focus on fusing low and medium spatial resolution satellite data in terms of model development and validation. When it comes to fusing medium and high spatial resolution images, the applicability remains unknown and may face various challenges. To address this issue, we propose a novel spatiotemporal fusion method, the dual-stream spatiotemporal decoupling fusion architecture model, to fully realize the prediction of high spatial resolution images. Compared with other fusion methods, the model has distinct advantages: (a) It maintains high fusion accuracy and good spatial detail by combining deep-learning-based super-resolution method and partial least squares regression model through edge and color-based weighting loss function; and (b) it demonstrates improved transferability over time by introducing image gradient maps and partial least squares regression model. We tested the StarFusion model at 3 experimental sites and compared it with 4 traditional methods: STARFM (spatial and temporal adaptive reflectance fusion), FSDAF (flexible spatiotemporal data fusion), Fit-FC (regression model fitting, spatial filtering, and residual compensation), FIRST (fusion incorporating spectral autocorrelation), and a deep learning base method—super-resolution generative adversarial network. In addition, we also investigated the possibility of our method to use multiple pairs of coarse and fine images in the training process. The results show that multiple pairs of images provide better overall performance but both of them are better than other comparison methods. Considering the difficulty in obtaining multiple cloud-free image pairs in practice, our method is recommended to provide high-quality Gaofen-1 data with improved temporal resolution in most cases since the performance degradation of single pair is not significant.

A review of remote sensing image spatiotemporal fusion: Challenges, applications and recent trends

Spatiotemporal Fusion for Spectral Remote Sensing: A Statistical Analysis and Review.

A Review of Spatiotemporal Super-Resolution Mapping for Remote Sensing Data Fusion

Spatiotemporal Fusion of Multisource Remote Sensing Data: Literature Survey, Taxonomy, Principles, Applications, and Future Directions

Multi-Sensor Image Fusion: A Survey of the State of the Art

Spatial, Temporal, and Geometric Fusion for Remote Sensing Images

A review of remote sensing image fusion methods

Spatio-temporal Fusion for Remote Sensing Data: an Overview and New Benchmark

Spatiotemporal Fusion in Remote Sensing

Developing and Evaluating the Feasibility of a New Spatiotemporal Fusion Framework to Improve Remote Sensing Reflectance and Dynamic LAI Monitoring

Image Fusion in Remote Sensing: An Overview and Meta Analysis

Sparse Representation based Multi-sensor Image Fusion: A Review

Enhanced Spatiotemporal Fusion via MODIS-Like Images

A Dual-Perspective Spatio-Temporal Fusion Model for Remote Sensing Images by Discriminative Learning of the Spatial and Temporal Mapping

Multisource and Multitemporal Data Fusion in Remote Sensing: A Comprehensive Review of the State of the Art

A critical review on multi-sensor and multi-platform remote sensing data fusion approaches: current status and prospects

A Hybrid Spatio-Temporal Fusion Method for High Spatial Resolution Imagery: Fusion of Gaofen-1 and Sentinel-2 over Agricultural Landscapes

A Rigorously-Incremental Spatiotemporal Data Fusion Method for Fusing Remote Sensing Images.

Review of Pixel-Level Remote Sensing Image Fusion Based on Deep Learning

Integrated Fusion Framework Based on Semicoupled Sparse Tensor Factorization for Spatio-Temporal-spectral Fusion of Remote Sensing Images.

A Generalization of Spatial and Temporal Fusion Methods for Remotely Sensed Surface Parameters