Abstract:Filling gaps in high-resolution satellite imagery is essential for tracking vegetation changes over time. Spatiotemporal fusion (STF) aims to create fusion products that improve both spatial resolution and temporal coverage by using images from various remote sensing sources. However, most existing STF methods rely on the assumption that reflectance values for the same land-cover type remain constant between base and prediction dates, a premise often invalidated by the variability in vegetation disturbance and recovery scenarios, where differences in disturbance intensity, patterns, and phenological stages challenge this uniformity. Therefore, we propose a novel Transformer-based method, the spatiotemporal integration network (STINet), which is effective in fusing multiscale spatiotemporal dynamic features. STINet is structured around three key components. The feature fusion (FF) block effectively integrates multiscale spatiotemporal information into a deep learning (DL) framework. The adaptive feature extraction (AFE) block significantly improves the precision of pixel-level features, essential for detecting subtle changes in diverse vegetation patterns. The spatiotemporal-wise multihead self-attention (ST-MSA) module through its innovative self-attention mechanism across spatiotemporal dimensions, facilitated the reconstruction of vegetation dynamics. To verify the effectiveness and robustness of the proposed method, we conducted experiments in three carefully selected scenarios using multisensor and multitemporal imagery to reconstruct the dynamic changes in vegetation due to various disturbances and recovery processes. Compared to the four typical fusion methods [enhanced spatial and temporal adaptive reflectance fusion model (ESTARFM), flexible spatiotemporal data fusion method (FSDAF), extended super-resolution convolutional neural network (ESRCNN), and multiscene spatiotemporal fusion network (MUSTFN)], STINet achieved the best performance in preserving both spatial texture and spectral value. Furthermore, we showcased the applicability and effectiveness of STINet in accurately capturing phenological changes and distinguishing various farming activities.

Remote Sensing Image Spatiotemporal Fusion Using a Generative Adversarial Network

MLFF-GAN: A Multilevel Feature Fusion With GAN for Spatiotemporal Remote Sensing Images

The Spatially Seamless Spatiotemporal Fusion Model Based on Generative Adversarial Networks

CycleGAN-STF: Spatiotemporal Fusion via CycleGAN-Based Image Generation

Deep-Learning-Based Spatio-Temporal-Spectral Integrated Fusion of Heterogeneous Remote Sensing Images

A Unified Generative Adversarial Network With Convolution and Transformer for Remote Sensing Image Fusion

An Integrated Framework for the Heterogeneous Spatio-Spectral-Temporal Fusion of Remote Sensing Images

Deep Learning for Feature-Level Data Fusion: Higher Resolution Reconstruction of Historical Landsat Archive

Deep Learning-Based Spatiotemporal Fusion Architecture of Landsat 8 and Sentinel-2 Data for 10 m Series Imagery

Single Remote Sensing Image Super-Resolution Via a Generative Adversarial Network with Stratified Dense Sampling and Chain Training

AT-GAN: A generative adversarial network with attention and transition for infrared and visible image fusion

A Two-Layers Super-Resolution Based Generation Adversarial Spatiotemporal Fusion Model.

MFFAGAN: Generative Adversarial Network With Multilevel Feature Fusion Attention Mechanism for Remote Sensing Image Super-Resolution

Enhanced Spatiotemporal Fusion via MODIS-Like Images

Spatiotemporal Data Augmentation of MODIS‐Landsat Water Bodies Using Adversarial Networks

MSAt-GAN: a generative adversarial network based on multi-scale and deep attention mechanism for infrared and visible light image fusion

DSG-Fusion: Infrared and visible image fusion via generative adversarial networks and guided filter

MCBAM-GAN: The Gan Spatiotemporal Fusion Model Based on Multiscale and CBAM for Remote Sensing Images

Mun-GAN: A Multiscale Unsupervised Network for Remote Sensing Image Pansharpening

STINet: Vegetation Changes Reconstruction Through a Transformer-Based Spatiotemporal Fusion Approach in Remote Sensing

Remote Sensing Data Fusion With Generative Adversarial Networks: State-of-the-art methods and future research directions