Abstract:Due to the influence of the external atmospheric environment and the physical constraints of the sensors, it is difficult for existing medium spatial resolution sensors to produce intensive time-series images. The easy accessibility and similar sensor configuration of Landsat and Sentinel-2 data make them suitable for synthesizing time-series images. Currently, some deep learning-based studies have achieved significant success in multi-sensor fusion, but these deep-learning fusion networks only use single features that are not enough to produce intensive time-series images. Shallow features are easy to extract but have weak representation, while the deep network can extract abstract features with strong representation ability but faces the problem of gradient disappearance and model degradation. To address these problems, we propose a novel spatial-temporal-spectral integration model that can simultaneously fuse Landsat-7,8 and Sentinel-2 sensors. First, an enhanced residual dense network (ERDN) is proposed to improve Landsat's four bands (Blue, Green, Red, and Near-infrared) to 10 m spatial resolution. ERDN's residual dense structure utilizes abundant shallow features (i.e. texture and spectral information) and deep features (i.e. angles of view and aspect ratios of objects) to reconstruct spatial structures. At the same time, the introduced high-resolution images can provide reliable spatial auxiliary information for the missing image parts. Second, we employ a time-series-based reflectance adjustment (TRA) method to reduce the reflectance differences between Landsat and Sentinel-2 images. The fusion results indicate that our fusion framework can not only fill the gap caused by the Landsat-7 scan-line corrector (SLC) failure but also restore more spatial details than previous methods. More importantly, it can predict tiny reflectance changes in land cover. Overall, our study can produce continuous reflection observations with a higher time density and provides important data guarantee for extracting dynamic changes in land coverages.

Blending the Innovations of LSTM and EfficientNet for Time-Series Image Synthesis in Environmental Monitoring

A spatio-temporal LSTM model to forecast across multiple temporal and spatial scales

Efficient Multi-temporal and In-season Crop Mapping with Landsat Analysis Ready Data via Long Short-term Memory Networks

A Robust Hybrid Deep Learning Model for Spatiotemporal Image Fusion

Deep Neural Networks for automatic extraction of features in time series satellite images

Modeling Long- and Short-Term Temporal Patterns with Deep Neural Networks

Spatiotemporal MultiWaveNet for Efficiently Generating Environmental Spatiotemporal Series

Learning Blind Video Temporal Consistency

U-Net-LSTM: Time Series-Enhanced Lake Boundary Prediction Model

Tools for Extracting Spatio-Temporal Patterns in Meteorological Image Sequences: From Feature Engineering to Attention-Based Neural Networks

Using An Attention-Based LSTM Encoder–Decoder Network for Near Real-Time Disturbance Detection

Self-Supervised and Supervised Image Enhancement Networks with Time-Shift Module

Fusing Landsat-7, Landsat-8 and Sentinel-2 Surface Reflectance to Generate Dense Time Series Images with 10m Spatial Resolution

End-to-End Time-Lapse Video Synthesis from a Single Outdoor Image

Deep Learning Method for Cell-Wise Object Tracking, Velocity Estimation and Projection of Sensor Data over Time

ConvLSTMTransNet: A Hybrid Deep Learning Approach for Internet Traffic Telemetry

Sequence Image Datasets Construction via Deep Convolution Networks

Land Cover Classification via Multi-temporal Spatial Data by Recurrent Neural Networks

An Enhanced LSTM for Trend Following of Time Series

Spatiotemporal Representation Learning for Short and Long Medical Image Time Series