Abstract:Predicting future frames using historical spatiotemporal data sequences is challenging and critical, and it is receiving a lot of attention these days from academic and industrial scholars. Most spatiotemporal predictive algorithms ignore the valuable backward reasoning ability and the disparate learning complexities among different layers and hence, cannot build good long-term dependencies and spatial correlations,resulting in suboptimal solutions. To address the aforementioned issues, we propose a two-stage coarse-to-fine spatiotemporal predictive model with bidirectional distillation and level-specific meta-adaptation(See More)in this paper, which includes a bidirectional distillation network(BDN) and a level-specific meta-adapter(LMA), to gain bidirectional multilevel reasoning. In the first stage, BDN concentrates on bidirectional dynamics modeling and coarsely constructs spatial correlations of different layers, while LMA is introduced in the second fine-tuning stage to refine the multilevel spatial correlations from a meta-learning perspective.In particular, BDN mimics the forward and backward reasoning abilities of humans in a distillation manner,which aids in the development of long-term dependencies. The LMA views learning of different layers as disparate but related tasks and guides the transfer of learning experiences among these tasks through learning complexities. Thus, each layer could be closer to its solutions and could extract more informative spatial correlations. By capturing the enhanced short-term spatial correlations and long-term temporal dependencies,the proposed model could extract adequate knowledge from sequential historical observations and accurately predict future frames whose backtracking preconditions are consistent with the historical sequence. Our work is general and robust enough to be integrated into most spatiotemporal predictive models without requiring additional computation or memory cost during inference. Extensive experiments on four widely used predictive learning benchmarks validated the proposed model's effectiveness in comparison to state-of-the-art approaches(e.g., 10.6% improvement of Mean Squared Error on the Moving MNIST dataset).

Learning Spatial–Temporal Consistency for Satellite Image Sequence Prediction

Exploiting Spatial–Temporal Dynamics for Satellite Image Sequence Prediction

Interpretable and Generalizable Spatiotemporal Predictive Learning with Disentangled Consistency

MSTCGAN: Multiscale Time Conditional Generative Adversarial Network for Long-Term Satellite Image Sequence Prediction

TinyPredNet: A Lightweight Framework for Satellite Image Sequence Prediction

A Multi-scale Spatiotemporal Attention Network for Ground-Based Remote Sensing Cloud Image Sequence Prediction

A Spatio-Temporal Sequence-to-sequence Network for Traffic Flow Prediction

A sequence-to-sequence based multi-scale deep learning model for satellite cloud image prediction

Predicting High-Dimensional Time Series Data with Spatial, Temporal and Global Information

CSIP-Net: Convolutional Satellite Image Prediction Network for Meteorological Satellite Infrared Observation Imaging

Satellite Image Prediction Relying on GAN and LSTM Neural Networks

Exploring and Exploiting High-Order Spatial-Temporal Dynamics for Long-Term Frame Prediction

BE-STI: Spatial-Temporal Integrated Network for Class-agnostic Motion Prediction with Bidirectional Enhancement

MCSTNet: a memory-contextual spatiotemporal transfer network for prediction of SST sequences and fronts with remote sensing data

A spatiotemporal deep learning model for sea surface temperature field prediction using time-series satellite data

SeeMore: a Spatiotemporal Predictive Model with Bidirectional Distillation and Level-Specific Meta-Adaptation

Is Single Enough? A Joint Spatiotemporal Feature Learning Framework for Multivariate Time Series Prediction

SAM-Net: Spatio-Temporal Sequence Typhoon Cloud Image Prediction Net with Self-Attention Memory

On Understanding of Spatiotemporal Prediction Model

Spatial-Temporal Semantic Perception Network for Remote Sensing Image Semantic Change Detection

A Short-Long Term Sequences Learning Network for Precipitation Nowcasting