Abstract:Predicting future frames using historical spatiotemporal data sequences is challenging and critical, and it is receiving a lot of attention these days from academic and industrial scholars. Most spatiotemporal predictive algorithms ignore the valuable backward reasoning ability and the disparate learning complexities among different layers and hence, cannot build good long-term dependencies and spatial correlations,resulting in suboptimal solutions. To address the aforementioned issues, we propose a two-stage coarse-to-fine spatiotemporal predictive model with bidirectional distillation and level-specific meta-adaptation(See More)in this paper, which includes a bidirectional distillation network(BDN) and a level-specific meta-adapter(LMA), to gain bidirectional multilevel reasoning. In the first stage, BDN concentrates on bidirectional dynamics modeling and coarsely constructs spatial correlations of different layers, while LMA is introduced in the second fine-tuning stage to refine the multilevel spatial correlations from a meta-learning perspective.In particular, BDN mimics the forward and backward reasoning abilities of humans in a distillation manner,which aids in the development of long-term dependencies. The LMA views learning of different layers as disparate but related tasks and guides the transfer of learning experiences among these tasks through learning complexities. Thus, each layer could be closer to its solutions and could extract more informative spatial correlations. By capturing the enhanced short-term spatial correlations and long-term temporal dependencies,the proposed model could extract adequate knowledge from sequential historical observations and accurately predict future frames whose backtracking preconditions are consistent with the historical sequence. Our work is general and robust enough to be integrated into most spatiotemporal predictive models without requiring additional computation or memory cost during inference. Extensive experiments on four widely used predictive learning benchmarks validated the proposed model's effectiveness in comparison to state-of-the-art approaches(e.g., 10.6% improvement of Mean Squared Error on the Moving MNIST dataset).

Spatial-Temporal-Decoupled Masked Pre-training for Spatiotemporal Forecasting

Spatio-Temporal-Decoupled Masked Pre-training: Benchmarked on Traffic Forecasting

Masked Spatio-Temporal Structure Prediction for Self-supervised Learning on Point Cloud Videos

MSTEM: Masked Spatiotemporal Event Series Modeling for Urban Undisciplined Events Forecasting

W-MAE: Pre-trained weather model with masked autoencoder for multi-variable weather forecasting

Revealing the Power of Masked Autoencoders in Traffic Forecasting

Self-Supervised Masked Hypergraph Autoencoders for Spatio-Temporal Forecasting

STDNet: A Spatio-Temporal Decomposition Neural Network for Multivariate Time Series Forecasting

Forecast-MAE: Self-supervised Pre-training for Motion Forecasting with Masked Autoencoders

HiMTM: Hierarchical Multi-Scale Masked Time Series Modeling with Self-Distillation for Long-Term Forecasting

GD-MAE: Generative Decoder for MAE Pre-training on LiDAR Point Clouds

UP2ME: Univariate Pre-training to Multivariate Fine-tuning As a General-purpose Framework for Multivariate Time Series Analysis

DEFM: Delay E mbedding based Forecast Machine for Time Series Forecasting by Spatiotemporal Information Transformation

A$^{2}$-MAE: A spatial-temporal-spectral unified remote sensing pre-training method based on anchor-aware masked autoencoder

PreSTNet: Pre-trained Spatio-Temporal Network for traffic forecasting

SeeMore: a Spatiotemporal Predictive Model with Bidirectional Distillation and Level-Specific Meta-Adaptation

Ti-MAE: Self-Supervised Masked Time Series Autoencoders

Graph Masked Autoencoder for Spatio-Temporal Graph Learning

GPT-ST: Generative Pre-Training of Spatio-Temporal Graph Neural Networks

SatMAE: Pre-training Transformers for Temporal and Multi-Spectral Satellite Imagery

A Deep Spatio-Temporal Forecasting Model for Multi-Site Weather Prediction Post-Processing