Abstract:Existing convolution recurrent neural networks (ConvRNNs)-based memory cells majorly take advantage of gated structures and attention mechanisms to extract discontinuous latent associations for spatial-temporal sequence forecast (STSF) problems, which may lead to serious over-fitting and spurious relationships with correlated noise. It is a consensus that incorporating cause-effect relationships in modeling can alleviate these problems. In this paper, we propose a Causality Attention Unit (CAU) to assist ConvRNNs by complementing the causal inference ability in a plug-and-play way. Specifically, CAU serially consists of the attention module and causality module. The former is constructed by a spatial-channel attention layer, which preliminarily generates the correlated future with the correlations between historical memories and the current state. The latter borrows the idea of transfer entropy ( TE ) to detect the latent cause-effect relationships and precisely correct the correlated future. A space-time exchange strategy for accelerating the calculation of TE in CAU is also designed. CAU can be easily combined with the existing ConvRNN cells, and we construct a simple general model to predict long-term spatial-temporal series, which consists of encoder/decoder and stacked CAU paralleled to stacked ConvRNN cells. After determining the optimal model structure, we carry out a series of experiments to evaluate model performance, including comparisons with other advanced models, training loss analysis, and multiple ablation and sensitivity studies. Experimental results show that our proposed model can effectively improve the performances of existing ConvRNNs to the state-of-the-are level on representative public datasets, including Moving MNIST, KTH, BAIR, and WeatherBench. The ablation and sensitivity studies verify the superiority of CAU. The learned causal maps precisely distinguish the pixel attributions and motion characteristics in sophisticated entangled scenarios.

Multivariate Time Series Forecasting with Causal-Temporal Attention Network.

Multivariate Time Series Forecasting Based on Causal Inference with Transfer Entropy and Graph Neural Network

Root Cause Diagnosis in Multivariate Time Series Based on Modified Temporal Convolution and Multi-Head Self-Attention

TC-GAT: Graph Attention Network for Temporal Causality Discovery

Multivariate Time Series Forecasting with Transfer Entropy Graph

Deciphering Spatio-Temporal Graph Forecasting: A Causal Lens and Treatment

Causalformer: Causal Discovery-based Transformer for Multivariate Time Series Forecasting.

CAU: A Causality Attention Unit for Spatial-temporal Sequence Forecast

TS-CausalNN: Learning Temporal Causal Relations from Non-linear Non-stationary Time Series Data

Correlational graph attention-based Long Short-Term Memory network for multivariate time series prediction

Spatiotemporal Learning with Decoupled Causal Attention for Multivariate Time Series

Generating Fine-Grained Causality in Climate Time Series Data for Forecasting and Anomaly Detection

A Two-Stage Causality Method for Time Series Prediction Based on Feature Selection and Momentary Conditional Independence

A Hybrid Method with Adaptive Sub-Series Clustering and Attention-Based Stacked Residual LSTMs for Multivariate Time Series Forecasting

Sequence Attention for Multivariate Time Series Forecasting

Deep Coupling Network For Multivariate Time Series Forecasting

Multiple Time Series Forecasting with Dynamic Graph Modeling

When Dynamic Causality Comes to Graph-Temporal Neural Network

CATN: Cross Attentive Tree-Aware Network for Multivariate Time Series Forecasting.

Temporal Self-Attention-based Conv-LSTM Network for Multivariate Time Series Prediction

Towards the Disappearing Truth: Fine-Grained Joint Causal Influences Learning with Hidden Variable-Driven Causal Hypergraphs in Time Series