Abstract:Due to the non-stationary nature, the distribution of real-world multivariate time series (MTS) changes over time, which is known as distribution drift. Most existing MTS forecasting models greatly suffer from distribution drift and degrade the forecasting performance over time. Existing methods address distribution drift via adapting to the latest arrived data or self-correcting per the meta knowledge derived from future data. Despite their great success in MTS forecasting, these methods hardly capture the intrinsic distribution changes, especially from a distributional perspective. Accordingly, we propose a novel framework temporal conditional variational autoencoder (TCVAE) to model the dynamic distributional dependencies over time between historical observations and future data in MTSs and infer the dependencies as a temporal conditional distribution to leverage latent variables. Specifically, a novel temporal Hawkes attention mechanism represents temporal factors subsequently fed into feed-forward networks to estimate the prior Gaussian distribution of latent variables. The representation of temporal factors further dynamically adjusts the structures of Transformer-based encoder and decoder to distribution changes by leveraging a gated attention mechanism. Moreover, we introduce conditional continuous normalization flow to transform the prior Gaussian to a complex and form-free distribution to facilitate flexible inference of the temporal conditional distribution. Extensive experiments conducted on six real-world MTS datasets demonstrate the TCVAE's superior robustness and effectiveness over the state-of-the-art MTS forecasting baselines. We further illustrate the TCVAE applicability through multifaceted case studies and visualization in real-world scenarios.

What problem does this paper attempt to address?

The paper attempts to address the issue of performance degradation in multivariate time series (MTS) forecasting due to distributional drift. Specifically: 1. **Problem Background**: Real-world multivariate time series data are often non-stationary, meaning their statistical properties change over time, a phenomenon known as "distributional drift." Most existing MTS forecasting models perform poorly when faced with distributional drift. 2. **Limitations of Existing Methods**: Current methods for handling distributional drift mainly adapt to the latest incoming data or self-correct based on meta-knowledge of future data. However, these methods struggle to capture the essential characteristics of distributional changes, especially at the distributional level. 3. **Proposed New Method**: To address the above issues, the authors propose a new framework—Temporal Conditional Variational Autoencoder (TCV AE), aimed at modeling the dynamic distributional dependencies between historical observations and future data over time and inferring them as a temporal conditional distribution. The specific implementation includes: - Using a novel temporal Hawkes attention mechanism to represent temporal factors and estimating the prior Gaussian distribution of latent variables through a feedforward network. - Introducing a gated attention mechanism to dynamically adjust the Transformer-based encoder and decoder structures to adapt to distributional changes. - Utilizing Conditional Continuous Normalizing Flow (CCNF) to transform the prior Gaussian distribution into a more complex, free-form distribution, thereby more flexibly inferring the temporal conditional distribution. 4. **Experimental Validation**: Extensive experiments on six real-world MTS datasets validate the superior robustness and effectiveness of TCV AE in handling distributional drift, and multiple case studies and visualizations demonstrate its application potential in real-world scenarios.

Distributional Drift Adaptation with Temporal Conditional Variational Autoencoder for Multivariate Time Series Forecasting

Distributional Drift Adaptation With Temporal Conditional Variational Autoencoder for Multivariate Time Series Forecasting

Temporal Conditional VAE for Distributional Drift Adaptation in Multivariate Time Series

TFEformer: Temporal Feature Enhanced Transformer for Multivariate Time Series Forecasting

Considering Nonstationary within Multivariate Time Series with Variational Hierarchical Transformer for Forecasting

TiVaT: Joint-Axis Attention for Time Series Forecasting with Lead-Lag Dynamics

A Novel Encoder-Decoder Model for Multivariate Time Series Forecasting

Robust Multivariate Time Series Forecasting against Intra- and Inter-Series Transitional Shift

VARIATIONAL ADAPTIVE GRAPH TRANSFORMER FOR MULTIVARIATE TIME SERIES MODELING

Generative Time Series Forecasting with Diffusion, Denoise, and Disentanglement

Generalizable Memory-driven Transformer for Multivariate Long Sequence Time-series Forecasting

A Multi-View Multi-Task Learning Framework for Multi-Variate Time Series Forecasting

Hybrid Variational Autoencoder for Time Series Forecasting

Revitalizing Multivariate Time Series Forecasting: Learnable Decomposition with Inter-Series Dependencies and Intra-Series Variations Modeling

Sequence Attention for Multivariate Time Series Forecasting

Dynamic Spatial-Temporal Embedding Via Neural Conditional Random Field for Multivariate Time Series Forecasting

Biased Temporal Convolution Graph Network for Time Series Forecasting with Missing Values.

VCformer: Variable Correlation Transformer with Inherent Lagged Correlation for Multivariate Time Series Forecasting

TCLN: A Transformer-based Conv-LSTM Network for Multivariate Time Series Forecasting

TDG4MSF: A temporal decomposition enhanced graph neural network for multivariate time series forecasting

Cluster-aware attentive convolutional recurrent network for multivariate time-series forecasting