Abstract:We introduce a new model for multivariate probabilistic time series prediction, designed to flexibly address a range of tasks including forecasting, interpolation, and their combinations. Building on copula theory, we propose a simplified objective for the recently-introduced transformer-based attentional copulas (TACTiS), wherein the number of distributional parameters now scales linearly with the number of variables instead of factorially. The new objective requires the introduction of a training curriculum, which goes hand-in-hand with necessary changes to the original architecture. We show that the resulting model has significantly better training dynamics and achieves state-of-the-art performance across diverse real-world forecasting tasks, while maintaining the flexibility of prior work, such as seamless handling of unaligned and unevenly-sampled time series. Code is made available at <a class="link-external link-https" href="https://github.com/ServiceNow/TACTiS" rel="external noopener nofollow">this https URL</a>.

What problem does this paper attempt to address?

The paper primarily addresses several key issues in multivariate time series forecasting and proposes an improved method to optimize forecasting performance. Specifically, the study focuses on the following aspects: 1. **Multivariate Probabilistic Time Series Forecasting**: The research aims to handle a range of tasks, including forecasting, interpolation, and combinations of these tasks. These problems are common in the real world but are usually studied in isolation. 2. **Handling Complex Data Distributions**: The paper discusses scenarios involving a large number of time series, arbitrarily complex data distributions, heterogeneous or irregular sampling frequencies, missing data, and conditionally deterministic covariates (e.g., holiday indicators). 3. **Flexibility and Generality**: A general model is proposed that can flexibly handle the various situations mentioned above to meet the diverse needs of practical applications. 4. **Optimizing the Training Process**: The paper addresses issues in the previously proposed Transformer-Attentional Copulas for Time Series (TACT iS) method, such as poor training dynamics and suboptimal prediction results due to the factorial growth of the number of factors with the number of variables. A simplified training process is proposed. 5. **Improved Model (TACT iS-2)**: This model simplifies the training objective function, making the number of parameters grow linearly with the number of variables instead of factorially. This significantly improves training dynamics, allowing the model to converge faster to better solutions. 6. **Empirical Analysis**: Experiments demonstrate that the proposed TACT iS-2 model not only achieves state-of-the-art levels of prediction accuracy but also significantly reduces the computational effort required for real-world forecasting tasks. In summary, the paper is primarily dedicated to addressing the generality and flexibility issues in multivariate time series forecasting. By introducing a new model—TACT iS-2, it achieves dual improvements in training efficiency and prediction accuracy.

TACTiS-2: Better, Faster, Simpler Attentional Copulas for Multivariate Time Series

TFEformer: Temporal Feature Enhanced Transformer for Multivariate Time Series Forecasting

Foreformer: an Enhanced Transformer-Based Framework for Multivariate Time Series Forecasting

Perceiver-based CDF Modeling for Time Series Forecasting

TwinS: Revisiting Non-Stationarity in Multivariate Time Series Forecasting

Expressing Multivariate Time Series as Graphs with Time Series Attention Transformer

2-Cats: 2D Copula Approximating Transforms

Multi-Task Copula by Sparse Graph Regression

VCformer: Variable Correlation Transformer with Inherent Lagged Correlation for Multivariate Time Series Forecasting

UniTST: Effectively Modeling Inter-Series and Intra-Series Dependencies for Multivariate Time Series Forecasting

TiVaT: Joint-Axis Attention for Time Series Forecasting with Lead-Lag Dynamics

Copula Conformal Prediction for Multi-step Time Series Forecasting

Are Self-Attentions Effective for Time Series Forecasting?

Transformer Multivariate Forecasting: Less is More?

Revitalizing Multivariate Time Series Forecasting: Learnable Decomposition with Inter-Series Dependencies and Intra-Series Variations Modeling

High-Dimensional Multivariate Forecasting with Low-Rank Gaussian Copula Processes

Biased Temporal Convolution Graph Network for Time Series Forecasting with Missing Values.

Spatial-Temporal Convolutional Transformer Network for Multivariate Time Series Forecasting

CATS: Enhancing Multivariate Time Series Forecasting by Constructing Auxiliary Time Series as Exogenous Variables

Spatiotemporal Attention for Multivariate Time Series Prediction and Interpretation

COPAR - Multivariate time series modeling using the COPula AutoRegressive model