Abstract:It is challenging to scale time series forecasting models such that they forecast accurately for multiple distinct domains and datasets, all with potentially different underlying collection procedures (e.g., sample resolution), patterns (e.g., periodicity), and prediction requirements (e.g., reconstruction vs. forecasting). We call this general task universal forecasting. Existing methods usually assume that input data is regularly sampled, and they forecast to pre-determined horizons, resulting in failure to generalise outside of the scope of their training. We propose the DAM - a neural model that takes randomly sampled histories and outputs an adjustable basis composition as a continuous function of time for forecasting to non-fixed horizons. It involves three key components: (1) a flexible approach for using randomly sampled histories from a long-tail distribution, that enables an efficient global perspective of the underlying temporal dynamics while retaining focus on the recent history; (2) a transformer backbone that is trained on these actively sampled histories to produce, as representational output, (3) the basis coefficients of a continuous function of time. We show that a single univariate DAM, trained on 25 time series datasets, either outperformed or closely matched existing SoTA models at multivariate long-term forecasting across 18 datasets, including 8 held-out for zero-shot transfer, even though these models were trained to specialise for each dataset-horizon combination. This single DAM excels at zero-shot transfer and very-long-term forecasting, performs well at imputation, is interpretable via basis function composition and attention, can be tuned for different inference-cost requirements, is robust to missing and irregularly sampled data {by design}.

Lag-Llama: Towards Foundation Models for Time Series Forecasting

Lag-Llama: Towards Foundation Models for Probabilistic Time Series Forecasting

Effective LSTMs with Seasonal-Trend Decomposition and Adaptive Learning and Niching-Based Backtracking Search Algorithm for Time Series Forecasting

A Mamba Foundation Model for Time Series Forecasting

Generalizable Memory-driven Transformer for Multivariate Long Sequence Time-series Forecasting

Scaleformer: Iterative Multi-scale Refining Transformers for Time Series Forecasting

LagCNN: A Fast Yet Effective Model for Multivariate Long-term Time Series Forecasting

Taming Pre-trained LLMs for Generalised Time Series Forecasting via Cross-modal Knowledge Distillation

TCLN: A Transformer-based Conv-LSTM Network for Multivariate Time Series Forecasting

Chronos: Learning the Language of Time Series

Time-LLM: Time Series Forecasting by Reprogramming Large Language Models

Unified Training of Universal Time Series Forecasting Transformers

Large Language Models for Financial Aid in Financial Time-series Forecasting

Scaling Law for Time Series Forecasting

DAM: Towards A Foundation Model for Time Series Forecasting

Optimal Latent Space Forecasting for Large Collections of Short Time Series Using Temporal Matrix Factorization

Test Time Learning for Time Series Forecasting

TimeGPT in Load Forecasting: A Large Time Series Model Perspective

Adaptive Multi-Scale Decomposition Framework for Time Series Forecasting

LeRet: Language-Empowered Retentive Network for Time Series Forecasting

Time Series Forecasting with Multi-scale Decomposition and Fourier Neural Operators