Abstract:Time series are generated in diverse domains such as economic, traffic, health, and energy, where forecasting of future values has numerous important applications. Not surprisingly, many forecasting methods are being proposed. To ensure progress, it is essential to be able to study and compare such methods empirically in a comprehensive and reliable manner. To achieve this, we propose TFB, an automated benchmark for Time Series Forecasting (TSF) methods. TFB advances the state-of-the-art by addressing shortcomings related to datasets, comparison methods, and evaluation pipelines: 1) insufficient coverage of data domains, 2) stereotype bias against traditional methods, and 3) inconsistent and inflexible pipelines. To achieve better domain coverage, we include datasets from 10 different domains : traffic, electricity, energy, the environment, nature, economic, stock markets, banking, health, and the web. We also provide a time series characterization to ensure that the selected datasets are comprehensive. To remove biases against some methods, we include a diverse range of methods, including statistical learning, machine learning, and deep learning methods, and we also support a variety of evaluation strategies and metrics to ensure a more comprehensive evaluations of different methods. To support the integration of different methods into the benchmark and enable fair comparisons, TFB features a flexible and scalable pipeline that eliminates biases. Next, we employ TFB to perform a thorough evaluation of 21 Univariate Time Series Forecasting (UTSF) methods on 8,068 univariate time series and 14 Multivariate Time Series Forecasting (MTSF) methods on 25 datasets. The results offer a deeper understanding of the forecasting methods, allowing us to better select the ones that are most suitable for particular datasets and settings. Overall, TFB and this evaluation provide researchers with improved means of designing new TSF methods.

Forecast Methods for Time Series Data: A Survey

Deep Time Series Forecasting Models: A Comprehensive Survey

Forecasting large collections of time series: feature-based methods

Time Series Dataset Survey for Forecasting with Deep Learning

Forecasting: theory and practice

Large Models for Time Series and Spatio-Temporal Data: A Survey and Outlook

Review of Time Series Traffic Forecasting Methods

Big Data in Forecasting Research: A Literature Review

A survey on machine learning models for financial time series forecasting

Deep Learning for Time Series Forecasting: A Survey

Machine Learning for Spatiotemporal Sequence Forecasting: A Survey

Forecasting in social settings: The state of the art

A Survey on Diffusion Models for Time Series and Spatio-Temporal Data

Time Series Analysis Based on Informer Algorithms: A Survey

Exploring Progress in Multivariate Time Series Forecasting: Comprehensive Benchmarking and Heterogeneity Analysis

TFB: Towards Comprehensive and Fair Benchmarking of Time Series Forecasting Methods

A Survey of Deep Learning and Foundation Models for Time Series Forecasting

Spatio-Temporal Predictive Modeling Techniques for Different Domains: a Survey

Fully Automated Correlated Time Series Forecasting in Minutes

Challenges and approaches to time-series forecasting in data center telemetry: A Survey

Parallel Extraction of Long-term Trends and Short-term Fluctuation Framework for Multivariate Time Series Forecasting