Abstract:Time series are generated in diverse domains such as economic, traffic, health, and energy, where forecasting of future values has numerous important applications. Not surprisingly, many forecasting methods are being proposed. To ensure progress, it is essential to be able to study and compare such methods empirically in a comprehensive and reliable manner. To achieve this, we propose TFB, an automated benchmark for Time Series Forecasting (TSF) methods. TFB advances the state-of-the-art by addressing shortcomings related to datasets, comparison methods, and evaluation pipelines: 1) insufficient coverage of data domains, 2) stereotype bias against traditional methods, and 3) inconsistent and inflexible pipelines. To achieve better domain coverage, we include datasets from 10 different domains: traffic, electricity, energy, the environment, nature, economic, stock markets, banking, health, and the web. We also provide a time series characterization to ensure that the selected datasets are comprehensive. To remove biases against some methods, we include a diverse range of methods, including statistical learning, machine learning, and deep learning methods, and we also support a variety of evaluation strategies and metrics to ensure a more comprehensive evaluations of different methods. To support the integration of different methods into the benchmark and enable fair comparisons, TFB features a flexible and scalable pipeline that eliminates biases. Next, we employ TFB to perform a thorough evaluation of 21 Univariate Time Series Forecasting (UTSF) methods on 8,068 univariate time series and 14 Multivariate Time Series Forecasting (MTSF) methods on 25 datasets. The benchmark code and data are available at <a class="link-external link-https" href="https://github.com/decisionintelligence/TFB" rel="external noopener nofollow">this https URL</a>.

A Comparative Study of the Forecasting Performance of Three International Organizations

Forecasting Bull and Bear Markets: Evidence from China

Comparative study on forecasting methods of major medical equipments in China:taking CT as an example

An evaluation of the inflation forecasting performance of the European Central Bank, the Federal Reserve, and the Bank of England

The Second Forecast Based On The Combined Models (Sfbcm) (Id : 6-113)

Econometric Modeling and Economic Forecasting

Overprecision in the Survey of Professional Forecasters

Forecasting in International Relations and Foreign Policy Studies

Is Single Forecast Method Better than Combined Forecast Method?

Economic System Forecasting Based on Temporal Fusion Transformers: Multi-dimensional Evaluation and Cross-Model Comparative Analysis

ProbTS: Benchmarking Point and Distributional Forecasting across Diverse Prediction Horizons

Investigation of Inflation Forecasting

Equal predictive ability tests based on panel data with applications to OECD and IMF forecasts

TFB: Towards Comprehensive and Fair Benchmarking of Time Series Forecasting Methods

"Combine to compete: Improving fiscal forecast accuracy over time"

ForecastBench: A Dynamic Benchmark of AI Forecasting Capabilities

Assessing the accuracy of directional forecasts

Empirical Quantitative Analysis of COVID-19 Forecasting Models

Uncertainty and macroeconomic forecasts: Evidence from survey data

Tourism forecasting: To combine or not to combine?