Abstract:Delivering precise point and distributional forecasts across a spectrum of prediction horizons represents a significant and enduring challenge in the application of time-series forecasting within various industries. Prior research on developing deep learning models for time-series forecasting has often concentrated on isolated aspects, such as long-term point forecasting or short-term probabilistic estimations. This narrow focus may result in skewed methodological choices and hinder the adaptability of these models to uncharted scenarios. While there is a rising trend in developing universal forecasting models, a thorough understanding of their advantages and drawbacks, especially regarding essential forecasting needs like point and distributional forecasts across short and long horizons, is still lacking. In this paper, we present ProbTS, a benchmark tool designed as a unified platform to evaluate these fundamental forecasting needs and to conduct a rigorous comparative analysis of numerous cutting-edge studies from recent years. We dissect the distinctive data characteristics arising from disparate forecasting requirements and elucidate how these characteristics can skew methodological preferences in typical research trajectories, which often fail to fully accommodate essential forecasting needs. Building on this, we examine the latest models for universal time-series forecasting and discover that our analyses of methodological strengths and weaknesses are also applicable to these universal models. Finally, we outline the limitations inherent in current research and underscore several avenues for future exploration.

A Distribution-Free Method for Probabilistic Prediction

DistPred: A Distribution-Free Probabilistic Inference Method for Regression and Forecasting

Constructing Probabilistic Load Forecast From Multiple Point Forecasts: A Bootstrap Based Approach

Distribution-Free Predictive Inference under Unknown Temporal Drift

Ensemble Multi-Quantiles: Adaptively Flexible Distribution Prediction for Uncertainty Quantification

Binary Search with Distributional Predictions

Distribution-Free Probability Density Forecast Through Deep Neural Networks

Distribution-free risk assessment of regression-based machine learning algorithms

Regression via Arbitrary Quantile Modeling

An Adaptive Weight Ensemble Learning Method for Probabilistic Wind Power Prediction

Distribution-in-distribution-out Regression

ProbTS: Benchmarking Point and Distributional Forecasting across Diverse Prediction Horizons

Model Free Prediction with Uncertainty Assessment

Distribution-Free One-Pass Learning

A Distributional Perspective on Remaining Useful Life Prediction With Deep Learning and Quantile Regression

Distributionally Robust Performative Prediction

Cumulative Probability Distribution Model for Evaluating User Behavior Prediction Algorithms

A conformal prediction inspired approach for distribution regression with random Fourier features

Distributional Gradient Boosting Machines

A Constrained Bayesian Approach to Out-of-distribution Prediction.

Invariant Probabilistic Prediction