Enhancing Time Series Predictors with Generalized Extreme Value Loss
Mi Zhang,Daizong Ding,Xudong Pan,Min Yang
DOI: https://doi.org/10.1109/tkde.2021.3108831
IF: 9.235
2021-01-01
IEEE Transactions on Knowledge and Data Engineering
Abstract:Time series prediction has wide applications in many safety-critical scenarios, including meteorology and finance. According to previous studies, time series of recorded events, e.g., river level and stock price, usually contain a non-trivial proportion of extreme events (e.g., flood and financial crisis), which are featured with extremely large/small values, occur in time series data with a relatively low frequency, and may have huge societal consequences if overlooked by a predictive model (i.e., predictor). Despite its significance in time series, we however observe the conventional square loss in time series prediction would ignore the modeling of extreme events. Specifically, we prove the square loss as a learning objective of the predictor behaves equivalently as a Gaussian kernel density estimator (KDE) on the recorded events, which is light-tailed itself and unable to model the ground-truth event distribution, usually heavy-tailed due to the existence of extreme events. Considering the benefits of forecasting extreme events, we propose a unified loss form called Generalized Extreme Value Loss (GEVL), which bridges the misalignment between the tail parts of the estimation and the ground-truth via transformations on either the observed events or the estimator. Following the proposed framework, we present three heavy-tailed kernels, i.e., shifted Gaussian, Gumbel and Frechet kernels, and derive the corresponding GEVLs which show different levels of trade-off between modeling effectiveness and computational resources, suitable for various downstream tasks. Comprehensive experiments on a diverse set of time series predictors and real-world datasets validate that, our novel loss form substantially enhances representative time series predictors in modeling extreme events. For example, for CO2 concentration rate prediction and stock price prediction, our proposed Frechet GEVL respectively reduces the RMSE of 6 representative DNN-based time series predictors on extreme events by over 20% and 17% on average, with a maximum reduction of 29:8%.