Bound on forecasting skill for models of North Atlantic tropical cyclone counts

Daniel Wesley,Michael E. Mann,Bhuvnesh Jain,Colin R. Twomey,Shannon Christiansen
2024-10-08
Abstract:Annual North Atlantic tropical cyclone (TC) counts are frequently modeled as a Poisson process with a state-dependent rate. We provide a lower bound on the forecasting error of this class of models. Remarkably we find that this bound is already saturated by a simple linear model that explains roughly 50 percent of the annual variance using three climate indices: El Niño Southern Oscillation (ENSO), average sea surface temperature (SST) in the main development region (MDR) of the North Atlantic and the North Atlantic oscillation (NAO) atmospheric circulation index (Kozar et al 2012). As expected under the bound, increased model complexity does not help: we demonstrate that allowing for quadratic and interaction terms, or using an Elastic Net to forecast TC counts using global SST maps, produces no detectable increase in skill. We provide evidence that observed TC counts are consistent with a Poisson process, limiting possible improvements in TC modeling by relaxing the Poisson assumption.
Atmospheric and Oceanic Physics,Data Analysis, Statistics and Probability
What problem does this paper attempt to address?
The problem that this paper attempts to solve is about the accuracy limit of the annual count forecast of tropical cyclones (TC) in the North Atlantic. Specifically, the paper explores whether there is a lower limit of model forecast error under the assumption that TC count follows a Poisson process, and whether this lower limit has been reached by the existing simple linear models. Through analysis, the author finds that the current model based on three climate indices (El Niño/Southern Oscillation (ENSO), the mean sea - surface temperature (SST) in the Main Development Region (MDR) of the North Atlantic, and the North Atlantic Oscillation (NAO) atmospheric circulation index) has reached this error lower limit. In addition, the paper also tests whether increasing the model complexity (such as introducing quadratic terms and interaction terms, using the Elastic Net method, etc.) will improve the forecast skill, and the results show that these complex models do not significantly improve the forecast effect. This indicates that, under the premise of maintaining the Poisson distribution assumption, the existing models have reached the theoretical lower limit of forecast error. ### Main research contents: 1. **The lower limit of forecast error in the Poisson regression framework**: - The paper derives the statistical lower limit of forecast error in the Poisson regression model. - Through theoretical analysis, a formula is proposed to calculate this lower limit. 2. **Performance evaluation of existing models**: - The model proposed by Kozar et al. (2012), which uses three climate indices of ENSO, MDR SST, and NAO as predictors, is evaluated. - The results show that this model has reached the theoretical lower limit of forecast error. 3. **Testing of complex models**: - Forecasts are made based on the global sea - surface temperature (SST) map using the Elastic Net method. - Quadratic terms and interaction terms are introduced to try to improve the model's forecast skill. - The results show that these complex models do not significantly improve the forecast effect. 4. **Verification of the Poisson distribution assumption**: - Methods such as the chi - square test are used to verify whether the observed TC count data conform to the Poisson distribution. - The results support the assumption that the observed data conform to an independent Poisson distribution. ### Conclusions: - The existing models based on the Poisson distribution assumption have reached the theoretical lower limit of forecast error. - Increasing the model complexity cannot significantly improve the forecast skill. - Further improvement of the forecast model may require abandoning the Poisson distribution assumption and exploring other more complex statistical models or methods. ### Formula presentation: - **Probability mass function of Poisson distribution**: \[ P(y_t)=\frac{\lambda_t^{y_t}}{y_t!}e^{-\lambda_t} \] - **Mean parameter in Poisson regression**: \[ \lambda_t = \exp(\beta_0+\beta_1x_{1t}+\beta_2x_{2t}+\cdots+\beta_p x_{pt}) \] - **Minimum absolute error (MAE)**: \[ E=\frac{1}{n}\sum_{t = 1}^n|y_t-\hat{y}_t| \] - **Lower limit of forecast error**: \[ b(z)=\min_\lambda\sum_{y\geq0}|y - z|P(y|\lambda) \] - **Minimum expected MAE**: \[ B=\frac{1}{n}\sum_{t = 1}^n b(y_t) \] These formulas and analysis results jointly support the main conclusion of the paper, that is, under the Poisson distribution assumption, the existing models have reached the theoretical lower limit of forecast error.