Evaluation of Metrics for Assessing Dipolar Climate Patterns in Climate Models

Sandro F. Veiga,Huiling Yuan

DOI: https://doi.org/10.1007/s00382-024-07220-3

IF: 4.901

2024-01-01

Climate Dynamics

Abstract:In climate model assessment, one of the most widely used procedures is to evaluate the large-scale spatial patterns simulated by models. In this study, we evaluated four non-complex metrics for assessing dipolar and multipolar climate patterns, aiming to ascertain their strengths and possible caveats. Three established metrics are employed: the Taylor skill (TS) score, the Arcsin-Mielke measure M (measure M), and the Spatial Efficiency (SPAEF) metric. Additionally, a fourth metric is introduced by adjusting the TS score (TSadj score), where the standard deviation ratio is substituted with the spatial root-mean-square error (RMSE). By applying these metrics to measure and rank the performance of six CMIP6 models in simulating the dipolar patterns of the East Asian Summer Monsoon and Atlantic Meridional Mode, as well as the quadripolar pattern of the Pacific-North American pattern, the results show that metrics considering spatial error (RMSE/MSE), such as the TSadj score and measure M, offer a more accurate assessment compared to metrics relying on variance comparison of the patterns (such as the TS score and SPAEF metric) since they account for the patterns’ spatial variance distribution. Furthermore, the results provided by the established metrics might not effectively assess the quality of the models’ simulation. Therefore, the TSadj score can be quantified using a threshold set at half of its maximum attainable value to identify well-performing models, corresponding to a minimum spatial correlation of 0.4 and a maximum normalized RMSE of 1. This modification of the TSadj score yields a more practical outcome for the assessment of models.