Comprehensive Evaluation of Decision Performance

X. Rong Li,Zhansheng Duan
2008-01-01
Abstract:Decision performance metrics abound, but they all have three fundamental limitations: (a) none are unified for diverse situations including, in particular, the case with mis-specified truth; (b) none are comprehensive; and (c) they all rely on perfect knowledge of the ground truth. In this paper, we present a unified metric that measures decision performance comprehensively without the need to know the ground truth. It is truly a distance metric and enjoys many nice properties.
What problem does this paper attempt to address?