Abstract:PurposeThis paper explores how well raters’ self-judgments of their rating and behavioral accuracy reflect reality. We also explore the role of motivation in determining the quality of raters’ self-judgments and their rating and behavioral accuracy.Design/Methodology/ApproachIn this study, 327 participants watched two videotaped lecturers and provided ratings and behavioral identifications, along with self-judgments of the accuracy of these ratings and behavioral identifications and self-reported motivation. Three types of monetary incentives were also offered.FindingsRaters’ self-judgments were related to how well they identified behaviors (i.e., behavioral accuracy) and the type of rating accuracy arguably most closely associated with behaviors (i.e., differential accuracy (DA)). Although more motivated raters believed that they provided both more accurate ratings and behavioral identifications, this was only the case for behavioral identifications. Financial incentives based on the quality of self-judgments only influenced whether self-judgments related to rating accuracy (DA).ImplicationsIf organizations are interested in improving the quality of behavioral feedback or assessment practices relying on ratings, it may be useful to ask raters/interviewers to what extent they believe that they correctly identified behaviors or gave accurate ratings, especially if the raters/interviewers are motivated to be accurate.Originality/ValueThis is one of the first studies to show that raters’ self-judgments can be useful indicators of whether they accurately recognized behavior and some kinds of accuracy. Furthermore, this is the first study to show that more motivated raters provide both more accurate self-judgments of behavioral accuracy and more accurate behavioral identifications.

Self-Judgments of Accuracy as Indicators of Performance Evaluation Quality: Should We Believe Them?