Assessing Verbal Eyewitness Confidence Statements Using Natural Language Processing

Rachel Leigh Greenspan,Alex Lyman,Paul Heaton
DOI: https://doi.org/10.1177/09567976241229028
IF: 10.172
2024-02-22
Psychological Science
Abstract:Psychological Science, Ahead of Print. After an eyewitness completes a lineup, officers are advised to ask witnesses how confident they are in their identification. Although researchers in the lab typically study eyewitness confidence numerically, confidence in the field is primarily gathered verbally. In the current study, we used a natural language-processing approach to develop an automated model to classify verbal eyewitness confidence statements. Across a variety of stimulus materials and witnessing conditions, our model correctly classified adult witnesses' (N = 4,541) level of confidence (i.e., high, medium, or low) 71% of the time. Confidence-accuracy calibration curves demonstrate that the model's confidence classification performs similarly in predicting eyewitness accuracy compared to witnesses' self-reported numeric confidence. Our model also furnishes a new metric, confidence entropy, that measures the vagueness of witnesses' confidence statements and provides independent information about eyewitness accuracy. These results have implications for how empirical scientists collect confidence data and how police interpret eyewitness confidence statements.
psychology, multidisciplinary
What problem does this paper attempt to address?