Reliability (or Lack Thereof) of Smartphone-Delivered Visual Dot Probe Attention Bias Indices

Nur Hani Hani Zainal,Nicholas C. Jacobson
DOI: https://doi.org/10.31219/osf.io/gj4eq
2020-04-28
Abstract:Cognitive bias theories posit that generalized anxiety disorder (GAD) and social anxiety disorder (SAD) are entwined with attention bias (AB) toward threats, commonly indexed by faster response time (RT) on threat- congruent (vs. incongruent) trials on the visual dot probe (VDP). Moreover, although smartphone VDP assessments have been developed, their psychometric properties are understudied. This study thus aimed to assess the retest-reliability and internal reliability of 6 smartphone-delivered VDP AB and related indices in persons with and without GAD and SAD (Global ABI, Disengagement Effect, Facilitation Bias, AB Variability [ABV], Congruent RT, Incongruent RT). Participants (n = 819) completed a five-trial smartphone-delivered VDP for at least one session. Of these, 294, 151, 94 participants completed 2, 5, and 12 sessions, respectively, across 2 weeks. Across 5 trials per session, Global ABI, Disengagement Effect, and Facilitation Bias had low reliability estimates (mean intra-class correlation coefficient [ICC] = 0.01–0.51; split-half reliability [r] = 0.01–0.18). Conversely, retest-reliability and internal reliability were good for Congruent RT and Incongruent RT (mean ICCs = 0.72–0.97; r = 0.37–0.92). Across 2, 5, and 12 sessions, poor retest-reliability and internal reliability estimates were consistently observed for all 6 AB and related indices. Compared to non-anxious controls, Global ABI and Facilitation Effect were stronger for those with GAD (d = 0.16–0.21) and SAD (d = 0.20–0.25), with negligible-to-small effect sizes. Theoretical and clinical implications, alongside suggestions to improve the reliability of smartphone VDP apps, are discussed.
What problem does this paper attempt to address?