Hierarchical-Model Insights for Planning and Interpreting Individual-Difference Studies of Cognitive Abilities

Jeffrey N. Rouder,Mahbod Mehrvarz
DOI: https://doi.org/10.1177/09637214231220923
IF: 7.867
2024-02-25
Current Directions in Psychological Science
Abstract:Current Directions in Psychological Science, Ahead of Print. Although individual-difference studies have been invaluable in several domains of psychology, there has been less success in cognitive domains using experimental tasks. The problem is often called one of reliability: Individual differences in cognitive tasks, especially cognitive-control tasks, seem too unreliable. In this article, we use the language of hierarchical models to define a novel reliability measure—a signal-to-noise ratio—that reflects the nature of tasks alone without recourse to sample sizes. Signal-to-noise reliability may be used to plan appropriately powered studies as well as understand the cause of low correlations across tasks should they occur. Although signal-to-noise reliability is motivated by hierarchical models, it may be estimated from a simple calculation using straightforward summary statistics.
psychology, multidisciplinary
What problem does this paper attempt to address?