Abstract:We introduce the test difficulty concept from classical test theory to tackle the issue of low predictive power of implicit association tests (IATs). Following classical test theory, we argue that IATs of moderate difficulty (defined as mean IAT scores of zero) have more predictive power than IATs of extreme difficulties (defined as mean IAT scores deviating strongly from zero). Furthermore, we assume this relationship to be mediated by the true-score variance in IAT scores, with moderate difficulty resulting in more true-score variance. To test our hypotheses, we used nonexperimental (Studies 1 and 2) and experimental designs (Study 3). In Studies 1 and 2, we compared IATs of different test difficulties with regard to their ability to predict direct attitude measures, drawing on the Attitudes, Identities, and Individual Differences study. In Study 1, a subset of 95 attitude IATs ( n = 127,259) was analyzed using multilevel structural equation models. As expected, IAT test difficulty strongly moderated the predictive power of IATs, and this effect was mediated by IAT true-score variance. In Study 2, we replicated the results with the same analyses but a different subset of 95 identity IATs ( n = 43,745). In Study 3, we experimentally manipulated the IAT test difficulty. In total, three IATs ( n = 480) were analyzed using multigroup structural equation models. Again, the IAT closer to moderate difficulty had more true-score variance and predictive power than the IATs of extreme difficulty. Accordingly, for correlational research, we recommend developing moderately difficult IATs to maximize IAT true-score variance and provide suggestions on how to achieve that. (PsycInfo Database Record (c) 2024 APA, all rights reserved)

Progress of Study on Implicit Association Test

Underlying Processes in the Implicit Association Test(IAT):A Review of the Literature

Measuring individual differences in implicit cognition: The implicit association test.

Understanding and using the Implicit Association Test: I. An improved scoring algorithm.

A Review on the New Variants of Implicit Association Test: Based on the Normative Analysis

The Re-examination on the Stability of the Implicit Association Test

Improving the reliability and validity of the IAT with a dynamic model driven by similarity

Beyond Reaction Times: Incorporating Mouse-Tracking Measures into the Implicit Association Test to Examine Its Underlying Process

A Study of the Predictive Validity of Explicit and Implicit Self-esteem for Subjective Well-being of University Students

A Research on the Relationship between Aggressiveness Construct and Self-esteem

Gender Identity,Self-esteem and Gender-based Ingroup Bias of College Students

Reproducibility of Implicit Association Test (IAT) -- Case study of meta-analysis of racial bias research claims

Understanding and using the Implicit Association Test: III. Meta-analysis of predictive validity.

A Study on Implicit Attribution Based on Supraliminal Priming and Projective Methods

The Implicit Association Test and its difficulty(ies): Introducing the test difficulty concept to increase the true-score variance and, consequently, the predictive power of implicit association tests.

Conflict Monitoring and Stimulus Categorization Processes Involved in the Prosocial Attitude Implicit Association Test: Evidence from Event-Related Potentials.

An Implicit Association Test for Assessing Athlete Burnout

A paper-format group performance test for measuring the implicit association of target concepts

A Study on Implicit Altruistic Behavior

A Diffusion Model Approach for Understanding the Impact of 17 Interventions on the Race Implicit Association Test

The Relationship between Implicit Self-esteem and Explicit Self-esteem: from a Perspective of Multiple Implicit Measures