Item Response Theory: A Modern Measurement Approach to Reliability and Precision for Counseling Researchers
Ryan M. Cook,Stefanie A. Wind
DOI: https://doi.org/10.1080/07481756.2023.2301284
2024-03-02
Measurement and Evaluation in Counseling and Development
Abstract:The purpose of this article is to discuss reliability and precision through the lens of a modern measurement approach, item response theory (IRT). Reliability evidence in the field of counseling is primarily generated using Classical Test Theory (CTT) approaches, although recent studies in the field of counseling have shown the benefits of using IRT approaches to explore measurement precision. We discuss the theoretical foundations and assumptions of CTT and IRT, and examine how modern measurement theory (i.e. IRT) poses advantages for capturing measurement precision. We use an example analysis to demonstrate the indices of measurement precision in IRT approaches (e.g. standard errors, person-fit, item-fit, targeting, residual-based fit statistics). Finally, we discuss practical and clinical implications for counseling researchers for using IRT approaches to measure precision, including insights into precision for persons, items, and measurement invariance as well as the utility of brief and adaptive scales. Using modern measurement theory approaches (i.e. item response theory), evidence of precision is generated for persons, items, and other relevant facets, and these indices can inform needed scale revisions, identify for whom the theorized model is a good fit, and the items that are well-targeted for the population of interest.
psychology, educational, applied