Abstract:Our situation: Classical test theory (CTT) and item response theory (IRT) are two measurement models used to evaluate results from examinations, questionnaires, and instruments. To illustrate the benefits of IRT, we compared how results from multiple-choice tests can be interpreted using CTT and IRT. Methodological literature review: IRT encompasses a collection of statistical models that estimate the probability of providing a correct response for a test item. The models are non-linear and generate item characteristic curves that illustrate the relationship between the examinee's ability level and whether they answered the item correctly. Several models can be used to estimate parameters such as item difficulty, discrimination, and guessing. In addition, IRT can generate item and test information functions to illustrate the accuracy of ability estimates. Our recommendations and their applications: Researchers interested in IRT should gather the necessary resources early in the research process and collaborate with those experienced in quantitative and advanced statistical models. Researchers should confirm IRT is the optimal choice and select the model ideal for their needs. Once data are acquired, confirm model assumptions are met and model fit is appropriate. Lastly, researchers should consider disseminating the findings with accompanying visuals. Potential impact: IRT can be a valuable approach in assessment design and evaluation. Potential opportunities include supporting the design of computer adaptive tests, creating equivalent test forms that evaluate a range of examinee abilities, and evaluating whether items perform differently for examinee sub-groups. Further, IRT can have noteworthy visuals such as test information and functions.

Approaches to Item-Level Data with Cross-Classified Structure: An Illustration with Student Evaluation of Teaching

Modeling Item-Level Heterogeneous Treatment Effects With the Explanatory Item Response Model: Leveraging Large-Scale Online Assessments to Pinpoint the Impact of Educational Interventions

Modeling Response Styles in Cross‐Classified Data Using a Cross‐Classified Multidimensional Nominal Response Model

Item Response Theory -- A Statistical Framework for Educational and Psychological Measurement

Evaluation of Student Performance through a Multidimensional Finite Mixture IRT Model

Item response theory in high-stakes pharmacy assessments

Estimating Heterogeneous Treatment Effects with Item-Level Outcome Data: Insights from Item Response Theory

Exploring Variability Sources in Student Evaluation of Teaching via Many-Facet Rasch Model

A multilevel finite mixture item response model to cluster examinees and schools

Investigating the reliability of CET-SET using Multi-Facet Rasch Model

Evaluation of student proficiency through a multidimensional finite mixture IRT model

An introduction to mixture item response theory models

Investigating Heterogeneity in Response Strategies: A Mixture Multidimensional IRTree Approach

An item response theory evaluation of the Light and Spectroscopy Concept Inventory national data set

Applying Unidimensional and Multidimensional Item Response Theory Models in Testlet-Based Reading Assessment

Exploring the Correlation Between Multiple Latent Variables and Covariates in Hierarchical Data Based on the Multilevel Multidimensional IRT Model

A structured regression approach for evaluating model performance across intersectional subgroups

2PL Model: Compare Generalized Linear Mixed Model with Latent Variable Model based IRT framework

Scalable Learning of Item Response Theory Models

Evaluation of covariate effects in item response theory models

Comparing Attitudes Across Groups: An IRT-Based Item-Fit Statistic for the Analysis of Measurement Invariance