Abstract:Educational testing intends to measure person's ability, and tests are designed for various objectives. While the ideal pure power tests and pure speed tests do not meet the practical needs in test administration, speeded tests become the primary option. The tradeoff between speed and accuracy is an important issue, especially in speeded tests, and being able to collect and analyze item response time, which is the amount of time an examinee devotes to a particular item until answering it correctly, provides an opportunity to investigate the relationship between response accuracy and response times. A few methods have been proposed to capture such relationship. This dissertation addresses the issue in the context of computer-based testing (CBT), where response times can be automatically recorded. An important component, time limits, in speeded tests is considered. The proposed mixture model incorporates both (potential) response accuracy and response times. In the spirit of speeded tests, it takes into account the possibility that some examinees are not potentially capable of solving a certain item, as in a pure power test, and decomposes response times into finite response times for those who are potentially capable and infinite response times for others. Time limits are represented in terms of censoring: some response times are censored because the problem solving procedures are terminated due to time limits. The proportion of examinees who are potentially capable of solving a particular item is modeled by a regular Item Response Theory (IRT) model. The parameter estimation method is first validated in extensive simulation studies, and the results of real data analysis indicate that the proposed mixture model outperforms the methods in which the impact of time limits is ignored.

Analyzing force concept inventory with item response theory

Exploring Pedagogical Content Knowledge of Physics Instructors and Teaching Assistants Using the Force Concept Inventory

Item Response Theory -- A Statistical Framework for Educational and Psychological Measurement

Replicating analyses of item response curves using data from the Force and Motion Conceptual Evaluation

Exploring pedagogical content knowledge of physics instructors using the force concept inventory

An Item Response Theory Model for Incorporating Response Times in Forced-Choice Measures

Interpreting gains and losses in conceptual test using Item Response Theory

Item response theory in high-stakes pharmacy assessments

An item response theory evaluation of the Light and Spectroscopy Concept Inventory national data set

Extending Modified Module Analysis to Include Correct Responses: An Analysis of the Force Concept Inventory

Applying Cognitive Diagnostic Models to Mechanics Concept Inventories

Applying Unidimensional and Multidimensional Item Response Theory Models in Testlet-Based Reading Assessment

Robust Measurement Via A Fused Latent and Graphical Item Response Theory Model

On the Validity of Forced Choice Scores Derived From the Thurstonian Item Response Theory Model

Comparing large lecture mechanics curricula using the Force Concept Inventory: A five thousand student study

Item response analysis of computational thinking practices: Test characteristics and students’ learning abilities in visual programming contexts

Contributions to the statistical analysis of item response time in educational testing

In-class vs. online administration of concept inventories and attitudinal assessments

Exploring the Structure of Misconceptions in the Force Concept Inventory with Modified Module Analysis

What are we assessing? An analysis of the most common concept inventories in physics

A Mixture Model for Random Responding Behavior in Forced-Choice Noncognitive Assessment: Implication and Application in Organizational Research