Measuring and modeling physics students' conceptual knowledge structures through term association times

Ian D. Beatty,William J. Gerace,Robert J. Dufresne

DOI: https://doi.org/10.48550/arXiv.physics/0702111

2007-02-14

Abstract:Traditional problem-based exams are not efficient instruments for assessing the "structure" of physics students' conceptual knowledge or for providing diagnostically detailed feedback to students and teachers. We present the Free Term Entry task, a candidate assessment instrument for exploring the connections between concepts in a student's understanding of a subject. In this task, a student is given a general topic area and asked to respond with as many terms from the topic area as possible in a given time; the "thinking time" between each term-entry event is recorded along with the response terms. The task was given to students from two different introductory physics classes. Response term thinking times were found to correlate with the strength of the association between two concepts. In addition, sets of thinking times from the task show distinct, characteristic patterns which might prove valuable for student assessment. We propose a quantitative dynamical model named the Matrix Walk Model which is able to match many aspects of the observed data. One particular feature of the data -- a distinct "spike" superimposed on the otherwise log-normal distribution of most thinking time sets -- has not been fit. The spike, other patterns observed in the data, and the proposed phenomenological model could all benefit from a grounding in cognitive theory.

Physics Education

What problem does this paper attempt to address?

### What problems does this paper attempt to solve?

This paper aims to address the deficiencies of traditional problem - based examinations in evaluating the Conceptual Knowledge Structure (CKS) of physics students. Specifically, the author believes that traditional examinations are unable to effectively assess students' conceptual knowledge structures and cannot provide detailed diagnostic feedback for students and teachers. To make up for this deficiency, the paper proposes a new assessment tool - the **Free Term Entry task (FTE)**, and verifies its effectiveness through experimental data and modeling.

#### Main problems include:

1. **Limitations of traditional assessment methods**:
- Traditional examinations mainly focus on whether students can solve problems, but cannot reveal why students can or cannot solve problems.
- Students may fail to solve problems correctly for various reasons, such as misinterpreting the problem situation, lacking physical intuition, not knowing the necessary principles, or making mistakes when applying the correct principles.
- Therefore, traditional assessment methods cannot provide enough details to guide students' learning and teachers' teaching interventions.

2. **New methods for exploring Conceptual Knowledge Structure (CKS)**:
- The paper proposes the FTE task, which allows students to freely associate and input related terms under a specific topic while recording the thinking time for each term.
- By analyzing these terms and their thinking time, the relationships and organization of students' concepts can be revealed.

3. **Constructing a quantitative model to explain experimental data**:
- The paper proposes a quantitative dynamic model named **Matrix Walk Model** to explain the data characteristics observed in the FTE task.
- The model can partially reproduce the characteristics in the experimental data, such as the log - normal distribution of thinking time and the peak phenomenon in it.

4. **Application of cognitive theory**:
- The paper emphasizes the need to combine experimental data and models with cognitive theory to better understand and explain students' conceptual knowledge structures.

### Summary

The core problem of this paper is to develop a more effective assessment tool for directly evaluating the conceptual knowledge structure of physics students and verify its effectiveness through empirical research and modeling. By introducing the FTE task and the Matrix Walk Model, the author hopes to provide a new assessment method for educational research and practice, thereby better guiding teaching and learning.