Abstract:Cognitive diagnostic assessment (CDA) has been developed rapidly to provide fine-grained diagnostic feedback on students' subskills and to provide insights on remedial instructions in specific domains. To date, most cognitive diagnostic studies on reading tests have focused on retrofitting a single booklet from a large-scale assessment (e.g., PISA and PIRLS). Critical issues in CDA involve the scarcity of research to develop diagnostic tests and the lack of reliability and validity evidence. This study explored the development and validation of the Diagnostic Chinese Reading Comprehension Assessment (DCRCA) for primary students under the CDA framework. Reading attributes were synthesized based on a literature review, the national curriculum criteria, the results of expert panel judgments, and student think-aloud protocols. Then, the tentative attributes were used to construct three booklets of reading comprehension items for 2-6 graders at three key stages. The assessment was administered to a large population of students (N = 21,466) in grades 2-6 from 20 schools in a district of Changchun City, China. Q-matrices were compared and refined using the model-data fit and an empirical validation procedure, and five representative cognitive diagnostic models (CDMs) were compared for optimal performance. The fit indices suggested that a six-attribute structure and the G-DINA model were best fitted for the reading comprehension assessment. In addition, diagnostic reliability, construct, internal and external validity results were provided, supporting CDM classifications as reliable, accurate, and useful. Such diagnostic information could be utilized by students, teachers, and administrators of reading programs and instructions.

Cognitive Diagnosis in Language Assessment: A Thematic Review

Validating a Reading Assessment Within the Cognitive Diagnostic Assessment Framework: Q-Matrix Construction and Model Comparisons for Different Primary Grades

Cognitive diagnostic approach — A new stage of language testing research

Diagnosing Writing Ability Using China’s Standards of English Language Ability: Application of Cognitive Diagnosis Models

Towards a New Generation of Cognitive Diagnosis

Developing individualized feedback for listening assessment: Combining standard setting and cognitive diagnostic assessment approaches

Examining the L2 reading comprehension ability of adult ELLs: Developing a diagnostic test within the cognitive diagnostic assessment framework

Towards Personalized English Learning Diagnosis: Cognitive Diagnostic Modelling for EFL Listening

Cognitive Diagnostic Assessment of IELTS Listening: Providing Feedback from its Internal Structure

Exploring ways to provide diagnostic feedback with an ESL placement test: Cognitive diagnostic assessment of L2 reading ability

Assessing Second Language Listening over the Past Twenty Years: A Review Within the Socio-Cognitive Framework

Assessing Students' Translation Competence: Integrating China's Standards of English with Cognitive Diagnostic Assessment Approaches

Diagnosing listening and reading skills in the Chinese EFL context: Performance stability and variability across modalities and performance levels

New development of cognitive diagnosis models

Assessing Students’ Translation Competence: Integrating China’s Standards of English With Cognitive Diagnostic Assessment Approaches

A Survey of Models for Cognitive Diagnosis: New Developments and Future Directions

Test Construction for Cognitive Diagnosis

Diagnosing Primary Students’ Reading Progression: is Cognitive Diagnostic Computerized Adaptive Testing the Way Forward?

The Development of Computerized Adaptive Testing with Cognitive Diagnosis for an English Achievement Test in China

Application of Bi-factor MIRT and Higher-order CDM Models to an In-house EFL Listening Test for Diagnostic Purposes

Characteristics of Cognitive Diagnosis Comparing with the Standard Test Theory