Abstract:Background: Non-word repetition (NWR) tests are an important way speech and language therapists (SaLTs) assess language development. NWR tests are often scored whilst participants make their responses (i.e., in real time) in clinical and research reports (documented here via a secondary analysis of a published systematic review). Aims: The main aim was to determine the extent to which real-time coding of NWR stimuli at the whole-item level (as correct/incorrect) was predicted by models that had varying levels of detail provided from phonemic transcriptions using several linear mixed method (LMM) models. Methods & procedures: Live scores and recordings of responses on the universal non-word repetition (UNWR) test were available for 146 children aged between 3 and 6 years where the sample included all children starting in five UK schools in one year or two consecutive years. Transcriptions were made of responses to two-syllable NWR stimuli for all children and these were checked for reliability within and between transcribers. Signal detection analysis showed that consonants were missed when judgments were made live. Statistical comparisons of the discrepancies between target stimuli and transcriptions of children's responses were then made and these were regressed against live score accuracy. Six LMM models (three normalized: 1a, 2a, 3a; and three non-normalized: 1b, 2b, 3b) were examined to identify which model(s) best captured the data variance. Errors on consonants for live scores were determined by comparison with the transcriptions in the following ways (the dependent variables for each pair of models): (1) consonants alone; (2) substitutions, deletions and insertions of consonants identified after automatic alignment of live and transcribed materials; and (3) as with (2) but where substitutions were coded further as place, manner and voicing errors. Outcomes & results: The normalized model that coded consonants in non-words as 'incorrect' at the level of substitutions, deletions and insertions (2b) provided the best fit to the real-time coding responses in terms of marginal R2, Akaike's information criterion (AIC) and Bayesian information criterion (BIC) statistics. Conclusions & implications: Errors that occur on consonants when non-word stimuli are scored in real time are characterized solely by the substitution, deletion and insertion measure. It is important to know that such errors arise when real-time judgments are made because NWR tasks are used to assess and diagnose several cognitive-linguistic impairments. One broader implication of the results is that future work could automate the analysis procedures to provide the required information objectively and quickly without having to transcribe data. What this paper adds: What is already known on this subject Children and patients with a wide range of cognitive and language difficulties are less accurate relative to controls when they attempt to repeat non-words. Responses to non-words are often scored as correct or incorrect at the time the test is conducted. Limited assessments of this scoring procedure have been conducted to date. What this study adds to the existing knowledge Live NWR scores made by 146 children were available and the accuracy of these judgements was assessed here against ones based on phonemic transcriptions. Signal detection analyses showed that live scoring missed consonant errors in children's responses. Further analyses, using linear mixed effect models, showed that live judgments led to consonant substitution, deletion and insertion errors. What are the practical and clinical implications of this work? Improved and practicable NWR scoring procedures are required to provide SaLTs with better indications about children's language development (typical and atypical) and for clinical assessments of older people. The procedures currently used miss substitutions, deletions and insertions. Hence, procedures are required that provide the information currently only available when materials are transcribed manually. The possibility of training automatic speech recognizers to provide this level of detail is raised.

Prior knowledge and other individual differences in the development of accuracy over the time-course of a pre-test/treatment/post-test study of instructed second language acquisition

Prior Knowledge Predicts Early Consolidation in Second Language Learning

The Influence of Task Complexity, Task Difficulty and Proficiency Level on Chinese Students' Accuracy in Performance

Processing Instruction Versus Traditional Instruction: The Transfer-of-Training Effects on Chinese EFL Learners

The effects of proactive form-focused instruction and individual differences on second language acquisition

Language aptitude and its relationship to instructional effectiveness in second language acquisition

The interplay of individual differences and context of learning in behavioral and neurocognitive second language development

Learning as a function of coordination bias: building upon pre-practice behaviours

Individual differences diminish the pretest effect under productive memory conditions

Energy conservation theory for second language acquisition (ECT-L2A): a partial validation of kinetic energy–aptitude and motivation

Critical period effects in second language learning: The influence of maturational state on the acquisition of English as a second language

Beyond Prior and Volatility: The Distinct Iterative Prior Account in ASD

Error-related activity and correlates of grammatical plasticity

Speed and Accuracy Training Affects Young Children's Cognitive Control

How Does Prior Knowledge Influence Eye Fixations and Sequences of Cognitive and Metacognitive SRL Processes during Learning with an Intelligent Tutoring System?

Speaking proficiency development in EFL classrooms: measuring the differential effect of TBLT and PPP teaching approaches

Path and rate of development in child heritage speakers: Evidence from Greek subject/object form and placement

Factors affecting judgment accuracy when scoring children's responses to non-word repetition stimuli in real time

The Effects of the Timing of Corrective Feedback on the Acquisition of A New Linguistic Structure

How and with what accuracy do children report self-regulated learning in contemporary EFL instructional settings?

The accuracy of student self-assessments of English-Chinese bidirectional interpretation: a longitudinal quantitative study