Developing Summary Content Scoring Criteria for University L2 Writing Instruction in Japan

Yasuyo Sawaki

DOI: https://doi.org/10.1007/978-981-15-8952-2_10

2020-01-01

Abstract:This study examined theSummary content scoringfunctioningScoring criteria of two types of summary content scoringSummary content scoring methods (content point scores and a holistic summary content rating scale called Integration) for low- to medium-stakes uses in university L2 academic writing instruction in Japan from the perspective of Bachman and Palmer’s (Language assessment in practice, Oxford University Press, Oxford, 2010) assessment use argumentAssessment Use Argument (AUA) (AUA) framework. Results of a multivariate generalizability theoryMultivariate generalizability theory analysis of summaries written by 130 Japanese university students suggested a satisfactory level of score dependabilityDependability of the Integration rating scale for the intended uses, supporting the rating consistency warrant for the assessment record claim in the AUAAssessment Use Argument (AUA), while the evidence concerning the score dependabilityDependability of the content point scores was mixed. Meanwhile, summary content scores based on both scoring methods were distinct from a language quality score. This suggests that employing either one with the language quality rating would enhance the representation of the summary writingSummary writingconstructConstruct, supporting the meaningfulness warrant for the test score interpretation claim in the AUAAssessment Use Argument (AUA).