Abstract:Translation and paraphrasing, as typical forms in second language (L2) communication, have been considered effective learning methods in second language acquisition (SLA). While many studies have investigated their similarities and differences in a process-oriented approach, little attention has been paid to the correlation in product quality between them, probably due to difficulties in assessing the quality of translation and paraphrasing. Current quality evaluation methods tend to be either subjective and one-sided or lack consistency and standardization. To address these limitations, we proposed preliminary evaluation frameworks for translation and paraphrasing by incorporating indices from natural language processing (NLP) tools into teachers' rating rubrics and further compared the product quality of the two activities. Twenty-nine translators were recruited to perform a translation task (translating from Chinese to English) and a paraphrasing task (paraphrasing in English). Their output products were recorded by key-logging technique and graded by three professional translation teachers by using a 10-point Likert Scale. This rating process adopted rubrics consisting of both holistic and analytical assessments. Besides, indices containing textual features from lexical and syntactic levels were extracted from TAASSC and TAALES. We identified indices that effectively predicted product quality using Pearson's correlation analysis and combined them with expert evaluation rubrics to establish NLP-assisted evaluation frameworks for translation and paraphrasing. With the help of these frameworks, we found a closely related performance between the two tasks, evidenced by several shared predictive indices in lexical sophistication and strong positive correlations between translated and paraphrased text quality according to all the rating metrics. These similarities suggest a shared language competence and mental strategies in different types of translation activities and perhaps in other forms of language tasks. Meanwhile, we also observed differences in the most salient textual features between translations and paraphrases, mainly due to the different processing costs required by the two tasks. These findings enrich our understanding of the shared ground and divergences in product quality between translation and paraphrasing and shed light on the pedagogical application of translation activities in classroom teaching. Moreover, the proposed evaluation framework can also bring insights into the development of standardized evaluation frameworks in translation and paraphrasing in the future.

Calibrated Parsing Items Evaluation: a step towards objectifying the translation assessment

Item-based assessment of translation competence: Chimera of objectivity versus prospect of reliable measurement

Coh-Metrix Model-Based Automatic Assessment of Interpreting Quality

A Corpus-Based Evaluation Approach to Translation Improvement

Translation Quality Evaluation

Framework and Resources for Natural Language Parser Evaluation

Parsisanj: a semi-automatic component-based approach towards search engine evaluation

Evaluating the evaluator: A novel perspective on translation quality

Translation Canvas: An Explainable Interface to Pinpoint and Analyze Translation Systems

Joss Moorkens, Sheila Castilho, Federico Gaspari, Stephen Doherty (eds): Translation quality assessment: from principles to practice

Assessing Students’ Translation Competence: Integrating China’s Standards of English With Cognitive Diagnostic Assessment Approaches

Evaluation Reportof the Third Chinese Parsing Evaluation: CIPS-SIGHAN-ParsEval-2012

An Evaluation of Translation Quality Assessment Course: Voices from Instructors

HOPE: A Task-Oriented and Human-Centric Evaluation Framework Using Professional Post-Editing Towards More Effective MT Evaluation

Quantitative Evaluation of Alternative Translations in a Corpus of Highly Dissimilar Finnish Paraphrases

Evaluation Reportof the fourth Chinese Parsing Evaluation: CIPS-SIGHAN-ParsEval-2014

Comparing product quality between translation and paraphrasing: Using NLP-assisted evaluation frameworks

CEFR-Based Sentence Difficulty Annotation and Assessment

Evaluative and Reflective Study of Translation Learning Design: An Instructional Analysis

A Large-Scale Evaluation of Pre-editing Strategies for Improving User-Generated Content Translation

Exploring the effectiveness of ChatGPT-based feedback compared with teacher feedback and self-feedback: Evidence from Chinese to English translation