Zero-shot Cross-Lingual Transfer for Synthetic Data Generation in Grammatical Error Detection

Gaetan Lopez Latouche,Marc-André Carbonneau,Ben Swanson

2024-07-16

Abstract:Grammatical Error Detection (GED) methods rely heavily on human annotated error corpora. However, these annotations are unavailable in many low-resource languages. In this paper, we investigate GED in this context. Leveraging the zero-shot cross-lingual transfer capabilities of multilingual pre-trained language models, we train a model using data from a diverse set of languages to generate synthetic errors in other languages. These synthetic error corpora are then used to train a GED model. Specifically we propose a two-stage fine-tuning pipeline where the GED model is first fine-tuned on multilingual synthetic data from target languages followed by fine-tuning on human-annotated GED corpora from source languages. This approach outperforms current state-of-the-art annotation-free GED methods. We also analyse the errors produced by our method and other strong baselines, finding that our approach produces errors that are more diverse and more similar to human errors.

Computation and Language,Artificial Intelligence

What problem does this paper attempt to address?

The paper aims to address the issue of the lack of manually annotated data for Grammatical Error Detection (GED) in low-resource languages. Specifically: 1. **Research Background**: Existing GED methods heavily rely on manually annotated error corpora, but many low-resource languages lack these resources. 2. **Main Objective**: The paper proposes a novel Zero-shot Cross-lingual Transfer method, utilizing Multilingual Pre-trained Language Models (mPLMs) to generate synthetic error corpora and using this synthetic data to train GED models. 3. **Innovations**: - A two-stage fine-tuning process is proposed, where the GED model is first fine-tuned on multilingual synthetic data and then further fine-tuned on manually annotated GED corpora in the source language. - This method surpasses the current state-of-the-art unsupervised GED methods without the need for manually annotated data. - The paper also provides a detailed analysis of the types of errors generated by this method, finding that the error diversity is higher and more closely resembles real human errors. Through the above methods, the paper aims to provide an effective GED solution for low-resource languages that lack annotated data.

Zero-shot Cross-Lingual Transfer for Synthetic Data Generation in Grammatical Error Detection

A Unified Strategy for Multilingual Grammatical Error Correction with Pre-trained Cross-Lingual Language Model

Wronging a Right: Generating Better Errors to Improve Grammatical Error Detection

Synthetic Data Generation for Grammatical Error Correction with Tagged Corruption Models

A Simple Recipe for Multilingual Grammatical Error Correction

Grammatical Error Generation Based on Translated Fragments

Detecting Errors with Zero-Shot Learning

Multilingual Generative Language Models for Zero-Shot Cross-Lingual Event Argument Extraction

An Automatic Error Detection Method for Machine Translation Results Via Deep Learning

Towards Zero-Shot Multilingual Synthetic Question and Answer Generation for Cross-Lingual Reading Comprehension

Improving Grammatical Error Correction with Machine Translation Pairs.

Semi-supervised learning and bidirectional decoding for effective grammar correction in low-resource scenarios

Model and Data Transfer for Cross-Lingual Sequence Labelling in Zero-Resource Settings

Cross-lingual Transfer of Monolingual Models

TransGEC: Improving Grammatical Error Correction with Translationese

Efficient Grammatical Error Correction Via Multi-Task Training and Optimized Training Schedule

Who Wrote This? The Key to Zero-Shot LLM-Generated Text Detection Is GECScore

Judge a Sentence by Its Content to Generate Grammatical Errors

Improving Grammatical Error Correction via Contextual Data Augmentation

Byte-Level Grammatical Error Correction Using Synthetic and Curated Corpora

Adversarial Grammatical Error Generation: Application to Persian Language