Abstract:In Grammatical Error Correction (GEC), it is crucial to ensure the user's comprehension of a reason for correction. Existing studies present tokens, examples, and hints as to the basis for correction but do not directly explain the reasons for corrections. Although methods that use Large Language Models (LLMs) to provide direct explanations in natural language have been proposed for various tasks, no such method exists for GEC. Generating explanations for GEC corrections involves aligning input and output tokens, identifying correction points, and presenting corresponding explanations consistently. However, it is not straightforward to specify a complex format to generate explanations, because explicit control of generation is difficult with prompts. This study introduces a method called controlled generation with Prompt Insertion (PI) so that LLMs can explain the reasons for corrections in natural language. In PI, LLMs first correct the input text, and then we automatically extract the correction points based on the rules. The extracted correction points are sequentially inserted into the LLM's explanation output as prompts, guiding the LLMs to generate explanations for the correction points. We also create an Explainable GEC (XGEC) dataset of correction reasons by annotating NUCLE, CoNLL2013, and CoNLL2014. Although generations from GPT-3 and ChatGPT using original prompts miss some correction points, the generation control using PI can explicitly guide to describe explanations for all correction points, contributing to improved performance in generating correction reasons.

What problem does this paper attempt to address?

The paper attempts to address the problem of how to generate natural language explanations in the task of grammatical error correction (GEC) so that users can understand the reasons for the corrections. Existing research, although providing some correction bases based on markers, examples, and prompts, does not directly explain the specific reasons for the corrections. Although large language models (LLMs) have demonstrated the ability to generate natural language explanations in other tasks, there is no method yet that can effectively generate these explanations in the GEC task. Specifically, the paper proposes a method called "Controlled Generation with Prompt Insertion (PI)," which guides LLMs to generate detailed explanations for each correction point by inserting prompts during the generation process. This method not only improves the accuracy and coverage of the generated explanations but also creates a new dataset, XGEC, for evaluating and training models that generate natural language explanations. In summary, the main contributions of the paper include: 1. **Proposing a new method**: The PI method controls LLMs to generate natural language explanations by inserting prompts, ensuring that all correction points have corresponding explanations. 2. **Creating a new dataset**: The XGEC dataset contains a large number of erroneous texts, correct texts, and natural language explanations for each correction point, providing rich resources for research. 3. **Experimental validation of the method's effectiveness**: Through experimental comparisons, it is proven that the PI method performs better than existing methods in generating natural language explanations.

Controlled Generation with Prompt Insertion for Natural Language Explanations in Grammatical Error Correction

Enhancing Grammatical Error Correction Systems with Explanations

Exploring Effectiveness of GPT-3 in Grammatical Error Correction: A Study on Performance and Controllability in Prompt-Based Methods

Interpretability for Language Learners Using Example-Based Grammatical Error Correction

EXCGEC: A Benchmark of Edit-wise Explainable Chinese Grammatical Error Correction

The Unreliability of Explanations in Few-shot Prompting for Textual Reasoning

Analyzing the Performance of GPT-3.5 and GPT-4 in Grammatical Error Correction

XPrompt:Explaining Large Language Model's Generation via Joint Prompt Attribution

An Analysis of GPT-3's Performance in Grammatical Error Correction

Evaluating Prompting Strategies for Grammatical Error Correction Based on Language Proficiency

Explanation Selection Using Unlabeled Data for Chain-of-Thought Prompting

Is ChatGPT a Highly Fluent Grammatical Error Correction System? A Comprehensive Evaluation

How Ready Are Generative Pre-trained Large Language Models for Explaining Bengali Grammatical Errors?

GPT-3.5 for Grammatical Error Correction

Prompting open-source and commercial language models for grammatical error correction of English learner text

Do Grammatical Error Correction Models Realize Grammatical Generalization?

Unsupervised Explanation Generation Via Correct Instantiations

ChatGPT or Grammarly? Evaluating ChatGPT on Grammatical Error Correction Benchmark

Efficient and Interpretable Grammatical Error Correction with Mixture of Experts

KACE: Generating Knowledge-Aware Contrastive Explanations for Natural Language Inference

PromptExp: Multi-granularity Prompt Explanation of Large Language Models