The CoNLL-2014 Shared Task on Grammatical Error Correction

Hwee Tou Ng,Siew Mei Wu,Yuanbin Wu,Christian Hadiwinoto,Joel R. Tetreault

DOI: https://doi.org/10.3115/v1/w14-1701

2013-01-01

Abstract:Hwee Tou Ng, Siew Mei Wu, Ted Briscoe, Christian Hadiwinoto, Raymond Hendy Susanto, Christopher Bryant. Proceedings of the Eighteenth Conference on Computational Natural Language Learning: Shared Task. 2014.

What problem does this paper attempt to address?

The CoNLL-2013 Shared Task on Grammatical Error Correction

Hwee Tou Ng,Siew Mei Wu,Yuanbin Wu,Christian Hadiwinoto,Joel Tetreault

2013-01-01

Abstract:The CoNLL-2013 shared task was devoted to grammatical error correction. In this paper, we give the task definition, present the data sets, and describe the evaluation metric and scorer used in the shared task. We also give an overview of the various approaches adopted by the participating teams, and present the evaluation results.
A Hybrid Model for Grammatical Error Correction.

Yang Xiang,Bo Yuan,Yaoyun Zhang,Xiaolong Wang,Wen Zheng,Chongqiang Wei

2013-01-01

Abstract:This paper presents a hybrid model for the CoNLL-2013 shared task which focuses on the problem of grammatical error correction. This year’s task includes determiner, preposition, noun number, verb form, and subject-verb agreement errors which is more comprehensive than previous error correction tasks. We correct these five types of errors in different modules where either machine learning based or rule-based methods are applied. Preprocessing and post-processing procedures are employed to keep idiomatic phrases from being corrected. We achieved precision of 35.65%, recall of 16.56%, F1 of 22.61% in the official evaluation and precision of 41.75%, recall of 20.29%, F1 of 27.3% in the revised version. Some further comparisons employing different strategies are made in our experiments.
Grammatical Error Correction As Multiclass Classification with Single Model

Zhongye Jia,Peilu Wang,Hai Zhao

2013-01-01

Abstract:This paper describes our system in the shared task of CoNLL-2013. We illustrate that grammatical error detection and correction can be transformed into a multiclass classification task and implemented as a single-model system regardless of various error types with the aid of maximum entropy modeling. Our system achieves the F1 score of 17.13% on the standard test set.
Overview of the NLPCC 2018 Shared Task: Grammatical Error Correction.

Yuanyuan Zhao,Nan Jiang,Weiwei Sun,Xiaojun Wan

DOI: https://doi.org/10.1007/978-3-319-99501-4_41

2018-01-01

Abstract:In this paper, we present an overview of the Grammatical Error Correction task in the NLPCC 2018 shared tasks. We give detailed descriptions of the task definition and the data for training as well as evaluation. We also summarize the approaches investigated by the participants of this task. Such approaches demonstrate the state-of-the-art of Grammatical Error Correction for Mandarin Chinese. The data set and evaluation tool used by this task is available at https://github.com/zhaoyyoo/NLPCC2018_GEC.
A Unified Framework for Grammar Error Correction

Longkai Zhang,Houfeng Wang

DOI: https://doi.org/10.3115/v1/w14-1713

2014-01-01

Abstract:In this paper we describe the PKU system for the CoNLL-2014 grammar error correction shared task. We propose a unified framework for correcting all types of errors. We use unlabeled news texts instead of large amount of human annotated texts as training data. Based on these data, a tri-gram language model is used to correct the replacement errors while two extra classification models are trained to correct errors related to determiners and prepositions. Our system achieves 25.32% in f0.5 on the original test data and 29.10% on the revised test data.
Grammatical Error Detection and Correction Using a Single Maximum Entropy Model.

Peilu Wang,Zhongye Jia,Hai Zhao

DOI: https://doi.org/10.3115/v1/w14-1710

2014-01-01

Abstract:This paper describes the system of Shanghai Jiao Tong Unvierity team in the CoNLL-2014 shared task. Error correction operations are encoded as a group of predefined labels and therefore the task is formulized as a multi-label classification task. For training, labels are obtained through a strict rule-based approach. For decoding, errors are detected and corrected according to the classification results. A single maximum entropy model is used for the classification implementation incorporated with an improved feature selection algorithm. Our system achieved precision of 29.83, recall of 5.16 and F 0.5 of 15.24 in the official evaluation.
Grammatical Error Correction: A Survey of the State of the Art

Christopher Bryant,Zheng Yuan,Muhammad Reza Qorib,Hannan Cao,Hwee Tou Ng,Ted Briscoe

DOI: https://doi.org/10.1162/coli_a_00478

IF: 7.778

2023-04-28

Computational Linguistics

Abstract:Abstract Grammatical Error Correction (GEC) is the task of automatically detecting and correcting errors in text. The task not only includes the correction of grammatical errors, such as missing prepositions and mismatched subject-verb agreement, but also orthographic and semantic errors, such as misspellings and word choice errors respectively. The field has seen significant progress in the last decade, motivated in part by a series of five shared tasks, which drove the development of rule-based methods, statistical classifiers, statistical machine translation, and finally neural machine translation systems which represent the current dominant state of the art. In this survey paper, we condense the field into a single article and first outline some of the linguistic challenges of the task, introduce the most popular datasets that are available to researchers (for both English and other languages), and summarise the various methods and techniques that have been developed with a particular focus on artificial error generation. We next describe the many different approaches to evaluation as well as concerns surrounding metric reliability, especially in relation to subjective human judgements, before concluding with an overview of recent progress and suggestions for future work and remaining challenges. We hope that this survey will serve as comprehensive resource for researchers who are new to the field or who want to be kept apprised of recent developments.

computer science, artificial intelligence, interdisciplinary applications,linguistics
Chinese Grammatical Error Diagnosis Using Ensemble Learning

Yang Xiang,Xiaolong Wang,Wenying Han,Qinghua Hong

DOI: https://doi.org/10.18653/v1/w15-4415

2015-01-01

Abstract:Automatic grammatical error detection for Chinese has been a big challenge for NLP researchers for a long time, mostly due to the flexible and irregular ways in the expressing of this language. Strictly speaking, there is no evidence of a series of formal and strict grammar rules for Chinese, especially for the spoken Chinese, making it hard for foreigners to master this language. The CFL shared task provides a platform for the researchers to develop automatic engines to detect grammatical errors based on a number of manually annotated Chinese spoken sentences. This paper introduces HITSZ’s system for this year’s Chinese grammatical error diagnosis (CGED) task. Similar to the last year’s task, we put our emphasis mostly on the error detection level and error type identification level but did little for the position level. For all our models, we simply use supervised machine learning methods constrained to the given training corpus, with neither any heuristic rules nor any other referenced materials (except for the last years’ data). Among the three runs of results we submitted, the one using the ensemble classifier Random Feature Subspace (HITSZ_Run1) gained the best performance, with an optimal F1 of 0.6648 for the detection level and 0.2675 for the identification level.
A Simple but Effective Classification Model for Grammatical Error Correction.

Zhu Kaili,Chuan Wang,Ruobing Li,Yang Liu,Tianlei Hu,Hui Lin

DOI: https://doi.org/10.48550/arxiv.1807.00488

2018-01-01

Abstract:We treat grammatical error correction (GEC) as a classification problem in this study, where for different types of errors, a target word is identified, and the classifier predicts the correct word form from a set of possible choices. We propose a novel neural network based feature representation and classification model, trained using large text corpora without human annotations. Specifically we use RNNs with attention to represent both the left and right context of a target word. All feature embeddings are learned jointly in an end-to-end fashion. Experimental results show that our novel approach outperforms other classifier methods on the CoNLL-2014 test set (F0.5 45.05 Our model is simple but effective, and is suitable for industrial production.
Chinese Grammatical Error Diagnosis with Long Short-Term Memory Networks.

Bo Zheng,Wanxiang Che,Jiang Guo,Ting Liu

2016-01-01

Abstract:Grammatical error diagnosis is an important task in natural language processing. This paper introduces our Chinese Grammatical Error Diagnosis (CGED) system in the NLP-TEA-3 shared task for CGED. The CGED system can diagnose four types of grammatical errors which are redundant words (R), missing words (M), bad word selection (S) and disordered words (W). We treat the CGED task as a sequence labeling task and describe three models, including a CRF-based model, an LSTM-based model and an ensemble model using stacking. We also show in details how we build and train the models. Evaluation includes three levels, which are detection level, identification level and position level. On the CGED-HSK dataset of NLP-TEA-3 shared task, our system presents the best F1-scores in all the three levels and also the best recall in the last two levels.
Bi-LSTM Neural Networks for Chinese Grammatical Error Diagnosis.

Shen Huang,Houfeng Wang

2016-01-01

Abstract:Grammatical Error Diagnosis for Chinese has always been a challenge for both foreign learners and NLP researchers, for the variousity of grammar and the flexibility of expression. In this paper, we present a model based on Bidirectional Long Short-Term Memory(Bi-LSTM) neural networks, which treats the task as a sequence labeling problem, so as to detect Chinese grammatical errors, to identify the error types and to locate the error positions. In the corpora of this year’s shared task, there can be multiple errors in a single offset of a sentence, to address which, we simutaneously train three Bi-LSTM models sharing word embeddings which label Missing, Redundant and Selection errors respectively. We regard word ordering error as a special kind of word selection error which is longer during training phase, and then separate them by length during testing phase. In NLP-TEA 3 shared task for Chinese Grammatical Error Diagnosis(CGED), Our system achieved relatively high F1 for all the three levels in the traditional Chinese track and for the detection level in the Simpified Chinese track.
Grammatical Error Correction with Dependency Distance

Haowen Lin,Jinlong Li,Xu Zhang,Huanhuan Chen

DOI: https://doi.org/10.1145/3459637.3482348

2021-01-01

Abstract:ABSTRACTGrammatical Error Correction (GEC) task is always considered as low resource machine translation task which translates a sentence in an ungrammatical language to a grammatical language. As the state-of-the-art approach to GEC task, transformer-based neural machine translation model takes input sentence as a token sequence without sentence's structure information, and may be misled by some strange ungrammatical contexts. In response, to lay more attention on a given token's correct collocation rather than the misleading tokens, we propose dependent self-attention to relatively increase the attention score between correct collocations according to the dependency distance between tokens. However, as the source sentence is ungrammatical in GEC task, the correct collocations can hardly be extracted by normal dependency parser. Therefore, we propose dependency parser for ungrammatical sentence to get the dependency distance between tokens in the ungrammatical sentence. Our method achieves competitive results on both BEA-2019 shared task, CoNLL-2014 shared task and JFLEG test sets.
Pillars of Grammatical Error Correction: Comprehensive Inspection Of Contemporary Approaches In The Era of Large Language Models

Kostiantyn Omelianchuk,Andrii Liubonko,Oleksandr Skurzhanskyi,Artem Chernodub,Oleksandr Korniienko,Igor Samokhin

2024-04-23

Abstract:In this paper, we carry out experimental research on Grammatical Error Correction, delving into the nuances of single-model systems, comparing the efficiency of ensembling and ranking methods, and exploring the application of large language models to GEC as single-model systems, as parts of ensembles, and as ranking methods. We set new state-of-the-art performance with F_0.5 scores of 72.8 on CoNLL-2014-test and 81.4 on BEA-test, respectively. To support further advancements in GEC and ensure the reproducibility of our research, we make our code, trained models, and systems' outputs publicly available.

Computation and Language
HWCGEC:HW-TSC's 2023 Submission for the NLPCC2023's Chinese Grammatical Error Correction Task.

Chang Su ,Xiaofeng Zhao,Xiaosong Qiao,Min Zhang,Hao Yang ,Junhao Zhu,Ming Zhu,Wenbing Ma

DOI: https://doi.org/10.1007/978-3-031-44699-3_6

2023-01-01

Abstract:Deep learning has shown remarkable effectiveness in various language tasks. This paper presents Huawei Translation Services Center’s (HW-TSC’s) work called HWCGEC which get the best performance among the seven submitted results in the NLPCC2023 shared task 1, namely Chinese grammatical error correction (CGEC). CGEC aims to automatically correct grammatical errors that violate language rules and converts the noisy input texts to clean output texts. This paper, through experiments, discovered that after model fine-tuning the BART a sequence to sequence (seq2seq) model performs better than the ChatGLM a large language model (LLM) in situations where training data is large while the LoRA mode has a smaller number of parameters for fine-tuning. Additionally, the BART model achieves good results in the CGEC task through data augmentation and curriculum learning methods. Although the performance of LLM is poor in experiments, they possess excellent logical abilities. With the training set becoming more diverse and the methods for training set data augmentation becoming more refined, the supervised fine-tuning (SFT) mode trained LLMs are expected to achieve significant improvements in CGEC tasks in the future.
Combining GCN and Transformer for Chinese Grammatical Error Detection

Jinhong Zhang Jinhong Zhang

DOI: https://doi.org/10.53106/160792642022122307020

2022-12-01

網際網路技術學刊

Abstract:This paper describes our system at a task: Chinese Grammatical Error Diagnosis (CGED). The task is held by the Natural Language Processing Techniques for Educational Applications (NLP-TEA) to encourage the development of automatic grammatical error diagnosis in Chinese learning since 2014. The goal of CGED is to diagnose four types of grammatical errors: word selection (S), redundant words (R), missing words (M), and disordered words (W). The automatic CGED system contains two parts including error detection and error correction and our system is designed to solve the error detection problem. Our system is built on three models: 1) a BERT-based model leveraging syntactic information; 2) a BERT-based model leveraging contextual embeddings; 3) a lexicon-based graph neural network leveraging lexical information. We also design an ensemble mechanism to improve the single model’s performance. Finally, our system achieves the highest F1 scores at detection level and identification level among all teams participating in the CGED 2020 task.  

computer science, information systems,telecommunications
Detection-Correction Structure via General Language Model for Grammatical Error Correction

Wei Li,Houfeng Wang

2024-05-28

Abstract:Grammatical error correction (GEC) is a task dedicated to rectifying texts with minimal edits, which can be decoupled into two components: detection and correction. However, previous works have predominantly focused on direct correction, with no prior efforts to integrate both into a single model. Moreover, the exploration of the detection-correction paradigm by large language models (LLMs) remains underdeveloped. This paper introduces an integrated detection-correction structure, named DeCoGLM, based on the General Language Model (GLM). The detection phase employs a fault-tolerant detection template, while the correction phase leverages autoregressive mask infilling for localized error correction. Through the strategic organization of input tokens and modification of attention masks, we facilitate multi-task learning within a single model. Our model demonstrates competitive performance against the state-of-the-art models on English and Chinese GEC datasets. Further experiments present the effectiveness of the detection-correction structure in LLMs, suggesting a promising direction for GEC.

Computation and Language
Corpus and Unsupervised Benchmark: Towards Tagalog Grammatical Error Correction

Nankai Lin,Hongbin Zhang,Menglan Shen,Yu Wang,Shengyi Jiang,Aimin Yang

DOI: https://doi.org/10.1016/j.csl.2024.101750

IF: 3.252

2024-01-01

Computer Speech & Language

Abstract:Grammatical error correction (GEC) is a challenging task for natural language processing techniques. Many efforts to address GEC have been made for high-resource languages such as English or Chinese. However, limited work has been done for low-resource languages because of the lack of large annotated corpora. In low-resource languages, the current unsupervised GEC based on language model scoring performs well. However, the pre-trained language model is still to be explored in this context. This study proposes a BERT-based unsupervised GEC framework that primarily addresses word-level errors, where GEC is viewed as a multi-class classification task. The framework contains three modules: a data flow construction module, a sentence perplexity scoring module, and an error detecting and correcting module. We propose a novel scoring method for pseudo-perplexity to evaluate a sentence’s probable correctness and construct a Tagalog corpus for Tagalog GEC research. It obtains competitive performance on the self-constructed Tagalog corpus and the open-source Indonesian corpus, and it demonstrates that our framework is complementary to the baseline methods for low-resource GEC tasks. Our corpus can be obtained from https://github.com/GKLMIP/TagalogGEC.
ETS: An Error Tolerable System for Coreference Resolution.

Hao Xiong,Linfeng Song,Fandong Meng,Yang Liu,Qun Liu,Yajuan Lv

2011-01-01

Abstract:This paper presents our error tolerable system for coreference resolution in CoNLL-2011 (Pradhan et al., 2011) shared task (closed track). Different from most previous reported work, we detect mention candidates based on packed forest instead of single parse tree, and we use beam search algorithm based on the Bell Tree to create entities. Experimental results show that our methods achieve promising results on the development set.
Improving Grammatical Error Correction with Data Augmentation by Editing Latent Representation

Zhaohong Wan,Xiaojun Wan,Wenguang Wang

DOI: https://doi.org/10.18653/v1/2020.coling-main.200

2020-01-01

Abstract:The incorporation of data augmentation method in grammatical error correction task has attracted much attention.However, existing data augmentation methods mainly apply noise to tokens, which leads to the lack of diversity of generated errors.In view of this, we propose a new data augmentation method that can apply noise to the latent representation of a sentence.By editing the latent representations of grammatical sentences, we can generate synthetic samples with various error types.Combining with some pre-defined rules, our method can greatly improve the performance and robustness of existing grammatical error correction models.We evaluate our method on public benchmarks of GEC task and it achieves the state-of-the-art performance on CoNLL-2014 and FCE benchmarks.
Neural Language Correction with Character-Based Attention

Ziang Xie,Anand Avati,Naveen Arivazhagan,Dan Jurafsky,Andrew Y. Ng

DOI: https://doi.org/10.48550/arXiv.1603.09727

2016-04-01

Abstract:Natural language correction has the potential to help language learners improve their writing skills. While approaches with separate classifiers for different error types have high precision, they do not flexibly handle errors such as redundancy or non-idiomatic phrasing. On the other hand, word and phrase-based machine translation methods are not designed to cope with orthographic errors, and have recently been outpaced by neural models. Motivated by these issues, we present a neural network-based approach to language correction. The core component of our method is an encoder-decoder recurrent neural network with an attention mechanism. By operating at the character level, the network avoids the problem of out-of-vocabulary words. We illustrate the flexibility of our approach on dataset of noisy, user-generated text collected from an English learner forum. When combined with a language model, our method achieves a state-of-the-art $F_{0.5}$-score on the CoNLL 2014 Shared Task. We further demonstrate that training the network on additional data with synthesized errors can improve performance.

Computation and Language,Artificial Intelligence

The CoNLL-2014 Shared Task on Grammatical Error Correction

The CoNLL-2013 Shared Task on Grammatical Error Correction

A Hybrid Model for Grammatical Error Correction.

Grammatical Error Correction As Multiclass Classification with Single Model

Overview of the NLPCC 2018 Shared Task: Grammatical Error Correction.

A Unified Framework for Grammar Error Correction

Grammatical Error Detection and Correction Using a Single Maximum Entropy Model.

Grammatical Error Correction: A Survey of the State of the Art

Chinese Grammatical Error Diagnosis Using Ensemble Learning

A Simple but Effective Classification Model for Grammatical Error Correction.

Chinese Grammatical Error Diagnosis with Long Short-Term Memory Networks.

Bi-LSTM Neural Networks for Chinese Grammatical Error Diagnosis.

Grammatical Error Correction with Dependency Distance

Pillars of Grammatical Error Correction: Comprehensive Inspection Of Contemporary Approaches In The Era of Large Language Models

HWCGEC:HW-TSC's 2023 Submission for the NLPCC2023's Chinese Grammatical Error Correction Task.

Combining GCN and Transformer for Chinese Grammatical Error Detection

Detection-Correction Structure via General Language Model for Grammatical Error Correction

Corpus and Unsupervised Benchmark: Towards Tagalog Grammatical Error Correction

ETS: An Error Tolerable System for Coreference Resolution.

Improving Grammatical Error Correction with Data Augmentation by Editing Latent Representation

Neural Language Correction with Character-Based Attention