IdEALS: Idiomatic Expressions for Advancement of Language Skills

Narutatsu Ri,Bill Sun,Sam Davidson,Zhou Yu

2023-05-25

Abstract:Although significant progress has been made in developing methods for Grammatical Error Correction (GEC), addressing word choice improvements has been notably lacking and enhancing sentence expressivity by replacing phrases with advanced expressions is an understudied aspect. In this paper, we focus on this area and present our investigation into the task of incorporating the usage of idiomatic expressions in student writing. To facilitate our study, we curate extensive training sets and expert-annotated testing sets using real-world data and evaluate various approaches and compare their performance against human experts.

Computation and Language

What problem does this paper attempt to address?

The problem that this paper attempts to solve is an important shortcoming of existing grammar error correction (GEC) methods in improving the quality of students' writing: that is, enhancing the expressiveness and naturalness of sentences by replacing phrases with more advanced expressions. Specifically, the authors focus on how to introduce more complex and authentic language structures such as idioms and idiomatic expressions in students' writing, thereby improving students' language proficiency.

### Specific problem description:
1. **Insufficient improvement suggestions for word selection**: Although significant progress has been made in grammar error correction, there are still a lack of effective tools and methods in providing better word selection suggestions.
2. **Insufficient research on enhancing sentence expressiveness**: There is relatively little research in the area of using more advanced expressions (such as idioms, idiomatic expressions) to replace the original expressions to enhance the expressiveness and naturalness of sentences.
3. **Inappropriate evaluation metrics**: Existing evaluation metrics are not suitable for writing improvement tasks, especially when it comes to the use of idioms.

### Main contributions of the paper:
1. **Constructing a dataset**: Compiled a dataset containing a large - scale training set and an expert - annotated test set. These datasets are composed of real students' compositions and are called the IdEALS (Idiomatic Expressions for Advancement of Language Skills) dataset.
2. **Proposing evaluation metrics**: Proposed precise evaluation metrics for the ISG - WI task and benchmarked two different methods.
3. **Exploring two methods**: Researched the performance of the fine - tuning and in - context learning methods in generating idiomatic expressions respectively, and compared their performance.

### Method overview:
- **Fine - tuning**: Fine - tune the Parrot model based on the T5 model, use the IdEALS dataset for training, and ensure the semantic, grammatical and structural correctness of the generated sentences through the post - processing layer.
- **In - context learning**: Utilize pre - trained language models (such as GPT - 3.5 - turbo and text - davinci - 003), and generate idiomatic expressions by providing task descriptions, examples and test instances.

### Results and conclusions:
The experimental results show that the model trained on the IdEALS dataset performs well in generating idiomatic expressions. Especially when combined with the post - processing layer, it can significantly reduce errors and improve the quality of generation. In addition, although the in - context learning method performs well in some aspects, it still faces challenges in maintaining grammatical correctness and richness.

Through these efforts, the paper fills the gap in existing GEC methods in terms of improvement suggestions for word selection and provides new directions and resources for future research.