IndoToD: A Multi-Domain Indonesian Benchmark For End-to-End Task-Oriented Dialogue Systems

Muhammad Dehan Al Kautsar,Rahmah Khoirussyifa' Nurdini,Samuel Cahyawijaya,Genta Indra Winata,Ayu Purwarianti

2023-11-02

Abstract:Task-oriented dialogue (ToD) systems have been mostly created for high-resource languages, such as English and Chinese. However, there is a need to develop ToD systems for other regional or local languages to broaden their ability to comprehend the dialogue contexts in various languages. This paper introduces IndoToD, an end-to-end multi domain ToD benchmark in Indonesian. We extend two English ToD datasets to Indonesian, comprising four different domains by delexicalization to efficiently reduce the size of annotations. To ensure a high-quality data collection, we hire native speakers to manually translate the dialogues. Along with the original English datasets, these new Indonesian datasets serve as an effective benchmark for evaluating Indonesian and English ToD systems as well as exploring the potential benefits of cross-lingual and bilingual transfer learning approaches.

Computation and Language,Artificial Intelligence

What problem does this paper attempt to address?

The main problem that this paper attempts to solve is the lack of Indonesian task - oriented dialogue system (ToD) datasets. Specifically, most of the existing ToD systems are developed for resource - rich languages such as English and Chinese, while the datasets of regional or local languages such as Indonesian are very limited. This restricts the expansion of the ToD system's ability to understand the context of conversations in different languages. Therefore, this paper proposes IndoToD, a multi - domain end - to - end ToD benchmark dataset for Indonesian. By extending two English ToD datasets to Indonesian and effectively reducing the amount of annotation through the delexicalization process, IndoToD aims to provide high - quality datasets to evaluate the performance of Indonesian and English ToD systems, while exploring the potential benefits of cross - language and bilingual transfer learning methods.

In short, the core problem of the paper is to fill the gap in the Indonesian ToD dataset by creating IndoToD, thereby promoting the development and evaluation of Indonesian ToD systems.