Retrieval-augmented Multilingual Knowledge Editing

Weixuan Wang,Barry Haddow,Alexandra Birch

2023-12-20

Abstract:Knowledge represented in Large Language Models (LLMs) is quite often incorrect and can also become obsolete over time. Updating knowledge via fine-tuning is computationally resource-hungry and not reliable, and so knowledge editing (KE) has developed as an effective and economical alternative to inject new knowledge or to fix factual errors in LLMs. Although there has been considerable interest in this area, current KE research exclusively focuses on the monolingual setting, typically in English. However, what happens if the new knowledge is supplied in one language, but we would like to query the LLM in a different language? To address the problem of multilingual knowledge editing, we propose Retrieval-augmented Multilingual Knowledge Editor (ReMaKE) to update new knowledge in LLMs. ReMaKE can perform model-agnostic knowledge editing in multilingual settings. ReMaKE concatenates the new knowledge retrieved from a multilingual knowledge base with prompts. Our experimental results show that ReMaKE outperforms baseline knowledge editing methods by a significant margin and is the first KE method to work in a multilingual setting. We provide our multilingual knowledge editing dataset (MzsRE) in 12 languages, which along with code, and additional project information is available at <a class="link-external link-https" href="https://github.com/Vicky-Wil/ReMaKE" rel="external noopener nofollow">this https URL</a>.

Computation and Language

What problem does this paper attempt to address?

### What problem does this paper attempt to solve?

This paper primarily addresses the following issues:

1. **Multilingual Knowledge Editing**: Knowledge in large language models (LLMs) is often inaccurate or outdated. Updating knowledge through fine-tuning not only consumes a lot of computational resources but also yields unstable results. Therefore, Knowledge Editing (KE) has been proposed as an effective and economical method to inject new knowledge or correct factual errors in LLMs. However, current research on knowledge editing mainly focuses on monolingual scenarios (usually English). If new knowledge is provided in one language but needs to query LLMs in different languages, existing methods cannot effectively handle this situation.

2. **Multilingual Knowledge Editing Framework**: To address the issue of multilingual knowledge editing, the paper proposes a method called **Retrieval-Enhanced Multilingual Knowledge Editor (ReMaKE)**. ReMaKE can perform model-agnostic knowledge editing in a multilingual environment. Specifically, ReMaKE combines new knowledge retrieved from a multilingual knowledge base with prompt information to guide LLMs in generating accurate responses.

3. **Experimental Validation**: The paper demonstrates that ReMaKE significantly outperforms existing baseline methods in 12 languages and provides a multilingual knowledge editing dataset (MzsRE), covering datasets in 12 languages, validating the effectiveness and scalability of ReMaKE in a multilingual environment.

In summary, this paper aims to develop an effective framework capable of handling multilingual knowledge editing, allowing new knowledge to be transmitted across multiple languages while maintaining accuracy.