Abstract:Large language models are often expected to constantly adapt to new sources of knowledge and knowledge editing techniques aim to efficiently patch the outdated model knowledge, with minimal modification. Most prior works focus on monolingual knowledge editing in English, even though new information can emerge in any language from any part of the world. We propose the Cross-Lingual Multi-Hop Knowledge Editing paradigm, for measuring and analyzing the performance of various SoTA knowledge editing techniques in a cross-lingual setup. Specifically, we create a parallel cross-lingual benchmark, CROLIN-MQUAKE for measuring the knowledge editing capabilities. Our extensive analysis over various knowledge editing techniques uncover significant gaps in performance between the cross-lingual and English-centric setting. Following this, we propose a significantly improved system for cross-lingual multi-hop knowledge editing, CLEVER-CKE. CLEVER-CKE is based on a retrieve, verify and generate knowledge editing framework, where a retriever is formulated to recall edited facts and support an LLM to adhere to knowledge edits. We develop language-aware and hard-negative based contrastive objectives for improving the cross-lingual and fine-grained fact retrieval and verification process used in this framework. Extensive experiments on three LLMs, eight languages, and two datasets show CLEVER-CKE's significant gains of up to 30% over prior methods.

What problem does this paper attempt to address?

The paper is primarily dedicated to addressing the issue of cross-lingual multi-hop knowledge editing. Specifically: 1. **Problem Background**: As large language models (LLMs) are increasingly adopted across different cultural and linguistic contexts, these models need to be continuously updated to adapt to the ever-changing knowledge of the world, in order to maintain their utility and reliability in various applications. However, due to the increasing computational resources and data required to train these models, knowledge editing techniques have emerged, aiming to efficiently modify language models to accommodate new factual information. 2. **Limitations of Existing Work**: Currently, most knowledge editing techniques focus on monolingual (primarily English) environments, and are unable to effectively handle new knowledge from different languages. Additionally, existing methods assume that the facts being edited are independent when dealing with multi-hop reasoning, without considering the chain reactions brought about by the multi-hop reasoning process. 3. **Research Objective**: This paper proposes a cross-lingual multi-hop knowledge editing paradigm (Cross-Lingual Multi-Hop Knowledge Editing) to measure and analyze the performance of various state-of-the-art knowledge editing techniques in cross-lingual environments. By creating a parallel cross-lingual benchmark test set (CROLIN-MQ UAKE), the authors provide a detailed analysis of the performance gaps in existing methods for cross-lingual knowledge editing and propose an improved method—CLEVER-CKE, which significantly enhances the effectiveness of cross-lingual multi-hop knowledge editing. 4. **Main Contributions**: - Created the first benchmark dataset for evaluating cross-lingual multi-hop knowledge editing capabilities; - Proposed a new method named CLEVER-CKE, which achieves up to 30% performance improvement across various language models, datasets, and languages by improving the retrieval and verification process; - Conducted a detailed analysis of existing methods, revealing the main challenges in cross-lingual knowledge editing and designed targeted solutions.

Cross-Lingual Multi-Hop Knowledge Editing -- Benchmarks, Analysis and a Simple Contrastive Learning based Approach

Cross-Lingual Knowledge Editing in Large Language Models

Learning to Edit: Aligning LLMs with Knowledge Editing

Knowledge Editing through Chain-of-Thought

MLaKE: Multilingual Knowledge Editing Benchmark for Large Language Models

A Comprehensive Study of Knowledge Editing for Large Language Models

Uncovering Overfitting in Large Language Model Editing

KEBench: A Benchmark on Knowledge Editing for Large Vision-Language Models

MEMLA: Enhancing Multilingual Knowledge Editing with Neuron-Masked Low-Rank Adaptation

ComprehendEdit: A Comprehensive Dataset and Evaluation Framework for Multimodal Knowledge Editing

Enhancing Multi-hop Reasoning through Knowledge Erasure in Large Language Model Editing

BMIKE-53: Investigating Cross-Lingual Knowledge Editing with In-Context Learning

Knowledge Editing on Black-box Large Language Models

Retrieval-augmented Multilingual Knowledge Editing

Language Anisotropic Cross-Lingual Model Editing

ConceptEdit: Conceptualization-Augmented Knowledge Editing in Large Language Models for Commonsense Reasoning

InstructEdit: Instruction-based Knowledge Editing for Large Language Models

MQuAKE: Assessing Knowledge Editing in Language Models via Multi-Hop Questions

Multilingual Knowledge Editing with Language-Agnostic Factual Neurons