Abstract:Knowledge stored in large language models requires timely updates to reflect the dynamic nature of real-world information. To update the knowledge, most knowledge editing methods focus on the low layers, since recent probes into the knowledge recall process reveal that the answer information is enriched in low layers. However, these probes only and could only reveal critical recall stages for the original answers, while the goal of editing is to rectify model's prediction for the target answers. This inconsistency indicates that both the probe approaches and the associated editing methods are deficient. To mitigate the inconsistency and identify critical editing regions, we propose a contrast-based probe approach, and locate two crucial stages where the model behavior diverges between the original and target answers: Information Enrichment in low layers and Probability Promotion in high layers. Building upon the insights, we develop the Joint knowledge Editing for information Enrichment and probability Promotion (JEEP) method, which jointly edits both the low and high layers to modify the two critical recall stages. Considering the mutual interference and growing forgetting due to dual modifications, JEEP is designed to ensure that updates to distinct regions share the same objectives and are complementary. We rigorously evaluate JEEP by editing up to thousands of facts on various models, i.e., GPT-J (6B) and LLaMA (7B), and addressing diverse editing objectives, i.e., adding factual and counterfactual knowledge. In all tested scenarios, JEEP achieves best performances, validating the effectiveness of the revealings of our probe approach and the designs of our editing method. Our code and data are available at <a class="link-external link-https" href="https://github.com/Eric8932/JEEP" rel="external noopener nofollow">this https URL</a>.

AKEW: Assessing Knowledge Editing in the Wild

Knowledge Editing on Black-box Large Language Models

Eva-KELLM: A New Benchmark for Evaluating Knowledge Editing of LLMs

KEBench: A Benchmark on Knowledge Editing for Large Vision-Language Models

EVEDIT: Event-based Knowledge Editing with Deductive Editing Boundaries

Event-level Knowledge Editing

Everything is Editable: Extend Knowledge Editing to Unstructured Data in Large Language Models

WilKE: Wise-Layer Knowledge Editor for Lifelong Knowledge Editing

A Comprehensive Study of Knowledge Editing for Large Language Models

Editing the Mind of Giants: An In-Depth Exploration of Pitfalls of Knowledge Editing in Large Language Models

Unveiling the Pitfalls of Knowledge Editing for Large Language Models

Propagation and Pitfalls: Reasoning-based Assessment of Knowledge Editing through Counterfactual Tasks

Pioneering Reliable Assessment in Text-to-Image Knowledge Editing: Leveraging a Fine-Grained Dataset and an Innovative Criterion

Editing Language Model-based Knowledge Graph Embeddings

Uncovering Overfitting in Large Language Model Editing

Joint Knowledge Editing for Information Enrichment and Probability Promotion

ComprehendEdit: A Comprehensive Dataset and Evaluation Framework for Multimodal Knowledge Editing