Abstract:Large language models (LLMs) have learned vast amounts of factual knowledge through self-supervised pre-training on large-scale corpora. Meanwhile, LLMs have also demonstrated excellent multilingual capabilities, which can express the learned knowledge in multiple languages. However, the knowledge storage mechanism in LLMs still remains mysterious. Some researchers attempt to demystify the factual knowledge in LLMs from the perspective of knowledge neurons, and subsequently discover language-agnostic knowledge neurons that store factual knowledge in a form that transcends language barriers. However, the preliminary finding suffers from two limitations: 1) High Uncertainty in Localization Results. Existing study only uses a prompt-based probe to localize knowledge neurons for each fact, while LLMs cannot provide consistent answers for semantically equivalent queries. Thus, it leads to inaccurate localization results with high uncertainty. 2) Lack of Analysis in More Languages. The study only analyzes language-agnostic knowledge neurons on English and Chinese data, without exploring more language families and languages. Naturally, it limits the generalizability of the findings. To address aforementioned problems, we first construct a new benchmark called Rephrased Multilingual LAMA (RML-LAMA), which contains high-quality cloze-style multilingual parallel queries for each fact. Then, we propose a novel method named Multilingual Integrated Gradients with Uncertainty Estimation (MATRICE), which quantifies the uncertainty across queries and languages during knowledge localization. Extensive experiments show that our method can accurately localize language-agnostic knowledge neurons. We also further investigate the role of language-agnostic knowledge neurons in cross-lingual knowledge editing, knowledge enhancement and new knowledge injection.

Cracking Factual Knowledge: A Comprehensive Analysis of Degenerate Knowledge Neurons in Large Language Models

Journey to the Center of the Knowledge Neurons: Discoveries of Language-Independent Knowledge Neurons and Degenerate Knowledge Neurons

One Mind, Many Tongues: A Deep Dive into Language-Agnostic Knowledge Neurons in Large Language Models

What does the Knowledge Neuron Thesis Have to do with Knowledge?

Neuron-Level Knowledge Attribution in Large Language Models

Unveiling Factual Recall Behaviors of Large Language Models through Knowledge Neurons

Do Large Language Models Know about Facts?

Identifying Query-Relevant Neurons in Large Language Models for Long-Form Texts

Measuring and Modifying Factual Knowledge in Large Language Models

Large Language Models and Knowledge Graphs: Opportunities and Challenges

Systematic Assessment of Factual Knowledge in Large Language Models

How Do Large Language Models Acquire Factual Knowledge During Pretraining?

Identification of Knowledge Neurons in Protein Language Models

Large Knowledge Model: Perspectives and Challenges

Investigating the Factual Knowledge Boundary of Large Language Models with Retrieval Augmentation

Knowledge Neurons in Pretrained Transformers

Physics of Language Models: Part 3.1, Knowledge Storage and Extraction

Finding Neurons in a Haystack: Case Studies with Sparse Probing

Unveiling LLMs: The Evolution of Latent Representations in a Dynamic Knowledge Graph

Beyond Factuality: A Comprehensive Evaluation of Large Language Models as Knowledge Generators

Large Language Models with Controllable Working Memory