Abstract:Pre-trained language models (PLMs) contain vast amounts of factual knowledge, but how the knowledge is stored in the parameters remains unclear. This paper delves into the complex task of understanding how factual knowledge is stored in multilingual PLMs, and introduces the Architecture-adapted Multilingual Integrated Gradients method, which successfully localizes knowledge neurons more precisely compared to current methods, and is more universal across various architectures and languages. Moreover, we conduct an in-depth exploration of knowledge neurons, leading to the following two important discoveries: (1) The discovery of Language-Independent Knowledge Neurons, which store factual knowledge in a form that transcends language. We design cross-lingual knowledge editing experiments, demonstrating that the PLMs can accomplish this task based on language-independent neurons; (2) The discovery of Degenerate Knowledge Neurons, a novel type of neuron showing that different knowledge neurons can store the same fact. Its property of functional overlap endows the PLMs with a robust mastery of factual knowledge. We design fact-checking experiments, proving that the degenerate knowledge neurons can help the PLMs to detect wrong facts. Experiments corroborate these findings, shedding light on the mechanisms of factual knowledge storage in multilingual PLMs, and contribute valuable insights to the field. The code is available at <a class="link-external link-https" href="https://github.com/heng840/AMIG" rel="external noopener nofollow">this https URL</a>.

What problem does this paper attempt to address?

### What problems does this paper attempt to solve? This paper aims to explore the storage mechanism of factual knowledge in pre - trained language models (PLMs). Specifically, the paper addresses the following two main issues: 1. **Knowledge location problems in different PLM architectures and multilingual environments**: - Current knowledge location methods perform inconsistently on different types of PLM architectures (such as the auto - encoding model BERT and the auto - regression model GPT). - Most existing methods only focus on English datasets and fail to fully explore the knowledge storage mechanism in multilingual environments. 2. **Discover new neuron types and their functions**: - The paper discovers two new types of neurons: **Language - Independent Knowledge Neurons (LIKN)** and **Degenerate Knowledge Neurons (DKN)**. - LIKN can store factual knowledge across languages, while DKN shows the characteristic of functional overlap, enhancing the robustness of PLM to factual knowledge. ### Solutions To solve the above problems, the authors propose the following methods and techniques: 1. **Architecture - adapted Multilingual Integrated Gradients (AMIG)**: - This is a new knowledge location method. By adapting to different PLM architectures and adjusting multilingual thresholds, it achieves more accurate location of knowledge neurons. - AMIG evaluates the importance of each neuron by calculating the gradient integral and designs baseline vectors suitable for different PLM architectures. 2. **Detection of language - independent knowledge neurons**: - Through cross - language query and dynamic threshold adjustment techniques, neurons that can store the same factual knowledge across languages are identified. - Experiments show that LIKN is helpful for cross - language knowledge editing tasks and improves the success rate of editing. 3. **Detection of degenerate knowledge neurons**: - It is found that certain subsets of neurons can store the same factual knowledge. Even if some subsets are suppressed, the model can still correctly express the facts; but if all subsets are suppressed, it cannot correctly express. - A fact - checking experiment based on DKN is designed, which proves that these neurons can help detect false facts, thereby enhancing the stable mastery of factual knowledge by PLM. ### Main contributions 1. The AMIG method is proposed, which effectively overcomes the limitations of traditional methods in different PLM architectures and multilingual environments and achieves more accurate location of knowledge neurons. 2. The language - independent knowledge neurons (LIKN) are discovered, and their superior performance in cross - language knowledge editing is verified. 3. The degenerate knowledge neurons (DKN) are discovered, and it is proved that their functional overlap characteristics enhance the robustness and stability of PLM to factual knowledge. Through these studies, the paper not only reveals the mechanism of factual knowledge storage in PLM, but also provides valuable insights for future cross - language knowledge processing and fact - checking.

Journey to the Center of the Knowledge Neurons: Discoveries of Language-Independent Knowledge Neurons and Degenerate Knowledge Neurons

One Mind, Many Tongues: A Deep Dive into Language-Agnostic Knowledge Neurons in Large Language Models

Cracking Factual Knowledge: A Comprehensive Analysis of Degenerate Knowledge Neurons in Large Language Models

What does the Knowledge Neuron Thesis Have to do with Knowledge?

Knowledge Neurons in Pretrained Transformers

Neuron-Level Knowledge Attribution in Large Language Models

Multilingual Knowledge Editing with Language-Agnostic Factual Neurons

Identifying Query-Relevant Neurons in Large Language Models for Long-Form Texts

Identification of Knowledge Neurons in Protein Language Models

On the Multilingual Ability of Decoder-based Pre-trained Language Models: Finding and Controlling Language-Specific Neurons

Physics of Language Models: Part 3.1, Knowledge Storage and Extraction

Knowledge Localization: Mission Not Accomplished? Enter Query Localization!

Unveiling Language Competence Neurons: A Psycholinguistic Approach to Model Interpretability

Language-Specific Neurons: The Key to Multilingual Capabilities in Large Language Models

Sharing Matters: Analysing Neurons Across Languages and Tasks in LLMs

Discovering Salient Neurons in Deep NLP Models

MMNeuron: Discovering Neuron-Level Domain-Specific Interpretation in Multimodal Large Language Model

Unveiling Factual Recall Behaviors of Large Language Models through Knowledge Neurons

Knowledge Inheritance for Pre-trained Language Models

Knowledge Circuits in Pretrained Transformers

Unveiling A Core Linguistic Region in Large Language Models