Abstract:Hallucination has been widely recognized to be a significant drawback for large language models (LLMs). There have been many works that attempt to reduce the extent of hallucination. These efforts have mostly been empirical so far, which cannot answer the fundamental question whether it can be completely eliminated. In this paper, we formalize the problem and show that it is impossible to eliminate hallucination in LLMs. Specifically, we define a formal world where hallucination is defined as inconsistencies between a computable LLM and a computable ground truth function. By employing results from learning theory, we show that LLMs cannot learn all of the computable functions and will therefore always hallucinate. Since the formal world is a part of the real world which is much more complicated, hallucinations are also inevitable for real world LLMs. Furthermore, for real world LLMs constrained by provable time complexity, we describe the hallucination-prone tasks and empirically validate our claims. Finally, using the formal world framework, we discuss the possible mechanisms and efficacies of existing hallucination mitigators as well as the practical implications on the safe deployment of LLMs.

What problem does this paper attempt to address?

The problem that this paper attempts to solve is: **Can the "hallucination" phenomenon in large - language models (LLMs) be completely eliminated?** ### Detailed Interpretation #### Research Background Large - language models (LLMs) have made remarkable progress in the field of natural - language processing, but one of the key challenges they face is the "hallucination" phenomenon. Hallucination refers to the model generating seemingly reasonable but actually wrong or meaningless information. This problem has raised concerns about safety and ethics, especially when LLMs are widely applied. #### Limitations of Existing Research Current research on LLM hallucination mainly focuses on empirical methods, which cannot answer a fundamental question: whether hallucination can be completely eliminated? Since it is impossible to enumerate and test all possible inputs through empirical methods, a formal definition and theoretical analysis are required to explore this problem. #### Core Contributions of the Paper 1. **Formal Definition and Theoretical Proof**: - The author defines hallucination in a formal world, where hallucination is defined as the inconsistency between a computable LLM and a computable true function. - Using the results of learning theory, the author proves that LLM cannot learn all computable functions, so hallucination is inevitable. 2. **Practical Verification**: - For real - world LLMs limited by time complexity, the author identifies some tasks that are prone to cause hallucination and verifies their theoretical results through empirical research. 3. **Discussion of Practical Impact**: - Based on the theoretical results, the author discusses the effectiveness of existing hallucination - mitigation mechanisms and their impact on the actual deployment of LLMs. #### Main Conclusions The main conclusion of the paper is: **Hallucination is inevitable in any formal world, so in the more complex real world, the hallucination phenomenon of LLMs is also inevitable**. This conclusion means that although the occurrence of hallucination can be reduced by various methods, it is impossible to completely eliminate it. ### Formula Representation To ensure the correctness and readability of the formulas, the following are the key formulas involved in the paper: 1. **Probability Distribution of Strings**: \[ p(w_0:n - 1)=p(w_0)\prod_{t = 1}^{n - 1}p(w_t|w_0:t - 1),\quad w_i\in A,n\in\mathbb{N} \] 2. **Probability of Completing Partial Strings**: \[ p(w_0:n - 1)\propto p(w_q:n|w_0:q - 1)=\prod_{t = q}^{n - 1}p(w_t|w_0:t - 1),\quad w_i\in A,q,n\in\mathbb{N},q < n \] 3. **Definition of Hallucination**: \[ h(s)\neq f(s),\quad\exists s\in S \] where \(h\) is the LLM, \(f\) is the true function, and \(S\) is the set of all finite - length strings. ### Summary This paper reveals the inevitability of the hallucination phenomenon in LLMs through formal methods and theoretical proofs, and verifies this conclusion through empirical research. This provides important theoretical basis and practical guidance for future LLM design and application.

Hallucination is Inevitable: An Innate Limitation of Large Language Models

Hallucination is Inevitable: An Innate Limitation of Large Language Models

LLMs Will Always Hallucinate, and We Need to Live With This

Banishing LLM Hallucinations Requires Rethinking Generalization

The Troubling Emergence of Hallucination in Large Language Models -- An Extensive Definition, Quantification, and Prescriptive Remediations

A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questions

Unravelling the Mysteries of Hallucination in Large Language Models: Strategies for Precision in Artificial Intelligence Language Generation

Sources of Hallucination by Large Language Models on Inference Tasks

LLM Internal States Reveal Hallucination Risk Faced With a Query

On Large Language Models' Hallucination with Regard to Known Facts

Quantifying and Attributing the Hallucination of Large Language Models via Association Analysis

Knowledge Overshadowing Causes Amalgamated Hallucination in Large Language Models

Hallucination Detection and Hallucination Mitigation: An Investigation

A Comprehensive Survey of Hallucination Mitigation Techniques in Large Language Models

A Survey on Hallucination in Large Vision-Language Models

Hallucination of Multimodal Large Language Models: A Survey

Investigating and Addressing Hallucinations of LLMs in Tasks Involving Negation