Abstract:Large language models (LLMs) are susceptible to hallucinations-outputs that are ungrounded, factually incorrect, or inconsistent with prior generations. We focus on close-book Question Answering (CBQA), where previous work has not fully addressed the distinction between two possible kinds of hallucinations, namely, whether the model (1) does not hold the correct answer in its parameters or (2) answers incorrectly despite having the required knowledge. We argue that distinguishing these cases is crucial for detecting and mitigating hallucinations. Specifically, case (2) may be mitigated by intervening in the model's internal computation, as the knowledge resides within the model's parameters. In contrast, in case (1) there is no parametric knowledge to leverage for mitigation, so it should be addressed by resorting to an external knowledge source or abstaining. To help distinguish between the two cases, we introduce Wrong Answer despite having Correct Knowledge (WACK), an approach for constructing model-specific datasets for the second hallucination type. Our probing experiments indicate that the two kinds of hallucinations are represented differently in the model's inner states. Next, we show that datasets constructed using WACK exhibit variations across models, demonstrating that even when models share knowledge of certain facts, they still vary in the specific examples that lead to hallucinations. Finally, we show that training a probe on our WACK datasets leads to better hallucination detection of case (2) hallucinations than using the common generic one-size-fits-all datasets. The code is available at <a class="link-external link-https" href="https://github.com/technion-cs-nlp/hallucination-mitigation" rel="external noopener nofollow">this https URL</a> .

The Axiom of Choice and Its Influence on LLM Hallucinations: An Exploration

LLMs Will Always Hallucinate, and We Need to Live With This

The Curious Case of Hallucinatory (Un)answerability: Finding Truths in the Hidden States of Over-Confident Large Language Models

Unravelling the Mysteries of Hallucination in Large Language Models: Strategies for Precision in Artificial Intelligence Language Generation

Hallucination is Inevitable: An Innate Limitation of Large Language Models

Banishing LLM Hallucinations Requires Rethinking Generalization

Redefining "Hallucination" in LLMs: Towards a psychology-informed framework for mitigating misinformation

Comprehending and Reducing LLM Hallucinations

A Debate-Driven Experiment on LLM Hallucinations and Accuracy

Beyond Words: A Mathematical Framework for Interpreting Large Language Models

Do LLMs Know about Hallucination? An Empirical Investigation of LLM's Hidden States

Large Legal Fictions: Profiling Legal Hallucinations in Large Language Models

Distinguishing Ignorance from Error in LLM Hallucinations

Order Matters in Hallucination: Reasoning Order as Benchmark and Reflexive Prompting for Large-Language-Models

Confabulation: The Surprising Value of Large Language Model Hallucinations

Look Within, Why LLMs Hallucinate: A Causal Perspective

Hallucination is the last thing you need

A Survey on Large Language Model Hallucination via a Creativity Perspective

A Novel Approach to Eliminating Hallucinations in Large Language Model-Assisted Causal Discovery

On Large Language Models' Hallucination with Regard to Known Facts