Abstract:As Large Language Models (LLMs) continue to advance in their ability to write human-like text, a key challenge remains around their tendency to hallucinate generating content that appears factual but is ungrounded. This issue of hallucination is arguably the biggest hindrance to safely deploying these powerful LLMs into real-world production systems that impact people's lives. The journey toward widespread adoption of LLMs in practical settings heavily relies on addressing and mitigating hallucinations. Unlike traditional AI systems focused on limited tasks, LLMs have been exposed to vast amounts of online text data during training. While this allows them to display impressive language fluency, it also means they are capable of extrapolating information from the biases in training data, misinterpreting ambiguous prompts, or modifying the information to align superficially with the input. This becomes hugely alarming when we rely on language generation capabilities for sensitive applications, such as summarizing medical records, financial analysis reports, etc. This paper presents a comprehensive survey of over 32 techniques developed to mitigate hallucination in LLMs. Notable among these are Retrieval Augmented Generation (Lewis et al, 2021), Knowledge Retrieval (Varshney et al,2023), CoNLI (Lei et al, 2023), and CoVe (Dhuliawala et al, 2023). Furthermore, we introduce a detailed taxonomy categorizing these methods based on various parameters, such as dataset utilization, common tasks, feedback mechanisms, and retriever types. This classification helps distinguish the diverse approaches specifically designed to tackle hallucination issues in LLMs. Additionally, we analyze the challenges and limitations inherent in these techniques, providing a solid foundation for future research in addressing hallucinations and related phenomena within the realm of LLMs.

Comprehending and Reducing LLM Hallucinations

Insights into Classifying and Mitigating LLMs' Hallucinations

Investigating and Addressing Hallucinations of LLMs in Tasks Involving Negation

Redefining "Hallucination" in LLMs: Towards a psychology-informed framework for mitigating misinformation

LLMs Will Always Hallucinate, and We Need to Live With This

Banishing LLM Hallucinations Requires Rethinking Generalization

Hallucination Detection and Hallucination Mitigation: An Investigation

Look Within, Why LLMs Hallucinate: A Causal Perspective

Don't Believe Everything You Read: Enhancing Summarization Interpretability through Automatic Identification of Hallucinations in Large Language Models

A Comprehensive Survey of Hallucination Mitigation Techniques in Large Language Models

Do LLMs Know about Hallucination? An Empirical Investigation of LLM's Hidden States

Hallucination is Inevitable: An Innate Limitation of Large Language Models

Unravelling the Mysteries of Hallucination in Large Language Models: Strategies for Precision in Artificial Intelligence Language Generation

A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questions

Iter-AHMCL: Alleviate Hallucination for Large Language Model via Iterative Model-level Contrastive Learning

Evaluating and Enhancing Trustworthiness of LLMs in Perception Tasks

LLM Internal States Reveal Hallucination Risk Faced With a Query

Large Legal Fictions: Profiling Legal Hallucinations in Large Language Models

Can a Hallucinating Model help in Reducing Human "Hallucination"?

InterrogateLLM: Zero-Resource Hallucination Detection in LLM-Generated Answers