Abstract:The recent advancements in Large Language Models (LLMs) have garnered widespread acclaim for their remarkable emerging capabilities. However, the issue of hallucination has parallelly emerged as a by-product, posing significant concerns. While some recent endeavors have been made to identify and mitigate different types of hallucination, there has been a limited emphasis on the nuanced categorization of hallucination and associated mitigation methods. To address this gap, we offer a fine-grained discourse on profiling hallucination based on its degree, orientation, and category, along with offering strategies for alleviation. As such, we define two overarching orientations of hallucination: (i) factual mirage (FM) and (ii) silver lining (SL). To provide a more comprehensive understanding, both orientations are further sub-categorized into intrinsic and extrinsic, with three degrees of severity - (i) mild, (ii) moderate, and (iii) alarming. We also meticulously categorize hallucination into six types: (i) acronym ambiguity, (ii) numeric nuisance, (iii) generated golem, (iv) virtual voice, (v) geographic erratum, and (vi) time wrap. Furthermore, we curate HallucInation eLiciTation (HILT), a publicly available dataset comprising of 75,000 samples generated using 15 contemporary LLMs along with human annotations for the aforementioned categories. Finally, to establish a method for quantifying and to offer a comparative spectrum that allows us to evaluate and rank LLMs based on their vulnerability to producing hallucinations, we propose Hallucination Vulnerability Index (HVI). We firmly believe that HVI holds significant value as a tool for the wider NLP community, with the potential to serve as a rubric in AI-related policy-making. In conclusion, we propose two solution strategies for mitigating hallucinations.

The Pitfalls of Defining Hallucination

The Pitfalls of Defining Hallucination

Hallucination Detection and Hallucination Mitigation: An Investigation

The Troubling Emergence of Hallucination in Large Language Models -- An Extensive Definition, Quantification, and Prescriptive Remediations

Unravelling the Mysteries of Hallucination in Large Language Models: Strategies for Precision in Artificial Intelligence Language Generation

Can We Catch the Elephant? A Survey of the Evolvement of Hallucination Evaluation on Natural Language Generation

Redefining "Hallucination" in LLMs: Towards a psychology-informed framework for mitigating misinformation

Survey of Hallucination in Natural Language Generation

Sources of Hallucination by Large Language Models on Inference Tasks

A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questions

An Audit on the Perspectives and Challenges of Hallucinations in NLP

Insights into Classifying and Mitigating LLMs' Hallucinations

Developing a Reliable, General-Purpose Hallucination Detection and Mitigation Service: Insights and Lessons Learned

Embedding and Gradient Say Wrong: A White-Box Method for Hallucination Detection

Look Within, Why LLMs Hallucinate: A Causal Perspective

The Dawn After the Dark: An Empirical Study on Factuality Hallucination in Large Language Models

Visual Hallucination: Definition, Quantification, and Prescriptive Remediations

Exploring and Evaluating Hallucinations in LLM-Powered Code Generation

Hallucination is Inevitable: An Innate Limitation of Large Language Models

Knowledge Graphs, Large Language Models, and Hallucinations: An NLP Perspective