Abstract:Large language models (LLMs) have demonstrated impressive proficiency in information retrieval, while they are prone to generating incorrect responses that conflict with reality, a phenomenon known as intrinsic hallucination. The critical challenge lies in the unclear and unreliable fact distribution within LLMs trained on vast amounts of data. The prevalent approach frames the factual detection task as a question-answering paradigm, where the LLMs are asked about factual knowledge and examined for correctness. However, existing studies primarily focused on deriving test cases only from several specific domains, such as movies and sports, limiting the comprehensive observation of missing knowledge and the analysis of unexpected hallucinations. To address this issue, we propose OntoFact, an adaptive framework for detecting unknown facts of LLMs, devoted to mining the ontology-level skeleton of the missing knowledge. Specifically, we argue that LLMs could expose the ontology-based similarity among missing facts and introduce five representative knowledge graphs (KGs) as benchmarks. We further devise a sophisticated ontology-driven reinforcement learning (ORL) mechanism to produce error-prone test cases with specific entities and relations automatically. The ORL mechanism rewards the KGs for navigating toward a feasible direction for unveiling factual errors. Moreover, empirical efforts demonstrate that dominant LLMs are biased towards answering Yes rather than No, regardless of whether this knowledge is included. To mitigate the overconfidence of LLMs, we leverage a hallucination-free detection (HFD) strategy to tackle unfair comparisons between baselines, thereby boosting the result robustness. Experimental results on 5 datasets, using 32 representative LLMs, reveal a general lack of fact in current LLMs. Notably, ChatGPT exhibits fact error rates of 51.6% on DBpedia and 64.7% on YAGO, respectively. Additionally, the ORL mechanism demonstrates promising error prediction scores, with F1 scores ranging from 70% to 90% across most LLMs. Compared to the exhaustive testing, ORL achieves an average recall of 80% while reducing evaluation time by 35.29% to 63.12%.

Do LLMs Dream of Ontologies?

Do LLMs Really Adapt to Domains? An Ontology Learning Perspective

Understanding Memorisation in LLMs: Dynamics, Influencing Factors, and Implications

LLMs4OL: Large Language Models for Ontology Learning

Do LLMs dream of elephants (when told not to)? Latent concept association and associative memory in transformers

Can Large Language Models Understand DL-Lite Ontologies? An Empirical Study

Large Language Models for Scholarly Ontology Generation: An Extensive Analysis in the Engineering Field

LLMs' Understanding of Natural Language Revealed

Large Language Models and Knowledge Graphs: Opportunities and Challenges

OntoFact: Unveiling Fantastic Fact-Skeleton of LLMs Via Ontology-Driven Reinforcement Learning

SoK: Memorization in General-Purpose Large Language Models

Unveiling LLMs: The Evolution of Latent Representations in a Dynamic Knowledge Graph

Large Language Models: The Need for Nuance in Current Debates and a Pragmatic Perspective on Understanding

Think or Remember? Detecting and Directing LLMs Towards Memorization or Generalization

Misinforming LLMs: vulnerabilities, challenges and opportunities

Stochastic LLMs do not Understand Language: Towards Symbolic, Explainable and Ontologically Based LLMs

Do large language models "understand" their knowledge?

Language models and psychological sciences

Undesirable Memorization in Large Language Models: A Survey

Embers of Autoregression: Understanding Large Language Models Through the Problem They are Trained to Solve

Large Language Models with Controllable Working Memory