Abstract:This paper focuses on the task of hallucination detection, which aims to determine the truthfulness of LLM-generated statements. To address this problem, a popular class of methods utilize the LLM's self-consistencies in its beliefs in a set of logically related augmented statements generated by the LLM, which does not require external knowledge databases and can work with both white-box and black-box LLMs. However, in many existing approaches, the augmented statements tend to be very monotone and unstructured, which makes it difficult to integrate meaningful information from the LLM beliefs in these statements. Also, many methods work with the binarized version of the LLM's belief, instead of the continuous version, which significantly loses information. To overcome these limitations, in this paper, we propose Belief Tree Propagation (BTProp), a probabilistic framework for LLM hallucination detection. BTProp introduces a belief tree of logically related statements by recursively decomposing a parent statement into child statements with three decomposition strategies, and builds a hidden Markov tree model to integrate the LLM's belief scores in these statements in a principled way. Experiment results show that our method improves baselines by 3%-9% (evaluated by AUROC and AUC-PR) on multiple hallucination detection benchmarks. Code is available at <a class="link-external link-https" href="https://github.com/UCSB-NLP-Chang/BTProp" rel="external noopener nofollow">this https URL</a>.

Bagging likelihood-based belief decision trees

A Classification Tree Method Based on Belief Entropy for Evidential Data

An Improved Belief Entropy and Its Application in Decision-Making

Evidential Decision Tree Based on Belief Entropy

Online active learning of decision trees with evidential data

Learning Belief Regression Trees From Evidential Data

Belief Entropy Tree and Random Forest: Learning from Data with Continuous Attributes and Evidential Labels

A Modified Belief Entropy in Dempster-Shafer Framework

Belief measures conditioned on random set

A Method of Inducing Ordered Basic Belief Assignment and Its Application in Risk Assessment of Tailing Dam

A new random forest method based on belief decision trees and its application in intention estimation

MTForest: Ensemble Decision Trees based on Multi-Task Learning

Bagged Voting Ensembles

DPT: An importance-based decision probability transformation method for uncertain belief in evidence theory

BoostTree and BoostForest for Ensemble Learning

Causal Inference Based Single-branch Ensemble Trees For Uplift Modeling

Generalizing Tree Probability Estimation Via Bayesian Networks

Improved Information Gain Estimates for Decision Tree Induction

An Improved Belief Entropy in Evidence Theory

A Probabilistic Framework for LLM Hallucination Detection via Belief Tree Propagation

MOB-ESP and other Improvements in Probability Estimation