Abstract:Recently, Knowledge Graphs (KGs) have been successfully coupled with Large Language Models (LLMs) to mitigate their hallucinations and enhance their reasoning capability, such as in KG-based retrieval-augmented frameworks. However, current KG-LLM frameworks lack rigorous uncertainty estimation, limiting their reliable deployment in high-stakes applications. Directly incorporating uncertainty quantification into KG-LLM frameworks presents challenges due to their complex architectures and the intricate interactions between the knowledge graph and language model components. To address this gap, we propose a new trustworthy KG-LLM framework, Uncertainty Aware Knowledge-Graph Reasoning (UAG), which incorporates uncertainty quantification into the KG-LLM framework. We design an uncertainty-aware multi-step reasoning framework that leverages conformal prediction to provide a theoretical guarantee on the prediction set. To manage the error rate of the multi-step process, we additionally introduce an error rate control module to adjust the error rate within the individual components. Extensive experiments show that our proposed UAG can achieve any pre-defined coverage rate while reducing the prediction set/interval size by 40% on average over the baselines.

A New Approach to Reasoning About Uncertainty

A Quantitative Approach to Reasoning about Incomplete Knowledge.

Approach to Uncertain Reasoning Based on Cloud Model

A causal-based symbolic reasoning framework for uncertain knowledge graphs

A Study of Probability-Based Uncertain Reasoning Models in Rule-Based Expert Systems: Prospector-Type Schemes

An uncertain reasoning approach with applications to target identification and software detection

Modal reasoning for uncertain information in expert system

AN UNCERTAIN REASONING APPROACH WITH APPLICATIONS TO IMAGE RECOGNITION

The Fuzzy Reasoning Rules Based on Universal Logic.

A New Method to Compute Truth-value Propagation in Uncertainty Reasoning

A unified algebraic structure for uncertain reasonings

Integrated Model For Uncertain Topological And Directional Relation Reasoning

The Measure Of Uncertainty Of The Positive Conjunction Reasoning Rules

Towards Trustworthy Knowledge Graph Reasoning: An Uncertainty Aware Perspective

Knowledge representation and reasoning algorithm based on fuzzy rule

Uncertain Inference

Logic, Reasoning under Uncertainty and Causality

Uncertainty reasoning based on cloud models in controllers

Reasoning with Vagueness in Hybrid MKNF Knowledge Bases

A Multiple and Multi-Dimensional Reasoning Method of Linguistic-Valued Intuitionistic Fuzzy 2-Tuple

Soft Reasoning on Uncertain Knowledge Graphs