Reasoning-Enhanced Healthcare Predictions with Knowledge Graph Community Retrieval

Pengcheng Jiang,Cao Xiao,Minhao Jiang,Parminder Bhatia,Taha Kass-Hout,Jimeng Sun,Jiawei Han

2024-10-07

Abstract:Large language models (LLMs) have demonstrated significant potential in clinical decision support. Yet LLMs still suffer from hallucinations and lack fine-grained contextual medical knowledge, limiting their high-stake healthcare applications such as clinical diagnosis. Traditional retrieval-augmented generation (RAG) methods attempt to address these limitations but frequently retrieve sparse or irrelevant information, undermining prediction accuracy. We introduce KARE, a novel framework that integrates knowledge graph (KG) community-level retrieval with LLM reasoning to enhance healthcare predictions. KARE constructs a comprehensive multi-source KG by integrating biomedical databases, clinical literature, and LLM-generated insights, and organizes it using hierarchical graph community detection and summarization for precise and contextually relevant information retrieval. Our key innovations include: (1) a dense medical knowledge structuring approach enabling accurate retrieval of relevant information; (2) a dynamic knowledge retrieval mechanism that enriches patient contexts with focused, multi-faceted medical insights; and (3) a reasoning-enhanced prediction framework that leverages these enriched contexts to produce both accurate and interpretable clinical predictions. Extensive experiments demonstrate that KARE outperforms leading models by up to 10.8-15.0% on MIMIC-III and 12.6-12.7% on MIMIC-IV for mortality and readmission predictions. In addition to its impressive prediction accuracy, our framework leverages the reasoning capabilities of LLMs, enhancing the trustworthiness of clinical predictions.

Computation and Language

What problem does this paper attempt to address?

The paper aims to address the limitations of large language models (LLMs) in clinical decision support, particularly the issues of hallucinations and lack of fine-grained contextual medical knowledge in high-risk medical applications such as clinical diagnosis. Traditional retrieval-augmented generation (RAG) methods, while attempting to solve these problems, often retrieve sparse or irrelevant information, thereby affecting prediction accuracy.

To address the above issues, the authors propose the KARE framework, a novel approach that combines knowledge graph community-level retrieval with LLM reasoning capabilities to enhance the accuracy of medical predictions. KARE constructs a comprehensive multi-source knowledge graph by integrating biomedical databases, clinical literature, and insights generated by LLMs, and organizes this information using hierarchical graph community detection and summarization techniques for precise and contextually relevant information retrieval.

Specifically, the main innovations of KARE include:
1. **Dense Medical Knowledge Structuring Method**: Capable of accurately retrieving relevant information.
2. **Dynamic Knowledge Retrieval Mechanism**: Provides targeted multi-faceted medical insights based on the specific condition of the patient.
3. **Inference-Enhanced Prediction Framework**: Utilizes this rich contextual information to generate clinical predictions that are both accurate and interpretable.

Experimental results show that KARE significantly outperforms existing models on mortality and readmission prediction tasks on the MIMIC-III and MIMIC-IV datasets, improving by 10.8%-15.0% and 12.6%-12.7%, respectively. Additionally, the framework leverages the reasoning capabilities of LLMs, enhancing the credibility of clinical predictions.