Abstract:Large language models have become integral to question-answering applications despite their propensity for generating hallucinations and factually inaccurate content. Querying knowledge graphs to reduce hallucinations in LLM meets the challenge of incomplete knowledge coverage in knowledge graphs. On the other hand, updating knowledge graphs by information extraction and knowledge graph completion faces the knowledge update misalignment issue. In this work, we introduce a collaborative augmentation framework, CogMG, leveraging knowledge graphs to address the limitations of LLMs in QA scenarios, explicitly targeting the problems of incomplete knowledge coverage and knowledge update misalignment. The LLMs identify and decompose required knowledge triples that are not present in the KG, enriching them and aligning updates with real-world demands. We demonstrate the efficacy of this approach through a supervised fine-tuned LLM within an agent framework, showing significant improvements in reducing hallucinations and enhancing factual accuracy in QA responses. Our code and video are publicly available.

What problem does this paper attempt to address?

The paper primarily addresses the issues encountered by large language models (LLMs) in question-answering (QA) applications and proposes a solution. Specifically, the paper focuses on two key challenges: 1. **Incomplete Knowledge Coverage**: Although knowledge graphs (KGs) theoretically can contain a vast amount of information, in practical applications, they often fail to comprehensively cover all the required knowledge, limiting their utility in QA scenarios. 2. **Misalignment in Knowledge Updates**: Current methods for updating knowledge graphs usually lack specificity, resulting in a mismatch between newly acquired knowledge and actual user needs. To address these issues, the paper proposes a collaborative enhancement framework named CogMG. This framework aims to improve the performance of LLMs in QA tasks through collaboration between LLMs and knowledge graphs. When a query exceeds the knowledge scope of the current knowledge graph, the LLM can explicitly decompose the required knowledge triples and supplement them based on the extensive knowledge within the model parameters. Additionally, the framework supports automatic verification or manual review of these newly added knowledge triples to ensure their accuracy, and ultimately incorporates them into the knowledge graph. This approach not only helps reduce hallucinated content and factual errors generated by LLMs but also gradually enhances the practical value of the knowledge graph. The main contributions of the paper include: - Proposing a collaborative enhancement framework named CogMG, which encourages proactive updates to the knowledge graph based on user needs. - Fine-tuning an open-source LLM within an agent framework to adapt to the CogMG collaborative enhancement paradigm, and demonstrating the application of this framework through the implementation of a website system. - Demonstrating the effectiveness of CogMG in proactively updating knowledge and improving QA response quality through case studies and experimental results in different contexts.

CogMG: Collaborative Augmentation Between Large Language Model and Knowledge Graph

Enhancing Large Language Models with Pseudo- and Multisource- Knowledge Graphs for Open-ended Question Answering

Enhancing Large Language Models with Knowledge Graphs for Robust Question Answering

Combining LLMs and Knowledge Graphs to Reduce Hallucinations in Question Answering

Harnessing Large Language Models for Knowledge Graph Question Answering via Adaptive Multi-Aspect Retrieval-Augmentation

Check Your Facts and Try Again: Improving Large Language Models with External Knowledge and Automated Feedback

Can Knowledge Graphs Make Large Language Models More Trustworthy? An Empirical Study over Open-ended Question Answering

KG-CoT: Chain-of-Thought Prompting of Large Language Models over Knowledge Graphs for Knowledge-Aware Question Answering

Mitigating Large Language Model Hallucinations via Autonomous Knowledge Graph-based Retrofitting

Generate-on-Graph: Treat LLM as both Agent and KG in Incomplete Knowledge Graph Question Answering

Retrieval and Reasoning on KGs: Integrate Knowledge Graphs into Large Language Models for Complex Question Answering

Addressing Hallucinations in Language Models with Knowledge Graph Embeddings as an Additional Modality

KG-EGV: A Framework for Question Answering with Integrated Knowledge Graphs and Large Language Models

Mitigating Hallucinations in Large Language Models via Self-Refinement-Enhanced Knowledge Retrieval

A Framework of Knowledge Graph-Enhanced Large Language Model Based on Question Decomposition and Atomic Retrieval

Learning to Plan for Retrieval-Augmented Large Language Models from Knowledge Graphs

Knowledge Graphs, Large Language Models, and Hallucinations: An NLP Perspective

Combining Knowledge Graphs and Large Language Models

GraphEval: A Knowledge-Graph Based LLM Hallucination Evaluation Framework