Abstract:Despite their competitive performance on knowledge-intensive tasks, large language models (LLMs) still have limitations in memorizing all world knowledge especially long tail knowledge. In this paper, we study the KG-augmented language model approach for solving the knowledge graph question answering (KGQA) task that requires rich world knowledge. Existing work has shown that retrieving KG knowledge to enhance LLMs prompting can significantly improve LLMs performance in KGQA. However, their approaches lack a well-formed verbalization of KG knowledge, i.e., they ignore the gap between KG representations and textual representations. To this end, we propose an answer-sensitive KG-to-Text approach that can transform KG knowledge into well-textualized statements most informative for KGQA. Based on this approach, we propose a KG-to-Text enhanced LLMs framework for solving the KGQA task. Experiments on several KGQA benchmarks show that the proposed KG-to-Text augmented LLMs approach outperforms previous KG-augmented LLMs approaches regarding answer accuracy and usefulness of knowledge statements.

What problem does this paper attempt to address?

The paper primarily addresses the limitations of large language models (LLMs) in the task of knowledge graph question answering (KGQA). Specifically, although LLMs perform excellently in many natural language processing tasks, they still struggle to remember all world knowledge, especially in the area of long-tail knowledge. This leads to poor performance in tasks that require rich factual knowledge. The paper points out that previous work attempted to enhance the capabilities of LLMs by introducing external knowledge, such as using knowledge graphs or web content. One approach is to construct knowledge-enhanced prompts containing fact information relevant to the question, thereby enriching the LLMs' knowledge in a more direct manner. However, these methods often overlook the importance of knowledge representation. To address these issues, the paper proposes a framework called Retrieve-Rewrite-Answer, which consists of three steps: 1. **Retrieve**: Retrieve subgraphs related to the question from the knowledge graph. 2. **Rewrite**: Convert the retrieved structured triples into natural language text descriptions. 3. **Answer**: Enhance the LLMs' reasoning based on the transformed knowledge text to generate answers. The core innovation of the paper lies in proposing a task-driven knowledge graph to text (KG-to-Text) method, which is used to transform the retrieved knowledge graph subgraphs into natural language descriptions that are most helpful for KGQA. Additionally, to overcome the lack of KG-to-Text annotated data in existing KGQA benchmark datasets, the researchers designed an automated KG-to-Text corpus generation method. Experimental results show that on multiple KGQA benchmark datasets, the proposed framework significantly improves performance on several large language models compared to previous KG-enhanced methods, demonstrating its effectiveness.

Retrieve-Rewrite-Answer: A KG-to-Text Enhanced LLMs Framework for Knowledge Graph Question Answering

Enhancing Large Language Models with Knowledge Graphs for Robust Question Answering

Retrieval and Reasoning on KGs: Integrate Knowledge Graphs into Large Language Models for Complex Question Answering

Enhancing Large Language Models with Pseudo- and Multisource- Knowledge Graphs for Open-ended Question Answering

CoTKR: Chain-of-Thought Enhanced Knowledge Rewriting for Complex Knowledge Graph Question Answering

Learning to Plan for Retrieval-Augmented Large Language Models from Knowledge Graphs

JellyBell at TextGraphs-17 Shared Task: Fusing Large Language Models with External Knowledge for Enhanced Question Answering

Structured Knowledge Grounding for Question Answering

EffiQA: Efficient Question-Answering with Strategic Multi-Model Collaboration on Knowledge Graphs

Knowledge Graph Question Answering with semantic oriented fusion model.

Prompting Large Language Models with Knowledge Graphs for Question Answering Involving Long-tail Facts

TransKGQA: Enhanced Knowledge Graph Question Answering With Sentence Transformers

Enhancing Question Answering for Enterprise Knowledge Bases using Large Language Models

keqing: knowledge-based question answering is a nature chain-of-thought mentor of LLM

MST5 -- Multilingual Question Answering over Knowledge Graphs

Interactive-KBQA: Multi-Turn Interactions for Knowledge Base Question Answering with Large Language Models

Leveraging LLMs in Scholarly Knowledge Graph Question Answering

Generate-on-Graph: Treat LLM as both Agent and KG in Incomplete Knowledge Graph Question Answering

ChatKBQA: A Generate-then-Retrieve Framework for Knowledge Base Question Answering with Fine-tuned Large Language Models

Leveraging Knowledge Graph Embeddings For Natural Language Question Answering