KG-Agent: An Efficient Autonomous Agent Framework for Complex Reasoning over Knowledge Graph

Jinhao Jiang,Kun Zhou,Wayne Xin Zhao,Yang Song,Chen Zhu,Hengshu Zhu,Ji-Rong Wen

2024-02-17

Abstract:In this paper, we aim to improve the reasoning ability of large language models (LLMs) over knowledge graphs (KGs) to answer complex questions. Inspired by existing methods that design the interaction strategy between LLMs and KG, we propose an autonomous LLM-based agent framework, called KG-Agent, which enables a small LLM to actively make decisions until finishing the reasoning process over KGs. In KG-Agent, we integrate the LLM, multifunctional toolbox, KG-based executor, and knowledge memory, and develop an iteration mechanism that autonomously selects the tool then updates the memory for reasoning over KG. To guarantee the effectiveness, we leverage program language to formulate the multi-hop reasoning process over the KG, and synthesize a code-based instruction dataset to fine-tune the base LLM. Extensive experiments demonstrate that only using 10K samples for tuning LLaMA-7B can outperform state-of-the-art methods using larger LLMs or more data, on both in-domain and out-domain datasets. Our code and data will be publicly released.

Computation and Language

What problem does this paper attempt to address?

This paper aims to enhance the reasoning capabilities of large language models (LLMs) on knowledge graphs (KGs) to answer complex questions. Specifically, the authors propose an autonomous LLM framework called KG-Agent, which enables a small LLM to make proactive decisions without human intervention until the entire reasoning process is completed. To achieve this goal, the paper integrates LLM, a multifunctional toolbox, a KG-based executor, and knowledge memory, and develops an iterative mechanism that can autonomously select tools and update memory for reasoning. The multi-hop reasoning process on KG is expressed through programming languages, and a code instruction dataset is synthesized to fine-tune the base LLM. Experimental results show that fine-tuning LLaMA-7B with only 10,000 samples outperforms existing methods that use larger LLMs or more data, with significant improvements on both in-domain and out-of-domain datasets. Moreover, this method does not rely on closed-source LLM APIs but instead utilizes a smaller open-source LLM to achieve autonomous reasoning.

KG-Agent: An Efficient Autonomous Agent Framework for Complex Reasoning over Knowledge Graph

Retrieval and Reasoning on KGs: Integrate Knowledge Graphs into Large Language Models for Complex Question Answering

A Framework of Knowledge Graph-Enhanced Large Language Model Based on Question Decomposition and Atomic Retrieval

Scalable and Accurate Graph Reasoning with LLM-based Multi-Agents

Evaluating and Enhancing Large Language Models for Conversational Reasoning on Knowledge Graphs

An Enhanced Prompt-Based LLM Reasoning Scheme via Knowledge Graph-Integrated Collaboration

Think-on-Graph: Deep and Responsible Reasoning of Large Language Model on Knowledge Graph

LLMs for Knowledge Graph Construction and Reasoning: Recent Capabilities and Future Opportunities

AGENTiGraph: An Interactive Knowledge Graph Platform for LLM-based Chatbots Utilizing Private Data

ODA: Observation-Driven Agent for integrating LLMs and Knowledge Graphs

KG-EGV: A Framework for Question Answering with Integrated Knowledge Graphs and Large Language Models

Thinking with Knowledge Graphs: Enhancing LLM Reasoning Through Structured Data

KnowledgeNavigator: Leveraging Large Language Models for Enhanced Reasoning over Knowledge Graph

Adversary and Attention Guided Knowledge Graph Reasoning Based on Reinforcement Learning

Reasoning on Efficient Knowledge Paths:Knowledge Graph Guides Large Language Model for Domain Question Answering

Textualized Agent-Style Reasoning for Complex Tasks by Multiple Round LLM Generation

Reasoning on Graphs: Faithful and Interpretable Large Language Model Reasoning

Complex Logical Reasoning over Knowledge Graphs using Large Language Models

Cooperative Reasoning on Knowledge Graph and Corpus: A Multi-agentReinforcement Learning Approach

KnowAgent: Knowledge-Augmented Planning for LLM-Based Agents