Abstract:In this paper, we aim to develop a large language model (LLM) with the reasoning ability on complex graph data. Currently, LLMs have achieved very impressive performance on various natural language learning tasks, extensions of which have also been applied to study the vision tasks with multi-modal data. However, when it comes to the graph learning tasks, existing LLMs present very serious flaws due to their several inherited weaknesses in performing {multi-step logic reasoning}, {precise mathematical calculation} and {perception about the spatial and temporal factors}. To address such challenges, in this paper, we will investigate the principles, methodologies and algorithms to empower existing LLMs with graph reasoning ability, which will have tremendous impacts on the current research of both LLMs and graph learning. Inspired by the latest ChatGPT and Toolformer models, we propose the Graph-ToolFormer (Graph Reasoning oriented Toolformer) framework to teach LLMs themselves with prompts augmented by ChatGPT to use external graph reasoning API tools. Specifically, we will investigate to teach Graph-ToolFormer to handle various graph data reasoning tasks in this paper, including both (1) very basic graph data loading and graph property reasoning tasks, ranging from simple graph order and size to the graph diameter and periphery, and (2) more advanced reasoning tasks on real-world graph data, such as bibliographic networks, protein molecules, sequential recommender systems, social networks and knowledge graphs.

What problem does this paper attempt to address?

The problem that this paper attempts to solve is: how to endow large - language models (LLMs) with the reasoning ability to handle complex graph data. Currently, although LLMs perform excellently in natural - language processing and multi - modal visual tasks, they have significant deficiencies in graph - learning tasks, mainly reflected in the following aspects: 1. **Lack of precise mathematical computing ability**: Unable to perform precise mathematical operations. 2. **Difficulty in multi - step logical reasoning**: It is difficult to deal with problems that require multi - step logical reasoning. 3. **Lack of understanding of space and topology**: Lack of understanding of spatial and topological factors. 4. **Limited ability to process time - dynamic data**: Unable to handle data that changes over time well. To solve these problems, the paper proposes a new framework named Graph - ToolFormer, aiming to enhance the graph - reasoning ability of existing LLMs through prompt augmentation and the use of external graph - reasoning tools. Specifically, the main objectives of the Graph - ToolFormer framework include: - **Endowing LLMs with graph - reasoning ability**: Enabling LLMs to handle various tasks from simple graph - property reasoning to complex real - world graph - data reasoning, such as literature citation networks, protein molecular graphs, recommendation systems, online social networks, and knowledge graphs. - **Creating a graph - reasoning prompt dataset**: Automatically generate a large amount of prompt data containing external graph - reasoning API calls through ChatGPT, and perform selective filtering and post - processing on it. - **Extensive experimental verification**: Verify the effectiveness of Graph - ToolFormer through multiple real - world graph - reasoning tasks. ### Main contributions of the paper 1. **LLM with graph - reasoning ability**: This is the first paper to propose a general LLM (i.e., Graph - ToolFormer) to handle graph - reasoning tasks, effectively弥补了现有LLMs在图推理方面的不足. 2. **Graph - reasoning prompt dataset**: Created a graph - reasoning prompt dataset consisting of a small number of human - written instructions and prompt examples, and used ChatGPT for annotation and expansion. 3. **Extensive experimental research**: Conducted extensive tests on a variety of practical application tasks, including basic graph - property calculation tasks and some more challenging high - level graph - reasoning tasks, such as literature - topic inference, molecular - graph - function prediction, online - social - network - community detection, personalized recommendation, and knowledge - graph - entity - relationship reasoning. Through these efforts, Graph - ToolFormer not only improves the performance of LLMs in graph - reasoning tasks but also provides new tools and methods for the graph - learning community to better integrate into the current development process of artificial general intelligence (AGI).

Graph-ToolFormer: To Empower LLMs with Graph Reasoning Ability via Prompt Augmented by ChatGPT

GraphText: Graph Reasoning in Text Space

GraphLLM: Boosting Graph Reasoning Ability of Large Language Model

GraphInstruct: Empowering Large Language Models with Graph Understanding and Reasoning Capability

Think-on-Graph: Deep and Responsible Reasoning of Large Language Model on Knowledge Graph

GraphReason: Enhancing Reasoning Capabilities of Large Language Models through A Graph-Based Verification Approach

Graph Chain-of-Thought: Augmenting Large Language Models by Reasoning on Graphs

CodeGraph: Enhancing Graph Reasoning of LLMs with Code

GraphTool-Instruction: Revolutionizing Graph Reasoning in LLMs through Decomposed Subtask Instruction

Can Language Models Solve Graph Problems in Natural Language?

Think-on-Graph: Deep and Responsible Reasoning of Large Language Model with Knowledge Graph

Can LLMs perform structured graph reasoning?

GPT4Graph: Can Large Language Models Understand Graph Structured Data ? an Empirical Evaluation and Benchmarking.

ChatCoT: Tool-Augmented Chain-of-Thought Reasoning on Chat-based Large Language Models

Large Language Models on Graphs: A Comprehensive Survey

VisionGraph: Leveraging Large Multimodal Models for Graph Theory Problems in Visual Context

Ask and You Shall Receive (a Graph Drawing): Testing ChatGPT's Potential to Apply Graph Layout Algorithms

Graph of Thoughts: Solving Elaborate Problems with Large Language Models

Can LLM Graph Reasoning Generalize beyond Pattern Memorization?