GraphText: Graph Reasoning in Text Space

Jianan Zhao,Le Zhuo,Yikang Shen,Meng Qu,Kai Liu,Michael Bronstein,Zhaocheng Zhu,Jian Tang

2023-10-02

Abstract:Large Language Models (LLMs) have gained the ability to assimilate human knowledge and facilitate natural language interactions with both humans and other LLMs. However, despite their impressive achievements, LLMs have not made significant advancements in the realm of graph machine learning. This limitation arises because graphs encapsulate distinct relational data, making it challenging to transform them into natural language that LLMs understand. In this paper, we bridge this gap with a novel framework, GraphText, that translates graphs into natural language. GraphText derives a graph-syntax tree for each graph that encapsulates both the node attributes and inter-node relationships. Traversal of the tree yields a graph text sequence, which is then processed by an LLM to treat graph tasks as text generation tasks. Notably, GraphText offers multiple advantages. It introduces training-free graph reasoning: even without training on graph data, GraphText with ChatGPT can achieve on par with, or even surpassing, the performance of supervised-trained graph neural networks through in-context learning (ICL). Furthermore, GraphText paves the way for interactive graph reasoning, allowing both humans and LLMs to communicate with the model seamlessly using natural language. These capabilities underscore the vast, yet-to-be-explored potential of LLMs in the domain of graph machine learning.

Computation and Language,Machine Learning

What problem does this paper attempt to address?

The main problem this paper attempts to address is the limitations of large language models (LLMs) in handling graph-structured data. Although LLMs have achieved significant success in natural language processing, their progress in the field of graph machine learning has been limited. The primary reason is that graph-structured data contains unique relational information that is difficult to directly convert into a natural language form that LLMs can understand. To bridge this gap, the paper proposes a new framework—GRAPH TEXT, which converts graph data into natural language.

Specifically, GRAPH TEXT constructs a graph-syntax tree that includes not only node attributes but also the relationships between nodes. Traversing this tree generates a graph text sequence, which is then processed by the LLM, thereby transforming graph tasks into text generation tasks. This approach has the following advantages:

1. **Training-free Graph Inference**: Even without training on graph data, GRAPH TEXT combined with LLMs like ChatGPT can achieve or exceed the performance of supervised graph neural networks (GNNs) through in-context learning (ICL).
2. **Interactive Graph Inference**: GRAPH TEXT allows seamless interaction between humans and LLMs through natural language, enhancing the flexibility and interpretability of graph inference.

These capabilities demonstrate the enormous potential of LLMs in the field of graph machine learning, especially when dealing with different types of graph data.