Abstract:Large language models (LLMs) have achieved significant success in reasoning tasks, including mathematical reasoning and logical deduction. Among these reasoning tasks, graph problems stand out due to their complexity and unique structural characteristics, attracting considerable attention from researchers. Previous studies have explored LLMs' graph reasoning abilities through various techniques, such as different encoding methods for graph structures and the use of carefully designed prompts. However, a critical factor has been mostly overlooked: the prompt sequential order in which graph descriptions are presented to the models. In this study, we present the first comprehensive analysis of how the order of graph descriptions impacts LLM performance. Specifically, we comprehensively evaluate four graph description orders across six graph problems using six mainstream LLMs. The results reveal that: (1) ordered graph descriptions significantly improve LLMs' comprehension of graph structures; (2) the robustness of LLMs to graph description order varies across different tasks; and (3) the impact of graph order on performance is closely related to the inherent characteristics of tasks. This study provides a critical advancement in the application of LLMs for solving graph-related problems, paving the way for future research to optimize model performance through strategic graph description ordering.

What problem does this paper attempt to address?

The problem that this paper attempts to solve is: whether the graph description order will affect the performance of large language models (LLMs) in solving graph problems. Specifically, the authors explored the impact of graph descriptions in different orders on LLMs' understanding of graph structures and their problem - solving abilities. ### Research Background In recent years, large language models (LLMs) have achieved remarkable success in reasoning tasks, including mathematical reasoning and logical deduction. Among them, graph problems have attracted wide attention from researchers due to their complexity and unique structural characteristics. Although previous studies have explored the graph - reasoning abilities of LLMs through different encoding methods and carefully designed prompts, a key factor has been overlooked: the order in which graph descriptions are presented to the model. ### Research Questions This study comprehensively analyzed the impact of graph description order on LLMs' performance for the first time. Specifically, the authors evaluated the performance of four graph description orders on six graph problems and used six mainstream LLMs for experiments. The main findings of the study are as follows: 1. **An ordered graph description significantly improves LLMs' understanding of graph structures**. 2. **In different tasks, the robustness of LLMs to graph description order varies**. 3. **The impact of graph order on performance is closely related to the intrinsic characteristics of the task**. ### Experimental Setup To verify these hypotheses, the authors designed four graph description orders: - **Random Order**: Edge lists are randomly arranged. - **Breadth - First - Search Order (BFS Order)**: Traverse the graph layer by layer starting from a random root node. - **Depth - First - Search Order (DFS Order)**: Start from a node and traverse the graph as deeply as possible. - **PageRank Order (PR Order)**: Sort according to the PageRank scores of nodes. - **Personalized PageRank Order (PPR Order)**: Introduce a personalized vector mechanism and give priority to paths close to specific target nodes. In addition, the authors also designed six graph tasks, covering different levels of complexity: - **Connectivity Detection** - **Cycle Detection** - **Hamilton Path** - **Shortest Path** - **Topological Sort** - **Node Classification** ### Main Contributions 1. **Prove for the first time the significant impact of graph description order on LLMs' graph - reasoning performance**. 2. **Through extensive experiments, analyze the differential impact of different description orders on LLMs' performance**. 3. **Introduce a new dataset, GraphDO, aiming to promote the community's understanding of how graph description order affects LLMs' reasoning**. ### Conclusion The research results show that an ordered graph description can significantly improve the performance of LLMs in solving graph problems, especially in tasks requiring global understanding. However, for local tasks such as node classification, this impact is relatively small. This provides a new idea for optimizing LLMs' performance in the future, that is, to improve the model's performance by strategically adjusting the graph description order.

Can Graph Descriptive Order Affect Solving Graph Problems with LLMs?

Concise and Organized Perception Facilitates Large Language Models for Deductive Reasoning.

Can LLMs perform structured graph reasoning?

Can Language Models Solve Graph Problems in Natural Language?

Revisiting the Graph Reasoning Ability of Large Language Models: Case Studies in Translation, Connectivity and Shortest Path

GraphLLM: Boosting Graph Reasoning Ability of Large Language Model

GraphReason: Enhancing Reasoning Capabilities of Large Language Models through A Graph-Based Verification Approach

Can LLM Graph Reasoning Generalize beyond Pattern Memorization?

Structure Guided Prompt: Instructing Large Language Model in Multi-Step Reasoning by Exploring Graph Structure of the Text

Are Large-Language Models Graph Algorithmic Reasoners?

Graph Reasoning with Large Language Models via Pseudo-code Prompting

GraphInsight: Unlocking Insights in Large Language Models for Graph Structure Understanding

Can LLMs Effectively Leverage Graph Structural Information through Prompts, and Why?

GraphInstruct: Empowering Large Language Models with Graph Understanding and Reasoning Capability

Premise Order Matters in Reasoning with Large Language Models

Enhancing Logical Reasoning in Large Language Models through Graph-based Synthetic Data

Graph-enhanced Large Language Models in Asynchronous Plan Reasoning

Reasoning on Graphs: Faithful and Interpretable Large Language Model Reasoning

Can Large Language Models Analyze Graphs like Professionals? A Benchmark, Datasets and Models

CodeGraph: Enhancing Graph Reasoning of LLMs with Code

Evaluating Large Language Models on Graphs: Performance Insights and Comparative Analysis