Abstract:Graphs play an important role in representing complex relationships in various domains like social networks, knowledge graphs, and molecular discovery. With the advent of deep learning, Graph Neural Networks (GNNs) have emerged as a cornerstone in Graph Machine Learning (Graph ML), facilitating the representation and processing of graph structures. Recently, LLMs have demonstrated unprecedented capabilities in language tasks and are widely adopted in a variety of applications such as computer vision and recommender systems. This remarkable success has also attracted interest in applying LLMs to the graph domain. Increasing efforts have been made to explore the potential of LLMs in advancing Graph ML's generalization, transferability, and few-shot learning ability. Meanwhile, graphs, especially knowledge graphs, are rich in reliable factual knowledge, which can be utilized to enhance the reasoning capabilities of LLMs and potentially alleviate their limitations such as hallucinations and the lack of explainability. Given the rapid progress of this research direction, a systematic review summarizing the latest advancements for Graph ML in the era of LLMs is necessary to provide an in-depth understanding to researchers and practitioners. Therefore, in this survey, we first review the recent developments in Graph ML. We then explore how LLMs can be utilized to enhance the quality of graph features, alleviate the reliance on labeled data, and address challenges such as graph heterogeneity and out-of-distribution (OOD) generalization. Afterward, we delve into how graphs can enhance LLMs, highlighting their abilities to enhance LLM pre-training and inference. Furthermore, we investigate various applications and discuss the potential future directions in this promising field.

What problem does this paper attempt to address?

The problems that this paper attempts to solve mainly focus on how to use large - language models (LLMs) to enhance the capabilities of graph machine learning (Graph ML), and explore how graph structures can strengthen the functions of LLMs. Specifically, the paper focuses on the following aspects: 1. **Improving the Quality of Graph Features**: By using LLMs to improve the quality of node features in graph data, thereby enhancing the performance of graph machine - learning models. 2. **Reducing Dependence on Labeled Data**: Explore how to use LLMs to reduce the dependence of traditional graph machine - learning methods on a large amount of labeled data, which is helpful for effective model training in the case of scarce data. 3. **Dealing with Graph Heterogeneity and Out - of - Distribution Generalization**: Research how to use LLMs to solve the heterogeneity problem in graph data and the generalization ability of models when facing unseen data. 4. **Enhancing the Reasoning Ability and Interpretability of LLMs**: By introducing graph structures, especially knowledge graphs, to enhance the reasoning ability of LLMs and reduce possible errors or inaccuracies when generating content. 5. **Promoting the Development of Graph Foundation Models**: Discuss how to combine LLMs and graph machine - learning techniques to promote the development of graph foundation models (GFMs), enabling them to better adapt to multiple downstream tasks. The paper also discusses the applications of these methods in multiple fields such as recommendation systems, knowledge graphs, scientific AI, and robot task planning, and looks forward to future research directions, including the universality and transferability of models, multi - modal graph learning, credibility and efficiency, etc.

Graph Machine Learning in the Era of Large Language Models (LLMs)

Graph Machine Learning in the Era of Large Language Models (LLMs)

A Survey of Graph Meets Large Language Model: Progress and Future Directions

A Survey of Large Language Models for Graphs

Large Language Models on Graphs: A Comprehensive Survey

A Survey of Large Language Models on Generative Graph Analytics: Query, Learning, and Applications

Exploring the Potential of Large Language Models (LLMs) in Learning on Graphs

Advancing Graph Representation Learning with Large Language Models: A Comprehensive Survey of Techniques

Integrating Graphs With Large Language Models: Methods and Prospects

GPT4Graph: Can Large Language Models Understand Graph Structured Data ? an Empirical Evaluation and Benchmarking.

Graph Learning and Its Advancements on Large Language Models: A Holistic Survey

Large Language Models and Knowledge Graphs: Opportunities and Challenges

Exploring the Potential of Large Language Models in Graph Generation

GraphLLM: Boosting Graph Reasoning Ability of Large Language Model

Enhancing Graph Neural Networks with Limited Labeled Data by Actively Distilling Knowledge from Large Language Models

Graph Meets LLMs: Towards Large Graph Models

Evaluating Large Language Models on Graphs: Performance Insights and Comparative Analysis

Large Language Model with Graph Convolution for Recommendation

Combining Knowledge Graphs and Large Language Models

Towards Versatile Graph Learning Approach: from the Perspective of Large Language Models