Abstract:Graphs play an important role in representing complex relationships in various domains like social networks, knowledge graphs, and molecular discovery. With the advent of deep learning, Graph Neural Networks (GNNs) have emerged as a cornerstone in Graph Machine Learning (Graph ML), facilitating the representation and processing of graph structures. Recently, LLMs have demonstrated unprecedented capabilities in language tasks and are widely adopted in a variety of applications such as computer vision and recommender systems. This remarkable success has also attracted interest in applying LLMs to the graph domain. Increasing efforts have been made to explore the potential of LLMs in advancing Graph ML's generalization, transferability, and few-shot learning ability. Meanwhile, graphs, especially knowledge graphs, are rich in reliable factual knowledge, which can be utilized to enhance the reasoning capabilities of LLMs and potentially alleviate their limitations such as hallucinations and the lack of explainability. Given the rapid progress of this research direction, a systematic review summarizing the latest advancements for Graph ML in the era of LLMs is necessary to provide an in-depth understanding to researchers and practitioners. Therefore, in this survey, we first review the recent developments in Graph ML. We then explore how LLMs can be utilized to enhance the quality of graph features, alleviate the reliance on labeled data, and address challenges such as graph heterogeneity and out-of-distribution (OOD) generalization. Afterward, we delve into how graphs can enhance LLMs, highlighting their abilities to enhance LLM pre-training and inference. Furthermore, we investigate various applications and discuss the potential future directions in this promising field.

Integrating Graphs With Large Language Models: Methods and Prospects

Large Language Models on Graphs: A Comprehensive Survey

GPT4Graph: Can Large Language Models Understand Graph Structured Data ? an Empirical Evaluation and Benchmarking.

A Survey of Large Language Models for Graphs

Advancing Graph Representation Learning with Large Language Models: A Comprehensive Survey of Techniques

A Survey of Graph Meets Large Language Model: Progress and Future Directions

A Survey of Large Language Models on Generative Graph Analytics: Query, Learning, and Applications

Can Large Language Models Analyze Graphs like Professionals? A Benchmark, Datasets and Models

Graph Machine Learning in the Era of Large Language Models (LLMs)

Evaluating Large Language Models on Graphs: Performance Insights and Comparative Analysis

Large Language Models and Knowledge Graphs: Opportunities and Challenges

Exploring the Potential of Large Language Models (LLMs) in Learning on Graphs

Towards Versatile Graph Learning Approach: from the Perspective of Large Language Models

Exploring the Potential of Large Language Models in Graph Generation

GUNDAM: Aligning Large Language Models with Graph Understanding

Combining Knowledge Graphs and Large Language Models

Graph Learning and Its Advancements on Large Language Models: A Holistic Survey

Graph Meets LLMs: Towards Large Graph Models

GraphLLM: Boosting Graph Reasoning Ability of Large Language Model