Abstract:Large language models (LLMs), such as GPT4 and LLaMA, are creating significant advancements in natural language processing, due to their strong text encoding/decoding ability and newly found emergent capability (e.g., reasoning). While LLMs are mainly designed to process pure texts, there are many real-world scenarios where text data is associated with rich structure information in the form of graphs (e.g., academic networks, and e-commerce networks) or scenarios where graph data is paired with rich textual information (e.g., molecules with descriptions). Besides, although LLMs have shown their pure text-based reasoning ability, it is underexplored whether such ability can be generalized to graphs (i.e., graph-based reasoning). In this paper, we provide a systematic review of scenarios and techniques related to large language models on graphs. We first summarize potential scenarios of adopting LLMs on graphs into three categories, namely pure graphs, text-attributed graphs, and text-paired graphs. We then discuss detailed techniques for utilizing LLMs on graphs, including LLM as Predictor, LLM as Encoder, and LLM as Aligner, and compare the advantages and disadvantages of different schools of models. Furthermore, we discuss the real-world applications of such methods and summarize open-source codes and benchmark datasets. Finally, we conclude with potential future research directions in this fast-growing field. The related source can be found at <a class="link-external link-https" href="https://github.com/PeterGriffinJin/Awesome-Language-Model-on-Graphs" rel="external noopener nofollow">this https URL</a>.

Large Language Models and Knowledge Graphs: Opportunities and Challenges

Large Knowledge Model: Perspectives and Challenges

Synergizing Knowledge Graphs with Large Language Models: A Comprehensive Review and Future Prospects

Combining Knowledge Graphs and Large Language Models

Research Trends for the Interplay between Large Language Models and Knowledge Graphs

Knowledge Engineering using Large Language Models

Accelerating Knowledge Graph and Ontology Engineering with Large Language Models

An Interdisciplinary Outlook on Large Language Models for Scientific Research

Knowledge Graphs, Large Language Models, and Hallucinations: An NLP Perspective

Large Language Models Can Better Understand Knowledge Graphs Than We Thought

Defining Knowledge: Bridging Epistemology and Large Language Models

Enhancing Knowledge Graph Construction Using Large Language Models

Knowledge Mechanisms in Large Language Models: A Survey and Perspective

A Principled Framework for Knowledge-enhanced Large Language Model

Large Language Models on Graphs: A Comprehensive Survey

Large Language Model Enhanced Knowledge Representation Learning: A Survey

Exploring the landscape of large language models: Foundations, techniques, and challenges

Combining large language models with enterprise knowledge graphs: a perspective on enhanced natural language understanding

Can Language Models Act as Knowledge Bases at Scale?