Abstract:Large models have emerged as the most recent groundbreaking achievements in artificial intelligence, and particularly machine learning. However, when it comes to graphs, large models have not achieved the same level of success as in other fields, such as natural language processing and computer vision. In order to promote applying large models for graphs forward, we present a perspective paper to discuss the challenges and opportunities associated with developing large graph models. First, we discuss the desired characteristics of large graph models. Then, we present detailed discussions from three key perspectives: representation basis, graph data, and graph models. In each category, we provide a brief overview of recent advances and highlight the remaining challenges together with our visions. Finally, we discuss valuable applications of large graph models. We believe this perspective can encourage further investigations into large graph models, ultimately pushing us one step closer towards artificial general intelligence (AGI). We are the first to comprehensively study large graph models, to the best of our knowledge.

What problem does this paper attempt to address?

The problems that this paper attempts to solve are: the challenges and opportunities faced when applying large - scale models (such as large language models LLMs) on graph - structured data. Although large - scale models have achieved remarkable success in fields such as natural language processing (NLP) and computer vision (CV), their application on graph data is relatively lagging. The paper explores how to overcome these obstacles by constructing large - scale graph models and discusses the potential application value of large - scale graph models in different fields, aiming to promote the development of the graph learning field and ultimately contribute to the progress towards artificial general intelligence (AGI). Specifically, the paper explores the key issues of large - scale graph models from the following aspects: 1. **Desired Characteristics of Large - Scale Graph Models**: - **Graph Models with Scaling Laws**: Similar to large - scale language models, the performance of large - scale graph models continuously improves as the model size, data set size, and training computation increase. - **Graph Foundation Models**: They can handle different graph tasks across multiple domains and have the ability to understand graph - structure information and attributes. - **Graph Understanding and Processing Capabilities in Context**: They can understand and process new graph data sets and tasks at the test stage, requiring only a small number of samples and without the need for extensive modification of the model. - **Diverse Graph Reasoning Abilities**: They can handle tasks involving local and global graph properties, such as node degree, shortest path, centrality, etc. 2. **Graph Representation Foundation**: - **Graph Domains and Transferability**: It explores the differences and commonalities between different graph domains and how to find a unified representation foundation suitable for multiple graph domains. - **Alignment with Natural Language**: It studies how to enhance the interactivity and instruction - execution ability of graph models through the alignment of natural language and graph data. 3. **Graph Data**: - It emphasizes the importance of high - quality, large - scale graph data sets and points out that the current scale of graph data sets is far smaller than that of data sets in the NLP and CV fields. - It proposes principles for collecting more graph data, including domain diversity, type diversity, statistical diversity, task diversity, and modality diversity. 4. **Graph Models**: - **Backbone Architectures**: It discusses two main graph model architectures, graph neural networks (GNNs) and graph Transformers, and analyzes their advantages, disadvantages, and future development directions. - **Pre - training**: It emphasizes the importance of pre - training in large - scale graph models and introduces pre - training strategies such as contrastive learning and generation / prediction methods. - **Post - processing**: It explores how to enhance the downstream task adaptability of pre - trained models through techniques such as prompting, parameter - efficient fine - tuning, reinforcement learning, and model compression. Overall, the paper aims to comprehensively analyze the development status and future direction of large - scale graph models and provide theoretical and practical guidance for researchers.

Graph Meets LLMs: Towards Large Graph Models

Large Graph Models: A Perspective

Large Language Models on Graphs: A Comprehensive Survey

Large Language Models and Knowledge Graphs: Opportunities and Challenges

A Survey of Graph Meets Large Language Model: Progress and Future Directions

Can Large Language Models Analyze Graphs like Professionals? A Benchmark, Datasets and Models

Large Language Models Meet Graph Neural Networks: A Perspective of Graph Mining

Exploring the Potential of Large Language Models in Graph Generation

GPT4Graph: Can Large Language Models Understand Graph Structured Data ? an Empirical Evaluation and Benchmarking.

GraphLLM: Boosting Graph Reasoning Ability of Large Language Model

Exploring the Potential of Large Language Models (LLMs) in Learning on Graphs

Graph2text or Graph2token: A Perspective of Large Language Models for Graph Learning

Graph Machine Learning in the Era of Large Language Models (LLMs)

A Survey of Large Language Models on Generative Graph Analytics: Query, Learning, and Applications

Integrating Graphs With Large Language Models: Methods and Prospects

A Survey of Large Language Models for Graphs

Advancing Graph Representation Learning with Large Language Models: A Comprehensive Survey of Techniques

Evaluating Large Language Models on Graphs: Performance Insights and Comparative Analysis

Large Language Model with Graph Convolution for Recommendation