Abstract:Large Generative Models (LGMs) such as GPT, Stable Diffusion, Sora, and Suno are trained on a huge amount of language corpus, images, videos, and audio that are extremely diverse from numerous domains. This training paradigm over diverse well-curated data lies at the heart of generating creative and sensible content. However, all previous graph generative models (e.g., GraphRNN, MDVAE, MoFlow, GDSS, and DiGress) have been trained only on one dataset each time, which cannot replicate the revolutionary success achieved by LGMs in other fields. To remedy this crucial gap, we propose a new class of graph generative model called Large Graph Generative Model (LGGM) that is trained on a large corpus of graphs (over 5000 graphs) from 13 different domains. We empirically demonstrate that the pre-trained LGGM has superior zero-shot generative capability to existing graph generative models. Furthermore, our pre-trained LGGM can be easily fine-tuned with graphs from target domains and demonstrate even better performance than those directly trained from scratch, behaving as a solid starting point for real-world customization. Inspired by Stable Diffusion, we further equip LGGM with the capability to generate graphs given text prompts (Text-to-Graph), such as the description of the network name and domain (i.e., "The power-1138-bus graph represents a network of buses in a power distribution system."), and network statistics (i.e., "The graph has a low average degree, suitable for modeling social media interactions."). This Text-to-Graph capability integrates the extensive world knowledge in the underlying language model, offering users fine-grained control of the generated graphs. We release the code, the model checkpoint, and the datasets at <a class="link-external link-https" href="https://lggm-lg.github.io/" rel="external noopener nofollow">this https URL</a>.

Large-scale graph representation learning with very deep GNNs and self-supervision

DiscoGNN: A Sample-Efficient Framework for Self-Supervised Graph Representation Learning

NGAT: Attention in Breadth and Depth Exploration for Semi-Supervised Graph Representation Learning

OGB-LSC: A Large-Scale Challenge for Machine Learning on Graphs

A Comprehensive Study on Large-Scale Graph Training: Benchmarking and Rethinking

On Size-Oriented Long-Tailed Graph Classification of Graph Neural Networks

GraphScale: A Framework to Enable Machine Learning over Billion-node Graphs

Large Generative Graph Models

LazyGNN: Large-Scale Graph Neural Networks via Lazy Propagation

Scalable Graph Neural Networks Via Bidirectional Propagation.

DistDGL: Distributed Graph Neural Network Training for Billion-Scale Graphs

GraphGPT: Graph Instruction Tuning for Large Language Models

Graph Attention Multi-Layer Perceptron

Node Feature Extraction by Self-Supervised Multi-scale Neighborhood Prediction

A Simple and Scalable Graph Neural Network for Large Directed Graphs

IGB: Addressing The Gaps In Labeling, Features, Heterogeneity, and Size of Public Graph Datasets for Deep Learning Research

Scalable and Efficient Full-Graph GNN Training for Large Graphs

LiGNN: Graph Neural Networks at LinkedIn

When transformer meets graph neural networks

Evaluating Deep Graph Neural Networks

On the Scalability of GNNs for Molecular Graphs